##// END OF EJS Templates
jro_apps » redmine
RSS
scm: git: performance improvements in fetching revisions (#8857, #9472)...
scm: git: performance improvements in fetching revisions (#8857, #9472) Parse a revision for a given branch, just if we haven't parsed it for any branches before. Moved the db check to for existing revisions into a grouped search. Search for many revisions at once: this reduces db load. Revisions are grouped into sets of 100. This is to improve memory consumption. There will be just one query instead of each 100. The above two methods significantly increase parsing speed. Test case was a git repo with 6000+ commits on a master branch, and several other branches originating for master. Speed improved from 1.4h to 18min. Contributed by Gergely Fábián. git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@9144 e93f8b46-1217-0410-a6f0-8f06a7374b81
Jean-Philippe Lang - - Load All Authors

File last commit:

r8895:ad2036aabaf4
r9024:999a4ba30d7b
Show More
cvs.rb
204 lines | 7.5 KiB | text/x-ruby | RubyLexer
/ app / models / repository / cvs.rb
History | Source | Raw |Copy content |Copy permalink
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 # Redmine - project management software
# Copyright (C) 2006-2011 Jean-Philippe Lang
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 #
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 #
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 #
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
require 'redmine/scm/adapters/cvs_adapter'
require 'digest/sha1'
class Repository::Cvs < Repository
Toshi MARUYAMA
scm: add feature of per project repository log encoding setting (#1735)....
 r4862 validates_presence_of :url, :root_url, :log_encoding
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556
Jean-Philippe Lang
human_attribute_name accepts optional argument....
 r8166 def self.human_attribute_name(attribute_key_name, *args)
Toshi MARUYAMA
Rails3: scm: cvs: use .to_s for overriding human_attribute_name parameter...
 r8852 attr_name = attribute_key_name.to_s
Toshi MARUYAMA
scm: cvs: use i18n string at 'CVSROOT' and 'Module' setting....
 r5415 if attr_name == "root_url"
attr_name = "cvsroot"
elsif attr_name == "url"
attr_name = "cvs_module"
end
Jean-Philippe Lang
human_attribute_name accepts optional argument....
 r8166 super(attr_name, *args)
Toshi MARUYAMA
scm: add scm specific human_attribute_name for input validation....
 r4855 end
Toshi MARUYAMA
scm: add scm command and version methods at repository models (#4273)....
 r4702 def self.scm_adapter_class
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 Redmine::Scm::Adapters::CvsAdapter
end
Toshi MARUYAMA
scm: add scm command and version methods at repository models (#4273)....
 r4702
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 def self.scm_name
'CVS'
end
Toshi MARUYAMA
scm: add scm command and version methods at repository models (#4273)....
 r4702
Jean-Philippe Lang
Fixed: view file at given revision with CVS....
 r1539 def entry(path=nil, identifier=nil)
rev = identifier.nil? ? nil : changesets.find_by_revision(identifier)
scm.entry(path, rev.nil? ? nil : rev.committed_on)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Toshi MARUYAMA
scm: cvs: model entries returns nil if revision is not stored in database....
 r5311
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 def entries(path=nil, identifier=nil)
Toshi MARUYAMA
scm: cvs: model entries returns nil if revision is not stored in database....
 r5311 rev = nil
if ! identifier.nil?
rev = changesets.find_by_revision(identifier)
return nil if rev.nil?
end
Jean-Philippe Lang
Fix repository browsing at given revision for various scm and add tests for this....
 r1314 entries = scm.entries(path, rev.nil? ? nil : rev.committed_on)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 if entries
entries.each() do |entry|
Toshi MARUYAMA
scm: cvs: fix missing author, revision and comment in tree view (#4270)....
 r4973 if ( ! entry.lastrev.nil? ) && ( ! entry.lastrev.revision.nil? )
Toshi MARUYAMA
scm: cvs: code clean up model....
 r4964 change=changes.find_by_revision_and_path(
entry.lastrev.revision,
scm.with_leading_slash(entry.path) )
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 if change
Toshi MARUYAMA
scm: cvs: code clean up model....
 r4964 entry.lastrev.identifier = change.changeset.revision
Toshi MARUYAMA
scm: cvs: fix missing author, revision and comment in tree view (#4270)....
 r4973 entry.lastrev.revision = change.changeset.revision
Toshi MARUYAMA
scm: cvs: code clean up model....
 r4964 entry.lastrev.author = change.changeset.committer
Toshi MARUYAMA
scm: cvs: fix missing author, revision and comment in tree view (#4270)....
 r4973 # entry.lastrev.branch = change.branch
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
end
end
end
entries
end
Toshi MARUYAMA
scm: cvs: override annotate() in model....
 r5290
Jean-Philippe Lang
Fixed: view file at given revision with CVS....
 r1539 def cat(path, identifier=nil)
Toshi MARUYAMA
scm: cvs: cat returns nil if revision does not exist at model....
 r5288 rev = nil
if ! identifier.nil?
rev = changesets.find_by_revision(identifier)
return nil if rev.nil?
end
Jean-Philippe Lang
Fixed: view file at given revision with CVS....
 r1539 scm.cat(path, rev.nil? ? nil : rev.committed_on)
end
Toshi MARUYAMA
scm: cvs: override annotate() in model....
 r5290
def annotate(path, identifier=nil)
rev = nil
if ! identifier.nil?
rev = changesets.find_by_revision(identifier)
return nil if rev.nil?
end
scm.annotate(path, rev.nil? ? nil : rev.committed_on)
end
Jean-Philippe Lang
Move unified diff parser out of the scm abstract adapter so it can be reused for viewing attached diffs (#1403)....
 r1499 def diff(path, rev, rev_to)
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5286 # convert rev to revision. CVS can't handle changesets here
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 diff=[]
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5286 changeset_from = changesets.find_by_revision(rev)
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 if rev_to.to_i > 0
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5286 changeset_to = changesets.find_by_revision(rev_to)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
changeset_from.changes.each() do |change_from|
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 revision_from = nil
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 revision_to = nil
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 if path.nil? || (change_from.path.starts_with? scm.with_leading_slash(path))
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5286 revision_from = change_from.revision
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 end
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 if revision_from
if changeset_to
changeset_to.changes.each() do |change_to|
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 revision_to = change_to.revision if change_to.path == change_from.path
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
end
unless revision_to
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 revision_to = scm.get_previous_revision(revision_from)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Jean-Philippe Lang
Fixed: unexpected nil when viewing differences on CVS (#1444)....
 r1511 file_diff = scm.diff(change_from.path, revision_from, revision_to)
diff = diff + file_diff unless file_diff.nil?
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
end
return diff
end
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 def fetch_changesets
# some nifty bits to introduce a commit-id with cvs
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 # natively cvs doesn't provide any kind of changesets,
# there is only a revision per file.
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # we now take a guess using the author, the commitlog and the commit-date.
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633
# last one is the next step to take. the commit-date is not equal for all
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # commits in one changeset. cvs update the commit-date when the *,v file was touched. so
# we use a small delta here, to merge all changes belonging to _one_ changeset
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 time_delta = 10.seconds
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 fetch_since = latest_changeset ? latest_changeset.committed_on : nil
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 transaction do
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 tmp_rev_num = 1
Toshi MARUYAMA
scm: cvs: convert author encoding with log encoding setting....
 r5336 scm.revisions('', fetch_since, nil, :log_encoding => repo_log_encoding) do |revision|
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # only add the change to the database, if it doen't exists. the cvs log
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 # is not exclusive at all.
Toshi MARUYAMA
scm: cvs: generate pseudo scmid for auto issue close text (#6706)....
 r4682 tmp_time = revision.time.clone
Toshi MARUYAMA
scm: cvs: code clean up....
 r4670 unless changes.find_by_path_and_revision(
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 scm.with_leading_slash(revision.paths[0][:path]),
revision.paths[0][:revision]
)
Toshi MARUYAMA
scm: ignore log encoding setting in Subversion and Mercurial (#7597)....
 r4842 cmt = Changeset.normalize_comments(revision.message, repo_log_encoding)
Toshi MARUYAMA
scm: cvs: fix parsing revisions if author is not ASCII....
 r5335 author_utf8 = Changeset.to_utf8(revision.author, repo_log_encoding)
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 cs = changesets.find(
:first,
:conditions => {
:committed_on => tmp_time - time_delta .. tmp_time + time_delta,
Toshi MARUYAMA
scm: cvs: fix parsing revisions if author is not ASCII....
 r5335 :committer => author_utf8,
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 :comments => cmt
}
)
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 # create a new changeset....
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 unless cs
# we use a temporaray revision number here (just for inserting)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 # later on, we calculate a continous positive number
Toshi MARUYAMA
scm: cvs: generate pseudo scmid for auto issue close text (#6706)....
 r4682 tmp_time2 = tmp_time.clone.gmtime
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 branch = revision.paths[0][:branch]
scmid = branch + "-" + tmp_time2.strftime("%Y%m%d-%H%M%S")
cs = Changeset.create(:repository => self,
:revision => "tmp#{tmp_rev_num}",
:scmid => scmid,
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633 :committer => revision.author,
Toshi MARUYAMA
scm: cvs: generate pseudo scmid for auto issue close text (#6706)....
 r4682 :committed_on => tmp_time,
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 :comments => revision.message)
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 tmp_rev_num += 1
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Toshi MARUYAMA
scm: cvs: code clean up model....
 r5330 # convert CVS-File-States to internal Action-abbrevations
# default action is (M)odified
action = "M"
if revision.paths[0][:action] == "Exp" && revision.paths[0][:revision] == "1.1"
action = "A" # add-action always at first revision (= 1.1)
elsif revision.paths[0][:action] == "dead"
action = "D" # dead-state is similar to Delete
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 Change.create(
:changeset => cs,
:action => action,
:path => scm.with_leading_slash(revision.paths[0][:path]),
:revision => revision.paths[0][:revision],
:branch => revision.paths[0][:branch]
)
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
end
Toshi MARUYAMA
scm: subversion: code clean up model....
 r5633
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 # Renumber new changesets in chronological order
Jean-Philippe Lang
Rails 3.1 compatibility (order declared on association takes precedence)....
 r8895 Changeset.all(
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256 :order => 'committed_on ASC, id ASC',
Jean-Philippe Lang
Rails 3.1 compatibility (order declared on association takes precedence)....
 r8895 :conditions => ["repository_id = ? AND revision LIKE 'tmp%'", id]
Toshi MARUYAMA
scm: cvs: code clean up....
 r4670 ).each do |changeset|
Jean-Philippe Lang
CVS duplicate key violation fix (#996, #1098)....
 r1340 changeset.update_attribute :revision, next_revision_number
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Jean-Philippe Lang
Merged Git support branch (r1200 to r1226)....
 r1222 end # transaction
Toshi MARUYAMA
scm: cvs: change temporary revision number from "_N" to "tmpN" (#996, #3761, #6706)....
 r4681 @current_revision_number = nil
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256
Jean-Philippe Lang
CVS duplicate key violation fix (#996, #1098)....
 r1340 private
Toshi MARUYAMA
scm: cvs: code clean up app/models/repository/cvs.rb....
 r5256
Jean-Philippe Lang
CVS duplicate key violation fix (#996, #1098)....
 r1340 # Returns the next revision number to assign to a CVS changeset
def next_revision_number
# Need to retrieve existing revision numbers to sort them as integers
Toshi MARUYAMA
scm: cvs: change temporary revision number from "_N" to "tmpN" (#996, #3761, #6706)....
 r4681 sql = "SELECT revision FROM #{Changeset.table_name} "
sql << "WHERE repository_id = #{id} AND revision NOT LIKE 'tmp%'"
@current_revision_number ||= (connection.select_values(sql).collect(&:to_i).max || 0)
Jean-Philippe Lang
CVS duplicate key violation fix (#996, #1098)....
 r1340 @current_revision_number += 1
end
Jean-Philippe Lang
Added basic support for CVS and Mercurial SCMs....
 r556 end