About the performance when showing the history of the selected resource

[fedejeanne]

I am looking at the performance when displaying the Git History of the currently selected resource:

This is quite slow in our case since our product has a huge history (lots of commits) and it takes several seconds to calculate it. Looking at this VisualVM Sample: classWithMultipleCommits.zip

I can see that:

1. It all starts with the first call to RevCommitList.fillTo(...) inside the GenerateHistoryJob

2. When TopoSortGenerator is instantiated, the underlying generator (which is the RewriteGenerator) is queried for all of its commits. The JavaDoc even says that explicitly:

Create a new sorter and completely spin the generator.

jgit/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/TopoSortGenerator.java

Lines 48 to 62 in 1ff3617

	for (;;) {
	final RevCommit c = s.next();
	if (c == null) {
	break;
	}
	for (RevCommit p : c.getParents()) {
	if (!p.has(RevFlag.UNSHALLOW)) {
	p.inDegree++;
	}
	if (firstParent) {
	break;
	}
	}
	pending.add(c);
	}

3. RewriteGenerator.applyFilterToParents ends up making lots of calls to the include(...) method. This is the filtering operation that gets me the commits of the current resource

jgit/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/PendingGenerator.java

Line 110 in 1ff3617

produce = filter.include(walker, c);

Questions

1. Is there any better way to filter the commits that belong to the currently selected resource? It seems to me that the current implementation is filtering from all commits in the repository
2. Why is the RewriteGenerator even needed? I don't quite get the reason from here:

(this is the one that ends up being packed into the RewriteGenerator)

jgit/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java

Line 117 in 1ff3617

g = new PendingGenerator(w, pending, rf, pendingOutputType);

(and here is the RewriteGenerator itself)

jgit/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/StartGenerator.java

Lines 128 to 130 in 1ff3617

	if ((g.outputType() & NEEDS_REWRITE) != 0) {
	g = new RewriteGenerator(g);
	}

3. Any other hints at what I could look into in order to improve performance?

Bonus

I also noticed that there is the notion of "pending" commits (look at org.eclipse.jgit.revwalk.RevCommitList.isPending() and org.eclipse.egit.ui.internal.history.GenerateHistoryJob.updateUI(boolean)) and there is the possibility to load commits lazily (which is what happens when I setup the Maximum number of commits to show under Version Control (Team) > Git > History in the preferences)

Preferences (Screenshot)

... but this only affects the way the commits are displayed in the commit list inside the GitHistoryPage, whic happens after the commits have been queried and filtered, so it does not help performance. The real bottleneck is having to go through the whole list of commits and filter them the very first time TopoSortGenerator is instantiated.