Home - Waterfall Grid T-Grid Console Builders Recent Builds Buildslaves Changesources - JSON API - About

Console View

Legend:   Passed Failed Warnings Failed Again Running Exception Offline No data

Pulkit Goyal
py3: fix test-narrow* which started failing because of recent changes

#skip-blame because just r'' prefix

Differential Revision: https://phab.mercurial-scm.org/D6447
Matt Harbison
manifest: add some documentation to _lazymanifest python code

It was not particularly easy figuring out the design of this class and keeping
track of how the pieces work.  So might as well write some of it down for the
next person.
Matt Harbison
tests: demonstrate broken manifest generation with the pure module

This will be fixed next.  But I don't fully understand how 'b.txt' is actually
removed properly in the second test, given what's broken.  Also, I'm not sure
why 'bb.txt' is flagged as not being in the manifest, when it clearly appears
to be.
Matt Harbison
manifest: avoid corruption by dropping removed files with pure (issue5801)

Previously, removed files would simply be marked by overwriting the first byte
with NUL and dropping their entry in `self.position`.  But no effort was made to
ignore them when compacting the dictionary into text form.  This allowed them to
slip into the manifest revision, since the code seems to be trying to minimize
the string operations by copying as large a chunk as possible.  As part of this,
compact() walks the existing text based on entries in the `positions` list, and
consumed everything up to the next position entry.  This typically resulted in
a ValueError complaining about unsorted manifest entries.

Sometimes it seems that files do get dropped in large repos- it seems to
correspond to there being a new entry that would take the same slot.  A much
more trivial problem is that if the only changes were removals, `_compact()`
didn't even run because `__delitem__` doesn't add anything to `self.extradata`.
Now there's an explicit variable to flag this, both to allow `_compact()` to
run, and to avoid searching the manifest in cases where there are no removals.

In practice, this behavior was mostly obscured by the check in fastdelta() which
takes a different path that explicitly drops removed files if there are fewer
than 1000 changes.  However, timeless has a repo where after rebasing tens of
commits, a totally different path[1] is taken that bypasses the change count
check and hits this problem.

[1] https://www.mercurial-scm.org/repo/hg/file/2338bdea4474/mercurial/manifest.py#l1511
Martin von Zweigbergk
context: add ctx.files{modified,added,removed}() methods

Changeset-centric copy tracing is currently very slow because it often
reads manifests. One place it needs the manifest is in _chain(), where
it removes a copy X->Y if Y has subsequently gotten removed. I want to
speed that up by keeping track directly in the changeset of which
files are removed in the changeset. These methods will be similar to
ctx.p[12]copies() in that way: they will either read from the
changeset or calculate the information from the manifests otherwise.

Note that these are different from ctx.{modified,added,removed}() on
merge commits. Those functions always compare to p1, but the new ones
compare to both parents. filesadded() means "file does not exist in
either parent but exists now", filesremoved() means "file existed in
either parent but does not exist now", and filesmodified() means "file
existed in either parent and still exists". The set of files in
ctx.files() is the union of the files from the three new functions
(and the three new ones are all disjoint sets).

Also note that uncommitted merges are weird as usual. The invariant
mentioned above still holds, but the functions compare to p1 (and are
thus identical to the existing methods).

Differential Revision: https://phab.mercurial-scm.org/D6367
Martin von Zweigbergk
tests: add test for {file_mods}, {file_adds}, {file_dels} on merge commit

Differential Revision: https://phab.mercurial-scm.org/D6368
Martin von Zweigbergk
relnotes: document changed behavior of ui.origbackuppath pointing to file

Differential Revision: https://phab.mercurial-scm.org/D6446
Martin von Zweigbergk
copies: split up _chain() in naive chaining and filtering steps

The function now has two clearly defined steps. The first step is the
actual chaining. This step is very cheap. The second step is filtering
out invalid copies. This step is expensive. For changeset-centric copy
tracing, I want to do the filtering step only at the end. This patch
prepares for that.

Differential Revision: https://phab.mercurial-scm.org/D6418
Martin von Zweigbergk
templatekw: move showfileadds() close to showfile{mods,dels}()

Differential Revision: https://phab.mercurial-scm.org/D6370
Pulkit Goyal
py3: use range() instead of xrange()

The latter does not exist on Python 3. This makes test-contrib-perf.t pass on
Python 3 again.

Differential Revision: https://phab.mercurial-scm.org/D6443
Pulkit Goyal
narrow: pass binary nodeids to generateellipsesbundle2()

We generally work with binary nodeids and it's should be expected that new
function gets the nodeids in binary form already.

Differential Revision: https://phab.mercurial-scm.org/D6444
Pulkit Goyal
narrow: move heads close to common as they are closely related

Differential Revision: https://phab.mercurial-scm.org/D6445
Pierre-Yves David
match: stabilize _rootsdirsandparents doctest

Changeset c4b8f8637d7a tried to stabilize some matcher test by using a set. This
did not work because the set order is not stable. To fix it, we post process the
result to display a sorted version of the set.
Pulkit Goyal
narrow: remove unrequired compat code for old versions of hg

As the comment says, that if is only required for servers having hg version 3.1
and 3.2. Any client connecting having hg 3.1 or 3.2 locally and trying to use
narrow should already be broken taking in account the changes which have been
done since narrow moved to core.

Differential Revision: https://phab.mercurial-scm.org/D6434
Pulkit Goyal
narrow: factor out logic to build ellipses related b2parts in separate fn

This will help us switch more cleanly to using wireprotocol commands instead of
using exchange.pull() which exchanges more things then required.

Differential Revision: https://phab.mercurial-scm.org/D6435
Pierre-Yves David
perf: make sure to explicitly disable any profiler after the first iteration

The current code work, because of some edge behavior of the `profile` class. We
make it explicit that the profiler is not in effect more than once.
Danny Hooper
test: add missing 'cd ..' to test case

Differential Revision: https://phab.mercurial-scm.org/D6439
Martin von Zweigbergk
match: remove an obsolete comment about util.finddirs()

Obsolete since 8e55c0c642c (util: make util.dirs() and util.finddirs()
include root directory (API), 2017-05-16).

Differential Revision: https://phab.mercurial-scm.org/D6433
Pierre-Yves David
perf: add an option to profile the benchmark section

Running a perf command with --profile gather data for the whole command
execution, including setup and cleanup. This can significantly alter the data.

To work around this we introduce a new option, it trigger the profiling of only one
iteration of the benchmarked section.
Pierre-Yves David
perf: add a `pre-run` option

sometimes, the initial run is necessary to warm some cache that are not relevant
for the current measurement. We add a new `perf.pre-run` option to specify a
number of run of the benchmark logic that will happens before measurement are
taken.
Martin von Zweigbergk
match: de-flake test-doctest.py by not depending on util.dirs() order

util.dirs() yields directories in arbitrary order, which has made
test-doctest.py flaky. I think they have been flaky since d8e55c0c642c
(util: make util.dirs() and util.finddirs() include root directory
(API), 2017-05-16). Before that commit, I think util.dirs() would
return at most one entry, so there was only one iteration order. This
patch fixes the problem by making _rootsdirsandparents() return a set
(whose __str__() is defined to be in sorted order, I believe). The
only caller wanted a set anyway.

Differential Revision: https://phab.mercurial-scm.org/D6432
Danny Hooper
narrow: consider empty commits to be "inside the narrow spec" for templates

It doesn't seem useful to exclude them, or harmful to include them. Users
writing log templates using outsidenarrow as a predicate might consider it
unexpected if their locally created empty drafts are treated as if they
contained something outside the clone.

Differential Revision: https://phab.mercurial-scm.org/D6414
Georges Racinet
rust-python3: useless python2 specific import

This python27_sys import prevents building with python3,
it had been previously removed in a5fa9140ce4c, but that
has been since pruned

Differential Revision: https://phab.mercurial-scm.org/D6415
Georges Racinet
rust-python3: compatibility fix for incoming PyLong

On Python3, PyInt is PyLong and it doesn't have the
`value()` method.
Re upcasting to PythonObj as done here works, but we
might prefer taking a PythonObj from the onset
(would require more testing)

Differential Revision: https://phab.mercurial-scm.org/D6397
Pulkit Goyal
py3: add one new passing test found by buildbot

Differential Revision: https://phab.mercurial-scm.org/D6412
Pierre-Yves David
discovery: slowly increase sampling size

Some pathological discovery runs can requires many roundtrip. When this happens
things can get very slow.

To make the algorithm more resilience again such pathological case. We slowly
increase the sample size with each roundtrip (+5%). This will have a negligible
impact on "normal" discovery with few roundtrips, but a large positive impact of
case with many roundtrips. Asking more question per roundtrip helps to reduce
the undecided set faster. Instead of reducing the undecided set a linear speed
(in the worst case), we reduce it as a guaranteed (small) exponential rate. The
data below show this slow ramp up in sample size:

round trip    |    1 |    5 |    10 |    20 |    50 |    100 |      130 |
sample size  |  200 |  254 |  321 |  517 |  2 199 |  25 123 |  108 549 |
covered nodes |  200 | 1 357 | 2 821 | 7 031 | 42 658 | 524 530 | 2 276 755 |

To be a bit more concrete, lets take a very pathological case as an example. We
are doing discovery from a copy of Mozilla-try to a more recent version of
mozilla-unified. Mozilla-unified heads are unknown to the mozilla-try repo and
there are over 1 million "missing" changesets. (the discovery is "local" to
avoid network interference)

Without this change, the discovery:
- last 1858 seconds (31 minutes),
- does 1700 round trip,
- asking about 340 000 nodes.

With this change, the discovery:
- last 218 seconds (3 minutes, 38 seconds a -88% improvement),
- does 94 round trip (-94%),
- asking about 344 211 nodes (+1%).

Of course, this is an extreme case (and 3 minutes is still slow). However this
give a good example of how this sample size increase act as a safety net
catching any bad situations.

We could image a steeper increase than 5%. For example 10% would give the
following number:

round trip    |    1 |    5 |    10 |    20 |    50  |        75 |        100 |
sample size  |  200 |  321 |  514 |  1 326 |  23 060 |  249 812 |  2 706 594 |
covered nodes |  200 | 1 541 | 3 690 | 12 671 | 251 871 | 2 746 254 | 29 770 966 |

In parallel, it is useful to understand these pathological cases and improve
them. However the current change provides a general purpose safety net to smooth
the impact of pathological cases.

To avoid issue with older http server, the increase in sample size only occurs
if the protocol has not limit on command argument size.
Juan Francisco Cantero Hurtado
tests: make the grep pattern in remotefilelog-gcrepack portable (issue6122)

test-remotefilelog-gcrepack was using "\" to escape "|" in the grep pattern.
The most of implementations ignore "\" when it is followed by "|", so the regex
works. However, OpenBSD doesn't ignore "\" and considers "|" part of the text
instead of create two branches. Neither of both behaviors violate POSIX.

This change removes the unnecessary escape character and changes grep to egrep,
so the extended regular expression works on every unix.

This is part of the bug 6122. Tested on OpenBSD, GNU, FreeBSD, NetBSD, Solaris
11 and BusyBox.

Credits to Todd C. Miller, Paul de Weerd and Ingo Schwarze for helping me with
it.
Martin von Zweigbergk
help: document new "bookmarksinstore" requirement in internals.requirements

Differential Revision: https://phab.mercurial-scm.org/D6413
Augie Fackler
tests: work around libressl being different about error strings (issue6122)

As far as I can tell, this is the right behavior. Thanks to Alex
Gaynor for checking what the string means by looking at libressl
sources for me.

Differential Revision: https://phab.mercurial-scm.org/D6410
Augie Fackler
absorb: fix interactive mode I didn't know existed

While investigating a bug in `hg absorb -e` I unintentionally
discovered `hg absorb --interactive` and its brokenness. This adds a
test and restores the functionality.

Note that this interface is still marked experimental, so we can
change this to be more sophisticated in the future.

Differential Revision: https://phab.mercurial-scm.org/D6411
Pierre-Yves David
localrepo: grab mixedrepostorecache class from 526750cdd02d

On default, Martin von Zweigbergk <martinvonz@google.com> introduced a more
advance filecache decorator. I need this decorator to fix a bug on stable. So I
am grafting the relevant part of 526750cdd02d.
Yuya Nishihara
templatekw: change default value of 'requires' to ()

Since we dropped support for the old-style template keywords, we no longer
have to distinguish None (old-style) and an empty requirement (new-style).
Martin von Zweigbergk
dirstate: drop workaround for '.' matching root directory

The check was added in 31abcae33b4f (dirstate: do not ignore current
directory '.' (issue 1078), 2008-04-05) to fix issue1078. Funnily
enough, comment #2 on that issue mentions using '' instead of '.' to
represent the root directory, just like my previous patch did.

test-hgignore.t fails with this patch without the previous patch.

Differential Revision: https://phab.mercurial-scm.org/D6402
Martin von Zweigbergk
commit: move sorting of added and removed files list to lower level

localrepo.commitctx() has lists of all changed files, as well as lists
of added and removed files. The list of all files is unsorted and
changelog.add() will sort it. Let's also sort the lists of added and
removed files at a lower level (manifestrevlog.add()) for
consistency. It also seems safer to do it there, just before we write
them to the store. That way other callers won't be able to create
invalid commits (or whatever the consequence is) by passing in
unsorted lists. Also, alternative storages may not care that the lists
are sorted. I don't think this will be a performance problem (someone
should have fixed the sorting in changelog.add() if it were).

Differential Revision: https://phab.mercurial-scm.org/D6390
Pierre-Yves David
bookmark: also make bookmark cache depends of the changelog

Since the changelog is also used during the parsing of bookmark data, it should
be listed as a file cache dependency. This fix the race condition we just
introduced a test for.

This is a simple fix that might lead bookmark data to be invalidated more often
than necessary. We could have more complicated code to deal with this race in a
more "optimal" way. I feel it would be unsuitable for stable.

In addition, the performance impact of this is probably minimal and I don't
foresee the more advanced fix to actually be necessary.
Martin von Zweigbergk
match: drop unnecessary adding of '' to set of dirs

This breaks some tests for "rootfilesin:" in a pattern matcher even
more, but that just shows how broken that case is.

Differential Revision: https://phab.mercurial-scm.org/D6406
Pierre-Yves David
test: update test for expected test output

In 1fac9b931d46 as new test session was introduced. It did not take in account
some part that only ran for pure.

The test is now fixed.
Martin von Zweigbergk
util: make util.dirs() and util.finddirs() include root directory (API)

This changes the behavior of test-origbackup-conflict.t so it no
longer errors out when the backup path points to an existing
file. Instead, it replaces the file by a directory. That seems
reasonable to me.

Differential Revision: https://phab.mercurial-scm.org/D6403
Martin von Zweigbergk
copies: remove hack for adding root dir to util.dirs object

Differential Revision: https://phab.mercurial-scm.org/D6404
Martin von Zweigbergk
convert: demonstrate broken {files} list in merge commits with file flags

When there is a merge in which the flags for a file from p2 is
non-empty, `hg convert` will incorrectly include that in the
changeset's files list.

Differential Revision: https://phab.mercurial-scm.org/D6408