Hi everyone,

as part of this:
https://issues.apache.org/jira/browse/LUCENE-3785

I've been working on moving away from ant-macros to a custom ANT task;
potential benefits are mentioned as part of the issue's description.

There is a lot of code I already pruned from LuceneTestCase as part of
this issue. Incidentally I also discovered a lot of places where the
random field proliferates to where it shouldn't be used. These are:

1) static initializers, including static field initializers;
2) constructors and field initializers in classes;

Anything using random should be placed either in setUp() override or a
custom @Before/ @BeforeClass method. The reason for this is deeply
embedded in how JUnit4 instantiates  classes and instances for tests;
I'd rather not go in the details, but it's really hard to make it work
right with static initializers or field initializers.

I've modified LuceneTestCase and added two methods instead of the
omnipresent 'random' field:

public static Random getStaticRandom() {...}
public          Random getRandom() {...}

these are returning static random context and instance (per-test)
random instance. They also throw exceptions if invoked out-of-scope
(for example from a static initializer). As you may imagine the patch
that adjusts to the above is huge and touches nearly every single test
class. I believe it's for the better -- randoms shouldn't be
uncontrollably shared and this is a first step to enforce that.

I have a working (after the above modifications) version of the trunk at:
https://github.com/dweiss/lucene_solr/tree/LUCENE-3785

and I would like to commit it in as soon as I can so that I don't have
to merge in changes from the trunk's tests all the time :) However,
this patch also modifies certain things everyone has probably gotten
used to:

a) the test seed format is (or rather will) change. It will be simpler
(single number), everything else will be derived from that single
number.
b) the filtering patterns will change. no more -Dtestpackage,
-Dtestpackageroot, etc. The filtering is based on globs now (both at
method and at class level).
c) the output from 'ant test' is slightly different (details in issue
comments'). In my opinion it is clearer. There are also additional
files emitted that can help in diagnosing errors:
tests-report.txt (like the console), tests-failures.txt (errors and
failures only), tests-timehints.txt (hints for the balancer; can be
used to update static snapshot contained in the project). I also emit
junit-report compatible XMLs, but don't know if these are useful for
anything (doesn't seem like we generate those reports).

As for pt (b) above, I've added 'ant test-help' target because I
constantly forget the command line; you can see how filters have
changed, for example:

test-help:
     [echo]
     [echo] #
     [echo] # Test case filtering:
     [echo] # - 'tests.filter' is a class-filtering shell-like glob pattern.
     [echo] # - 'tests.filter.method' is a method-filtering shell-like
glob pattern.
     [echo] #
     [echo]
     [echo] # Run a single test case (verbose)
     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
     [echo]
     [echo] # Run a single test case (simpler)
     [echo] ant test -Dtests.filter=*.ClassName
     [echo]
     [echo] # Run all tests in a package and sub-packages
     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
     [echo]
     [echo] # Run all test methods that contain 'esi' substring
(...r*esi*ze...).
     [echo] ant test -Dtests.filter.method=*esi*
     [echo]
     [echo] #
     [echo] # Load balancing and caches.
     [echo] #
     [echo]
     [echo] # Run with a constant suite order on slave JVM (no job stealing).
     [echo] ant -Djunit4.dynamicAssignmentRatio=0 test
     [echo]
     [echo] # Update global (versioned) cache (from the top-level).
     [echo] ant clean test
     [echo] ant -f lucene/build.xml test-updatecache
     [echo]
     [echo] #
     [echo] # Miscellaneous
     [echo] #
     [echo]
     [echo] # Run all tests without stopping on errors (can be
diagnosed from logs).
     [echo] # from the top level:
     [echo] ant -Djunit4.haltonfailure=false test
     [echo]

Voices of constructive criticism (and support...) are very welcome. I
will keep working on this for the next few days (there are still some
cleanups to do), but I didn't want to catch you by surprise with such
a big change.

Dawid

Sweet.

On Fri, Feb 17, 2012 at 12:56 PM, Dawid Weiss
<[EMAIL PROTECTED]>wrote:

> Hi everyone,
>
> as part of this:
> https://issues.apache.org/jira/browse/LUCENE-3785
>
> I've been working on moving away from ant-macros to a custom ANT task;
> potential benefits are mentioned as part of the issue's description.
>
> There is a lot of code I already pruned from LuceneTestCase as part of
> this issue. Incidentally I also discovered a lot of places where the
> random field proliferates to where it shouldn't be used. These are:
>
> 1) static initializers, including static field initializers;
> 2) constructors and field initializers in classes;
>
> Anything using random should be placed either in setUp() override or a
> custom @Before/ @BeforeClass method. The reason for this is deeply
> embedded in how JUnit4 instantiates  classes and instances for tests;
> I'd rather not go in the details, but it's really hard to make it work
> right with static initializers or field initializers.
>
> I've modified LuceneTestCase and added two methods instead of the
> omnipresent 'random' field:
>
> public static Random getStaticRandom() {...}
> public          Random getRandom() {...}
>
> these are returning static random context and instance (per-test)
> random instance. They also throw exceptions if invoked out-of-scope
> (for example from a static initializer). As you may imagine the patch
> that adjusts to the above is huge and touches nearly every single test
> class. I believe it's for the better -- randoms shouldn't be
> uncontrollably shared and this is a first step to enforce that.
>
> I have a working (after the above modifications) version of the trunk at:
> https://github.com/dweiss/lucene_solr/tree/LUCENE-3785
>
> and I would like to commit it in as soon as I can so that I don't have
> to merge in changes from the trunk's tests all the time :) However,
> this patch also modifies certain things everyone has probably gotten
> used to:
>
> a) the test seed format is (or rather will) change. It will be simpler
> (single number), everything else will be derived from that single
> number.
> b) the filtering patterns will change. no more -Dtestpackage,
> -Dtestpackageroot, etc. The filtering is based on globs now (both at
> method and at class level).
> c) the output from 'ant test' is slightly different (details in issue
> comments'). In my opinion it is clearer. There are also additional
> files emitted that can help in diagnosing errors:
> tests-report.txt (like the console), tests-failures.txt (errors and
> failures only), tests-timehints.txt (hints for the balancer; can be
> used to update static snapshot contained in the project). I also emit
> junit-report compatible XMLs, but don't know if these are useful for
> anything (doesn't seem like we generate those reports).
>
> As for pt (b) above, I've added 'ant test-help' target because I
> constantly forget the command line; you can see how filters have
> changed, for example:
>
> test-help:
>     [echo]
>     [echo] #
>     [echo] # Test case filtering:
>     [echo] # - 'tests.filter' is a class-filtering shell-like glob pattern.
>     [echo] # - 'tests.filter.method' is a method-filtering shell-like
> glob pattern.
>     [echo] #
>     [echo]
>     [echo] # Run a single test case (verbose)
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
>     [echo]
>     [echo] # Run a single test case (simpler)
>     [echo] ant test -Dtests.filter=*.ClassName
>     [echo]
>     [echo] # Run all tests in a package and sub-packages
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
>     [echo]
>     [echo] # Run all test methods that contain 'esi' substring
> (...r*esi*ze...).
>     [echo] ant test -Dtests.filter.method=*esi*
>     [echo]
>     [echo] #
>     [echo] # Load balancing and caches.
>     [echo] #
>     [echo]
>     [echo] # Run with a constant suite order on slave JVM (no job
> stealing).
>     [echo] ant -Djunit4.dynamicAssignmentRatio=0 test
- Mark

http://www.lucidimagination.com

Nice work! I also like the ant test-help target (I also forget the
test options quite often).
Are all options going to be specified in there? I don't see the
tests.iter option.

Martijn

On 17 February 2012 20:17, Mark Miller <[EMAIL PROTECTED]> wrote:
> Sweet.
>
>
> On Fri, Feb 17, 2012 at 12:56 PM, Dawid Weiss <[EMAIL PROTECTED]>
> wrote:
>>
>> Hi everyone,
>>
>> as part of this:
>> https://issues.apache.org/jira/browse/LUCENE-3785
>>
>> I've been working on moving away from ant-macros to a custom ANT task;
>> potential benefits are mentioned as part of the issue's description.
>>
>> There is a lot of code I already pruned from LuceneTestCase as part of
>> this issue. Incidentally I also discovered a lot of places where the
>> random field proliferates to where it shouldn't be used. These are:
>>
>> 1) static initializers, including static field initializers;
>> 2) constructors and field initializers in classes;
>>
>> Anything using random should be placed either in setUp() override or a
>> custom @Before/ @BeforeClass method. The reason for this is deeply
>> embedded in how JUnit4 instantiates  classes and instances for tests;
>> I'd rather not go in the details, but it's really hard to make it work
>> right with static initializers or field initializers.
>>
>> I've modified LuceneTestCase and added two methods instead of the
>> omnipresent 'random' field:
>>
>> public static Random getStaticRandom() {...}
>> public          Random getRandom() {...}
>>
>> these are returning static random context and instance (per-test)
>> random instance. They also throw exceptions if invoked out-of-scope
>> (for example from a static initializer). As you may imagine the patch
>> that adjusts to the above is huge and touches nearly every single test
>> class. I believe it's for the better -- randoms shouldn't be
>> uncontrollably shared and this is a first step to enforce that.
>>
>> I have a working (after the above modifications) version of the trunk at:
>> https://github.com/dweiss/lucene_solr/tree/LUCENE-3785
>>
>> and I would like to commit it in as soon as I can so that I don't have
>> to merge in changes from the trunk's tests all the time :) However,
>> this patch also modifies certain things everyone has probably gotten
>> used to:
>>
>> a) the test seed format is (or rather will) change. It will be simpler
>> (single number), everything else will be derived from that single
>> number.
>> b) the filtering patterns will change. no more -Dtestpackage,
>> -Dtestpackageroot, etc. The filtering is based on globs now (both at
>> method and at class level).
>> c) the output from 'ant test' is slightly different (details in issue
>> comments'). In my opinion it is clearer. There are also additional
>> files emitted that can help in diagnosing errors:
>> tests-report.txt (like the console), tests-failures.txt (errors and
>> failures only), tests-timehints.txt (hints for the balancer; can be
>> used to update static snapshot contained in the project). I also emit
>> junit-report compatible XMLs, but don't know if these are useful for
>> anything (doesn't seem like we generate those reports).
>>
>> As for pt (b) above, I've added 'ant test-help' target because I
>> constantly forget the command line; you can see how filters have
>> changed, for example:
>>
>> test-help:
>>     [echo]
>>     [echo] #
>>     [echo] # Test case filtering:
>>     [echo] # - 'tests.filter' is a class-filtering shell-like glob
>> pattern.
>>     [echo] # - 'tests.filter.method' is a method-filtering shell-like
>> glob pattern.
>>     [echo] #
>>     [echo]
>>     [echo] # Run a single test case (verbose)
>>     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
>>     [echo]
>>     [echo] # Run a single test case (simpler)
>>     [echo] ant test -Dtests.filter=*.ClassName
>>     [echo]
>>     [echo] # Run all tests in a package and sub-packages
>>     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
>>     [echo]
>>     [echo] # Run all test methods that contain 'esi' substring

Met vriendelijke groet,

Martijn van Groningen

Nice improvements! +1
Tommaso

2012/2/17 Dawid Weiss <[EMAIL PROTECTED]>

> Hi everyone,
>
> as part of this:
> https://issues.apache.org/jira/browse/LUCENE-3785
>
> I've been working on moving away from ant-macros to a custom ANT task;
> potential benefits are mentioned as part of the issue's description.
>
> There is a lot of code I already pruned from LuceneTestCase as part of
> this issue. Incidentally I also discovered a lot of places where the
> random field proliferates to where it shouldn't be used. These are:
>
> 1) static initializers, including static field initializers;
> 2) constructors and field initializers in classes;
>
> Anything using random should be placed either in setUp() override or a
> custom @Before/ @BeforeClass method. The reason for this is deeply
> embedded in how JUnit4 instantiates  classes and instances for tests;
> I'd rather not go in the details, but it's really hard to make it work
> right with static initializers or field initializers.
>
> I've modified LuceneTestCase and added two methods instead of the
> omnipresent 'random' field:
>
> public static Random getStaticRandom() {...}
> public          Random getRandom() {...}
>
> these are returning static random context and instance (per-test)
> random instance. They also throw exceptions if invoked out-of-scope
> (for example from a static initializer). As you may imagine the patch
> that adjusts to the above is huge and touches nearly every single test
> class. I believe it's for the better -- randoms shouldn't be
> uncontrollably shared and this is a first step to enforce that.
>
> I have a working (after the above modifications) version of the trunk at:
> https://github.com/dweiss/lucene_solr/tree/LUCENE-3785
>
> and I would like to commit it in as soon as I can so that I don't have
> to merge in changes from the trunk's tests all the time :) However,
> this patch also modifies certain things everyone has probably gotten
> used to:
>
> a) the test seed format is (or rather will) change. It will be simpler
> (single number), everything else will be derived from that single
> number.
> b) the filtering patterns will change. no more -Dtestpackage,
> -Dtestpackageroot, etc. The filtering is based on globs now (both at
> method and at class level).
> c) the output from 'ant test' is slightly different (details in issue
> comments'). In my opinion it is clearer. There are also additional
> files emitted that can help in diagnosing errors:
> tests-report.txt (like the console), tests-failures.txt (errors and
> failures only), tests-timehints.txt (hints for the balancer; can be
> used to update static snapshot contained in the project). I also emit
> junit-report compatible XMLs, but don't know if these are useful for
> anything (doesn't seem like we generate those reports).
>
> As for pt (b) above, I've added 'ant test-help' target because I
> constantly forget the command line; you can see how filters have
> changed, for example:
>
> test-help:
>     [echo]
>     [echo] #
>     [echo] # Test case filtering:
>     [echo] # - 'tests.filter' is a class-filtering shell-like glob pattern.
>     [echo] # - 'tests.filter.method' is a method-filtering shell-like
> glob pattern.
>     [echo] #
>     [echo]
>     [echo] # Run a single test case (verbose)
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
>     [echo]
>     [echo] # Run a single test case (simpler)
>     [echo] ant test -Dtests.filter=*.ClassName
>     [echo]
>     [echo] # Run all tests in a package and sub-packages
>     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
>     [echo]
>     [echo] # Run all test methods that contain 'esi' substring
> (...r*esi*ze...).
>     [echo] ant test -Dtests.filter.method=*esi*
>     [echo]
>     [echo] #
>     [echo] # Load balancing and caches.
>     [echo] #
>     [echo]
>     [echo] # Run with a constant suite order on slave JVM (no job
> stealing).
>     [echo] ant -Djunit4.dynamicAssignmentRatio=0 test

thanks david for the hard work.... this is awesome!

simon

On Fri, Feb 17, 2012 at 11:16 PM, Tommaso Teofili <[EMAIL PROTECTED]
> wrote:

> Nice improvements! +1
> Tommaso
>
>
> 2012/2/17 Dawid Weiss <[EMAIL PROTECTED]>
>
>> Hi everyone,
>>
>> as part of this:
>> https://issues.apache.org/jira/browse/LUCENE-3785
>>
>> I've been working on moving away from ant-macros to a custom ANT task;
>> potential benefits are mentioned as part of the issue's description.
>>
>> There is a lot of code I already pruned from LuceneTestCase as part of
>> this issue. Incidentally I also discovered a lot of places where the
>> random field proliferates to where it shouldn't be used. These are:
>>
>> 1) static initializers, including static field initializers;
>> 2) constructors and field initializers in classes;
>>
>> Anything using random should be placed either in setUp() override or a
>> custom @Before/ @BeforeClass method. The reason for this is deeply
>> embedded in how JUnit4 instantiates  classes and instances for tests;
>> I'd rather not go in the details, but it's really hard to make it work
>> right with static initializers or field initializers.
>>
>> I've modified LuceneTestCase and added two methods instead of the
>> omnipresent 'random' field:
>>
>> public static Random getStaticRandom() {...}
>> public          Random getRandom() {...}
>>
>> these are returning static random context and instance (per-test)
>> random instance. They also throw exceptions if invoked out-of-scope
>> (for example from a static initializer). As you may imagine the patch
>> that adjusts to the above is huge and touches nearly every single test
>> class. I believe it's for the better -- randoms shouldn't be
>> uncontrollably shared and this is a first step to enforce that.
>>
>> I have a working (after the above modifications) version of the trunk at:
>> https://github.com/dweiss/lucene_solr/tree/LUCENE-3785
>>
>> and I would like to commit it in as soon as I can so that I don't have
>> to merge in changes from the trunk's tests all the time :) However,
>> this patch also modifies certain things everyone has probably gotten
>> used to:
>>
>> a) the test seed format is (or rather will) change. It will be simpler
>> (single number), everything else will be derived from that single
>> number.
>> b) the filtering patterns will change. no more -Dtestpackage,
>> -Dtestpackageroot, etc. The filtering is based on globs now (both at
>> method and at class level).
>> c) the output from 'ant test' is slightly different (details in issue
>> comments'). In my opinion it is clearer. There are also additional
>> files emitted that can help in diagnosing errors:
>> tests-report.txt (like the console), tests-failures.txt (errors and
>> failures only), tests-timehints.txt (hints for the balancer; can be
>> used to update static snapshot contained in the project). I also emit
>> junit-report compatible XMLs, but don't know if these are useful for
>> anything (doesn't seem like we generate those reports).
>>
>> As for pt (b) above, I've added 'ant test-help' target because I
>> constantly forget the command line; you can see how filters have
>> changed, for example:
>>
>> test-help:
>>     [echo]
>>     [echo] #
>>     [echo] # Test case filtering:
>>     [echo] # - 'tests.filter' is a class-filtering shell-like glob
>> pattern.
>>     [echo] # - 'tests.filter.method' is a method-filtering shell-like
>> glob pattern.
>>     [echo] #
>>     [echo]
>>     [echo] # Run a single test case (verbose)
>>     [echo] ant test -Dtests.filter=org.apache.lucene.package.ClassName
>>     [echo]
>>     [echo] # Run a single test case (simpler)
>>     [echo] ant test -Dtests.filter=*.ClassName
>>     [echo]
>>     [echo] # Run all tests in a package and sub-packages
>>     [echo] ant test -Dtests.filter=org.apache.lucene.package.*
>>     [echo]
>>     [echo] # Run all test methods that contain 'esi' substring
>> (...r*esi*ze...).
>>     [echo] ant test -Dtests.filter.method=*esi*

On Fri, Feb 17, 2012 at 12:56 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
>
> There is a lot of code I already pruned from LuceneTestCase as part of

+1: this thing is out of control and not really maintainable from just
adding this test-feature and that... And currently nobody 'tests the
tester' so we don't know if all that crazy code even really works.

> this issue. Incidentally I also discovered a lot of places where the
> random field proliferates to where it shouldn't be used. These are:
>

Yeah: this is part of why there is a lot of hairiness: we keep track
of these different seeds and set one random to the right values. Its
easy on the test cases, but it definitely makes the base test class
really hairy.

> and I would like to commit it in as soon as I can so that I don't have
> to merge in changes from the trunk's tests all the time :) However,
> this patch also modifies certain things everyone has probably gotten
> used to:

Personally I say 'go for it'. My only concern is that test failures
reproduce, but this is likely just paranoia from the existing
code...I'm guessing you have this under control.

> b) the filtering patterns will change. no more -Dtestpackage,
> -Dtestpackageroot, etc. The filtering is based on globs now (both at
> method and at class level).

Can we keep -Dtestcase? I see the -Dtests.filter=".*Foo" but is it
possible we can just add -Dtestcase=X as a 'macro' to
-Dtests.filter=".*X" ?
--
lucidimagination.com

---------------------------------------------------------------------

> Personally I say 'go for it'. My only concern is that test failures
> reproduce, but this is likely just paranoia from the existing
> code...I'm guessing you have this under control.

You mean past test failures (with known seeds)? This won't be possible
I'm afraid. Any change to how seeds are propagated will ruin archival
known failures. As for "after the change" failure reproduction; if
anything happens it should be reproducible pretty much as it was
before (or better).

> Can we keep -Dtestcase? I see the -Dtests.filter=".*Foo" but is it
> possible we can just add -Dtestcase=X as a 'macro' to
> -Dtests.filter=".*X" ?

Sure we can add it, you don't even need a macro -- a simple condition
will do. My thinking was that if we're making major changes to the
testing factor we may as well consolidate and simplify to as few
options as possible. There are other non-backward compatible changes
like -Dtestpackageroot is not really covered by globbing filters; you
can simulate it via:

-Dtests.filter=*.package.Test*

but for this to work all your tests really need to follow a naming
convention (start with "Test"). For *Test this isn't going to work
because subpackages will match. I was playing with various options
(ant-like ** patterns, regexps) but really shell globbing was the
easiest one to use and most natural.

There is one showstopper issue I'm still trying to track (slave
failing with JSON serialization weirdness). Don't know if it's a
dependency bug (gson) or something else. Once I have that solved, I'll
warn again and commit this to trunk. If there's anything weird going
on with tests on Jenkins we will either revert or fix as we go.

Dawid

---------------------------------------------------------------------

On Sat, Feb 18, 2012 at 11:41 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
>> Personally I say 'go for it'. My only concern is that test failures
>> reproduce, but this is likely just paranoia from the existing
>> code...I'm guessing you have this under control.
>
> You mean past test failures (with known seeds)?

no that would be crazy!!!!

I mean "reproducibility bugs" where somehow a test doesn't reproduce
because of bugs in LuceneTestCase :)

>
> Sure we can add it, you don't even need a macro -- a simple condition
> will do. My thinking was that if we're making major changes to the
> testing factor we may as well consolidate and simplify to as few
> options as possible. There are other non-backward compatible changes
> like -Dtestpackageroot is not really covered by globbing filters; you
> can simulate it via:

Right, I think that we can consolidate more obscure stuff (like
testpackageroot), but I was just mentioning i hand-type in -Dtestcase
and also -Dtestmethod quite often when debugging and writing tests.

>
> There is one showstopper issue I'm still trying to track (slave
> failing with JSON serialization weirdness). Don't know if it's a
> dependency bug (gson) or something else. Once I have that solved, I'll
> warn again and commit this to trunk. If there's anything weird going
> on with tests on Jenkins we will either revert or fix as we go.

Well at some point we have to cut over to this... so I think its
expected we might hit a few snags... but we are probably also fixing a
few bugs we just dont know about in the test infrastructure at the
same time.

--
lucidimagination.com

---------------------------------------------------------------------

> I mean "reproducibility bugs" where somehow a test doesn't reproduce
> because of bugs in LuceneTestCase :)

Oh no -- the whole point of all this is so that we have 100%
reproducibility (is there even a word like that?) or failures saying
something like: "this random shouldn't be shared and is, fix your test
case named: ...".

> Right, I think that we can consolidate more obscure stuff (like
> testpackageroot), but I was just mentioning i hand-type in -Dtestcase
> and also -Dtestmethod quite often when debugging and writing tests.

Sure, we can add some shortcuts to avoid typing. Remind me about it if
I forget, ok?

> Well at some point we have to cut over to this... so I think its
> expected we might hit a few snags... but we are probably also fixing a
> few bugs we just dont know about in the test infrastructure at the
> same time.

Yep, that's my guess too. There's this one thing with gson that I
don't have a reproducible test case for myself -- once I figure out
what this is, I think it'll be ready to land on trunk.

Dawid

---------------------------------------------------------------------

> > I mean "reproducibility bugs" where somehow a test doesn't reproduce
> > because of bugs in LuceneTestCase :)
>
> Oh no -- the whole point of all this is so that we have 100% reproducibility (is
> there even a word like that?) or failures saying something like: "this random
> shouldn't be shared and is, fix your test case named: ...".

:-) It would be great if we could ever have that, but I don’t think that’s possible! Most non-reproducibility in Lucene came from multi-threaded tests where the different threads interact with each other... And that cannot be solved in any case without serial execution.

Uwe
---------------------------------------------------------------------

> :-) It would be great if we could ever have that, but I don’t think that’s possible!
> Most non-reproducibility in Lucene came from multi-threaded tests where the different threads interact with > each other... And that cannot be solved in any case without serial execution.

If there are race conditions in threads involved in the suite (or if
there are unpredictable timing schedules resulting from network or
disk operations) then sure -- it's an impossible case. But race
conditions shouldn't be a result of sharing a single random generator;
it's possible to avoid that by giving each thread its own copy of a
Random instance if it asks for one (predictably initialized to the
same starting seed from each thread). This is effectively a
thread-local randomness and ensures any call sequence to it from
within a thread will be reproducible. I've implemented this as part of
RandomizedRunner and it works really well. Upgrading to
RandomizedRunner is a long-term goal though, I need to refactor LTC in
smaller steps to avoid going insane ;)

You could even go a step further and make sure Randoms are not shared
by locking-in and verifying the owner thread in each call... This
would break a LOT of existing tests though -- I don't have any
estimates how much effort would be needed to straighten these up.

---------------------------------------------------------------------