Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/323/

1 tests failed.
REGRESSION:  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([1A47DEC61940914D:8264B90547362D05]:0)
at org.apache.lucene.util.fst.FST.<init>(FST.java:331)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsReader.<init>(MemoryPostingsFormat.java:811)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat.fieldsProducer(MemoryPostingsFormat.java:858)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:186)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:250)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:107)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:55)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:639)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64)
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:602)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:143)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:472)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
Build Log:
[...truncated 4666 lines...]
   [junit4] Suite: org.apache.lucene.analysis.ja.TestJapaneseTokenizer
   [junit4] ERROR   28.6s J0 | TestJapaneseTokenizer.testRandomHugeStrings
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > at __randomizedtesting.SeedInfo.seed([1A47DEC61940914D:8264B90547362D05]:0)
   [junit4]    > at org.apache.lucene.util.fst.FST.<init>(FST.java:331)
   [junit4]    > at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsReader.<init>(MemoryPostingsFormat.java:811)
   [junit4]    > at org.apache.lucene.codecs.memory.MemoryPostingsFormat.fieldsProducer(MemoryPostingsFormat.java:858)
   [junit4]    > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:186)
   [junit4]    > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:250)
   [junit4]    > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:107)
   [junit4]    > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:55)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
   [junit4]    > at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:639)
   [junit4]    > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
   [junit4]    > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64)
   [junit4]    > at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:602)
   [junit4]    > at org.apache.lucene.util.IOUtils.close(IOUtils.java:143)
   [junit4]    > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:472)
   [junit4]    > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
   [junit4]    > at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
   [junit4]    > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]    > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.lang.reflect.Method.invoke(Method.java:601)
   [junit4]    > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
   [junit4]    > at com.carrotsearch.randomizedtesting.RandomizedRunner.acce

This is an OOM.

Dawid

On Tue, Jun 19, 2012 at 9:01 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/963/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings
>
> Error Message:
> some thread(s) failed
>
> Stack Trace:
> java.lang.RuntimeException: some thread(s) failed
>        at __randomizedtesting.SeedInfo.seed([370A7DC6CEEB9088:AF291A05909D2CC0]:0)
>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:466)
>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
>        at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>        at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>        at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>        at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

I'll dig.

Mike McCandless

http://blog.mikemccandless.com
On Tue, Jun 19, 2012 at 3:24 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
> This is an OOM.
>
> Dawid
>
> On Tue, Jun 19, 2012 at 9:01 AM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/963/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings
>>
>> Error Message:
>> some thread(s) failed
>>
>> Stack Trace:
>> java.lang.RuntimeException: some thread(s) failed
>>        at __randomizedtesting.SeedInfo.seed([370A7DC6CEEB9088:AF291A05909D2CC0]:0)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:466)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
>>        at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
>>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>>        at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>        at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>>        at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)

I think this is because we are getting MemoryPostingsFormat in
testRandomHugeStrings, when it does create a RandomIndexWriter for
indexing.

I added @Suppress({"Memory"}) and it no longer OOMEs; though, I'm
baffled why the other tests that also call checkRandomData don't also
sometimes OOME.  I looked at the heap dump and on quick inspection it
does look like the RAM is used up by the FSTs for MemoryPF....

To be sure it wasn't eg a memory leak of some kind w/ Kuromoji, I ran
with 50 multiplier and the tests still passes...

Mike McCandless

http://blog.mikemccandless.com

On Tue, Jun 19, 2012 at 7:00 AM, Michael McCandless
<[EMAIL PROTECTED]> wrote:
> I'll dig.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Jun 19, 2012 at 3:24 AM, Dawid Weiss
> <[EMAIL PROTECTED]> wrote:
>> This is an OOM.
>>
>> Dawid
>>
>> On Tue, Jun 19, 2012 at 9:01 AM, Policeman Jenkins Server
>> <[EMAIL PROTECTED]> wrote:
>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/963/
>>>
>>> 1 tests failed.
>>> REGRESSION:  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings
>>>
>>> Error Message:
>>> some thread(s) failed
>>>
>>> Stack Trace:
>>> java.lang.RuntimeException: some thread(s) failed
>>>        at __randomizedtesting.SeedInfo.seed([370A7DC6CEEB9088:AF291A05909D2CC0]:0)
>>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:466)
>>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
>>>        at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>>>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)

Mike did you see the class profile from that dump? If there is something (a
lot) printed to sysout it may be the rinner's buffering causing this. I
have it on the list of thngs to fix.
On Jun 19, 2012 2:03 PM, "Michael McCandless" <[EMAIL PROTECTED]>
wrote:

> I think this is because we are getting MemoryPostingsFormat in
> testRandomHugeStrings, when it does create a RandomIndexWriter for
> indexing.
>
> I added @Suppress({"Memory"}) and it no longer OOMEs; though, I'm
> baffled why the other tests that also call checkRandomData don't also
> sometimes OOME.  I looked at the heap dump and on quick inspection it
> does look like the RAM is used up by the FSTs for MemoryPF....
>
> To be sure it wasn't eg a memory leak of some kind w/ Kuromoji, I ran
> with 50 multiplier and the tests still passes...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Tue, Jun 19, 2012 at 7:00 AM, Michael McCandless
> <[EMAIL PROTECTED]> wrote:
> > I'll dig.
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> >
> > On Tue, Jun 19, 2012 at 3:24 AM, Dawid Weiss
> > <[EMAIL PROTECTED]> wrote:
> >> This is an OOM.
> >>
> >> Dawid
> >>
> >> On Tue, Jun 19, 2012 at 9:01 AM, Policeman Jenkins Server
> >> <[EMAIL PROTECTED]> wrote:
> >>> Build:
> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/963/
> >>>
> >>> 1 tests failed.
> >>> REGRESSION:
>  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings
> >>>
> >>> Error Message:
> >>> some thread(s) failed
> >>>
> >>> Stack Trace:
> >>> java.lang.RuntimeException: some thread(s) failed
> >>>        at
> __randomizedtesting.SeedInfo.seed([370A7DC6CEEB9088:AF291A05909D2CC0]:0)
> >>>        at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:466)
> >>>        at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
> >>>        at
> org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
> >>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>        at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>>        at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>        at java.lang.reflect.Method.invoke(Method.java:597)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
> >>>        at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> >>>        at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
> >>>        at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> >>>        at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> >>>        at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> >>>        at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> >>>        at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
> >>>        at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)

The test doesn't print anything to stdout/stderr unless verbose is on
(or on failure), so I think we are OK there ...

Mike McCandless

http://blog.mikemccandless.com

On Tue, Jun 19, 2012 at 11:57 AM, Dawid Weiss <[EMAIL PROTECTED]> wrote:
> Mike did you see the class profile from that dump? If there is something (a
> lot) printed to sysout it may be the rinner's buffering causing this. I have
> it on the list of thngs to fix.
>
> On Jun 19, 2012 2:03 PM, "Michael McCandless" <[EMAIL PROTECTED]>
> wrote:
>>
>> I think this is because we are getting MemoryPostingsFormat in
>> testRandomHugeStrings, when it does create a RandomIndexWriter for
>> indexing.
>>
>> I added @Suppress({"Memory"}) and it no longer OOMEs; though, I'm
>> baffled why the other tests that also call checkRandomData don't also
>> sometimes OOME.  I looked at the heap dump and on quick inspection it
>> does look like the RAM is used up by the FSTs for MemoryPF....
>>
>> To be sure it wasn't eg a memory leak of some kind w/ Kuromoji, I ran
>> with 50 multiplier and the tests still passes...
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Tue, Jun 19, 2012 at 7:00 AM, Michael McCandless
>> <[EMAIL PROTECTED]> wrote:
>> > I'll dig.
>> >
>> > Mike McCandless
>> >
>> > http://blog.mikemccandless.com
>> >
>> >
>> > On Tue, Jun 19, 2012 at 3:24 AM, Dawid Weiss
>> > <[EMAIL PROTECTED]> wrote:
>> >> This is an OOM.
>> >>
>> >> Dawid
>> >>
>> >> On Tue, Jun 19, 2012 at 9:01 AM, Policeman Jenkins Server
>> >> <[EMAIL PROTECTED]> wrote:
>> >>> Build:
>> >>> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/963/
>> >>>
>> >>> 1 tests failed.
>> >>> REGRESSION:
>> >>>  org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings
>> >>>
>> >>> Error Message:
>> >>> some thread(s) failed
>> >>>
>> >>> Stack Trace:
>> >>> java.lang.RuntimeException: some thread(s) failed
>> >>>        at
>> >>> __randomizedtesting.SeedInfo.seed([370A7DC6CEEB9088:AF291A05909D2CC0]:0)
>> >>>        at
>> >>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:466)
>> >>>        at
>> >>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
>> >>>        at
>> >>> org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
>> >>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>>        at
>> >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >>>        at
>> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>>        at java.lang.reflect.Method.invoke(Method.java:597)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>> >>>        at
>> >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>> >>>        at
>> >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>> >>>        at
>> >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> >>>        at
>> >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>> >>>        at
>> >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)

I fixed this quickly.  A file was affected by another changeset for another issue.

On Jun 20, 2012, at 12:42 AM, Policeman Jenkins Server wrote:

> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/975/
>
> All tests passed
>
> Build Log:
> [...truncated 7439 lines...]
>    [javac] Compiling 49 source files to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/solr/build/solr-solrj/classes/test
>    [javac] /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/solr/solrj/src/test/org/apache/solr/client/solrj/SolrExampleTests.java:1167: cannot find symbol
>    [javac] symbol  : method setRequestHandler(java.lang.String)
>    [javac] location: class org.apache.solr.client.solrj.SolrQuery
>    [javac]     q.setRequestHandler("/get");
>    [javac]      ^
>    [javac] Note: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/solr/solrj/src/test/org/apache/solr/client/solrj/SolrQueryTest.java uses or overrides a deprecated API.
>    [javac] Note: Recompile with -Xlint:deprecation for details.
>    [javac] Note: Some input files use unchecked or unsafe operations.
>    [javac] Note: Recompile with -Xlint:unchecked for details.
>    [javac] 1 error
>
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/build.xml:29: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/solr/build.xml:151: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/module-build.xml:62: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:646: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:660: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:1288: Compile failed; see the compiler error output for details.
>
> Total time: 20 minutes 55 seconds
> Build step 'Execute shell' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
---------------------------------------------------------------------

This failure is due to a javadocs warning:

  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.6.0_32
  [javadoc] Building tree for all the packages and classes...
  [javadoc] /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/spatial/src/java/org/apache/lucene/spatial/SpatialStrategy.java:71: warning - @return tag has no arguments.
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/build/docs/spatial/stylesheet.css...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 warning

I'm looking into why this info didn't make it into the notification mail.

-----Original Message-----
From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 28, 2012 3:39 AM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1078 - Failure!

Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1078/

All tests passed

Build Log:
[...truncated 17695 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/build.xml:262: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:1435: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/spatial/build.xml:28: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/module-build.xml:81: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-64/checkout/lucene/common-build.xml:1418: Javadocs warnings were found!

Total time: 1 minute 54 seconds
Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure
---------------------------------------------------------------------

FYI I fixed this javadoc warning in recent commits.

-----
 Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book
--
View this message in context: http://lucene.472066.n3.nabble.com/JENKINS-Lucene-Solr-trunk-Linux-Java6-64-Build-1078-Failure-tp3991771p3991951.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

---------------------------------------------------------------------

I've seen this one on my build server too.

Dawid

On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings
>
> Error Message:
> some thread(s) failed
>
> Stack Trace:
> java.lang.RuntimeException: some thread(s) failed
>        at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463)
>        at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
>        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>        at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>        at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
>        at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
>        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)

This is the root cause:

     [junit4]    > Caused by: java.lang.AssertionError: ram was
41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192
pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948
  [junit4]    >        at
__randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0)
  [junit4]    >        at
org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114)
  [junit4]    >        at
org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181)
  [junit4]    >        at
org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348)
  [junit4]    >        at
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174)
  [junit4]    >        at
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134)
  [junit4]    >        at
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157)
  [junit4]    >        at
org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)
  [junit4]    >        at
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555)
  [junit4]    >        at
org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57)
  [junit4]    >        at
org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414)

Is there an issue open for this?  I couldn't find it on quick search...

Mike McCandless

http://blog.mikemccandless.com

On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
> I've seen this one on my build server too.
>
> Dawid
>
> On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings
>>
>> Error Message:
>> some thread(s) failed
>>
>> Stack Trace:
>> java.lang.RuntimeException: some thread(s) failed
>>        at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463)
>>        at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>        at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)

I don't think there's a jira entry for this, but I also saw it on my
build server (ram was X expected Y). Don't know what to call it though
(what this assertion actually does).

Dawid

On Fri, Jun 29, 2012 at 11:59 AM, Michael McCandless
<[EMAIL PROTECTED]> wrote:
> This is the root cause:
>
>     [junit4]    > Caused by: java.lang.AssertionError: ram was
> 41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192
> pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948
>  [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0)
>  [junit4]    >        at
> org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114)
>  [junit4]    >        at
> org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181)
>  [junit4]    >        at
> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348)
>  [junit4]    >        at
> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174)
>  [junit4]    >        at
> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134)
>  [junit4]    >        at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157)
>  [junit4]    >        at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)
>  [junit4]    >        at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555)
>  [junit4]    >        at
> org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57)
>  [junit4]    >        at
> org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414)
>
> Is there an issue open for this?  I couldn't find it on quick search...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss
> <[EMAIL PROTECTED]> wrote:
>> I've seen this one on my build server too.
>>
>> Dawid
>>
>> On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server
>> <[EMAIL PROTECTED]> wrote:
>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/
>>>
>>> 1 tests failed.
>>> REGRESSION:  org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings
>>>
>>> Error Message:
>>> some thread(s) failed
>>>
>>> Stack Trace:
>>> java.lang.RuntimeException: some thread(s) failed
>>>        at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
>>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463)
>>>        at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>>>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>>        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>>        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>>        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)

OK I opened LUCENE-4182.

Mike McCandless

http://blog.mikemccandless.com

On Fri, Jun 29, 2012 at 6:03 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
> I don't think there's a jira entry for this, but I also saw it on my
> build server (ram was X expected Y). Don't know what to call it though
> (what this assertion actually does).
>
> Dawid
>
> On Fri, Jun 29, 2012 at 11:59 AM, Michael McCandless
> <[EMAIL PROTECTED]> wrote:
>> This is the root cause:
>>
>>     [junit4]    > Caused by: java.lang.AssertionError: ram was
>> 41605728 expected: 40986328 flush mem: 27371536 activeMem: 14234192
>> pendingMem: 0 flushingMem: 2 blockedMem: 0 peakDeltaMem: 3715948
>>  [junit4]    >        at
>> __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A]:0)
>>  [junit4]    >        at
>> org.apache.lucene.index.DocumentsWriterFlushControl.assertMemory(DocumentsWriterFlushControl.java:114)
>>  [junit4]    >        at
>> org.apache.lucene.index.DocumentsWriterFlushControl.doAfterDocument(DocumentsWriterFlushControl.java:181)
>>  [junit4]    >        at
>> org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:348)
>>  [junit4]    >        at
>> org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1174)
>>  [junit4]    >        at
>> org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1134)
>>  [junit4]    >        at
>> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:157)
>>  [junit4]    >        at
>> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)
>>  [junit4]    >        at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:555)
>>  [junit4]    >        at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase.access$000(BaseTokenStreamTestCase.java:57)
>>  [junit4]    >        at
>> org.apache.lucene.analysis.BaseTokenStreamTestCase$AnalysisThread.run(BaseTokenStreamTestCase.java:414)
>>
>> Is there an issue open for this?  I couldn't find it on quick search...
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Fri, Jun 29, 2012 at 3:22 AM, Dawid Weiss
>> <[EMAIL PROTECTED]> wrote:
>>> I've seen this one on my build server too.
>>>
>>> Dawid
>>>
>>> On Fri, Jun 29, 2012 at 9:19 AM, Policeman Jenkins Server
>>> <[EMAIL PROTECTED]> wrote:
>>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1090/
>>>>
>>>> 1 tests failed.
>>>> REGRESSION:  org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings
>>>>
>>>> Error Message:
>>>> some thread(s) failed
>>>>
>>>> Stack Trace:
>>>> java.lang.RuntimeException: some thread(s) failed
>>>>        at __randomizedtesting.SeedInfo.seed([256D0DE54BD0473A:ADE40D5BE8D4100F]:0)
>>>>        at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:463)
>>>>        at org.apache.lucene.analysis.ngram.NGramTokenizerTest.testRandomStrings(NGramTokenizerTest.java:106)
>>>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
>>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
>>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
>>>>        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
>>>>        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)

Hi all,

on my Jenkins box I installed the Java 8 preview (b45 from end of June). This time we found no hotpot bugs, and I can say, Lucene seems to work fine with Java 8.

Some Solr tests failed as always. One of them seems to rely on the order of HashSet/HashMap, which is a test bug.

The test run with LuSolr trunk, Java 8 will run once per day. Please look at the failing tests and fix those broken tests! I will do the same for 4.x, too.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Sunday, July 01, 2012 7:16 PM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 1 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java8-
> 64/1/
>
> 3 tests failed.
> FAILED:
> org.apache.solr.handler.component.DistributedQueryElevationComponentTest.t
> estDistribSearch
>
> Error Message:
> Index: 18, Size: 18
>
> Stack Trace:
> java.lang.IndexOutOfBoundsException: Index: 18, Size: 18
> at
> __randomizedtesting.SeedInfo.seed([CE81EF7A000D6B19:4F67616277520B25]:
> 0)
> at java.util.ArrayList.rangeCheck(ArrayList.java:603)
> at java.util.ArrayList.get(ArrayList.java:381)
> at
> org.apache.solr.common.util.NamedList.getName(NamedList.java:127)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:467)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:613)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:501)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:613)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:501)
> at
> org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistrib
> utedSearchTestCase.java:668)
> at
> org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTe
> stCase.java:393)
> at
> org.apache.solr.handler.component.DistributedQueryElevationComponentTest.
> doTest(DistributedQueryElevationComponentTest.java:81)
> at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistribut
> edSearchTestCase.java:680)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:474)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1969)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:814)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:875)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:889)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.eval
> uate(SystemPropertiesRestoreRule.java:53)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea

My fault. ill fix

On Mon, Jul 9, 2012 at 11:57 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1191/
>
> All tests passed
>
> Build Log:
> [...truncated 19680 lines...]
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ar...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.bg...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.br...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ca...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.cjk...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.compound...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.core...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.cz...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.da...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.de...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.el...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.en...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.es...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.eu...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.fa...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.fi...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.fr...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ga...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.gl...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.hi...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.hu...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.hunspell...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.hy...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.id...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.in...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.it...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.lv...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.miscellaneous...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ngram...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.nl...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.no...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.path...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.pattern...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.payloads...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.position...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.pt...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.query...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.reverse...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ro...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.ru...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.shingle...
>   [javadoc] Loading source files for package org.apache.lucene.analysis.sinks...

lucidimagination.com

This one is interesting, as the VM was hanging in one test, but randomizedrunner did not send heartbeats! It looks like this could be a bug in randomizedrunner. But it was also hanging in Solr, approx. at the same time like the otter one. Stack trace is in build log as Linux!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Friday, July 13, 2012 9:33 AM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-
> 64/1240/
>
> All tests passed
>
> Build Log:
> [...truncated 16373 lines...]
> [junit4:junit4] 2012-07-13 07:32:56
> [junit4:junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.8-b03
> mixed mode):
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007ffb8005d000
> nid=0x4daf waiting on condition [0x00007ffb213d1000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> )
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007ffb8017a800
> nid=0x4dac waiting on condition [0x00007ffb2b0ad000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> )
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007ffb802e9000
> nid=0x4da9 waiting for monitor entry [0x00007ffb2b6f5000]
> [junit4:junit4]    java.lang.Thread.State: BLOCKED (on object monitor)
> [junit4:junit4] at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1016)
> [junit4:junit4] - waiting to lock <0x00000000e28468d0> (a
> java.util.LinkedHashMap)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:195)
> [junit4:junit4]
> [junit4:junit4] "Thread-734" daemon prio=10 tid=0x00007ffb8006c800
> nid=0x4da5 waiting on condition [0x00007ffb21cda000]
> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for  <0x00000000e53f8330> (a
> java.util.concurrent.CountDownLatch$Sync)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> AbstractQueuedSynchronizer.java:811)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> ptibly(AbstractQueuedSynchronizer.java:969)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> bly(AbstractQueuedSynchronizer.java:1281)
> [junit4:junit4] at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQ
> ueue.java:185)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:291)
> [junit4:junit4] at
> org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProc
> essor.java:81)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:662)
> [junit4:junit4]
> [junit4:junit4] "Thread-733" daemon prio=10 tid=0x00007ffb8007f800
> nid=0x4da4 waiting on condition [0x00007ffb217d5000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)

> This one is interesting, as the VM was hanging in one test, but randomizedrunner did not send heartbeats! It looks like this could be a bug in randomizedrunner. But it was also hanging in Solr, approx. at the same time like the otter one. Stack trace is in build log as Linux!

Heartbeats are sent only if there are no events from the test -- it
could be as well that the test is writing something to stdout/stderr
periodically? Now that I think of it heartbeats should be sent even if
the test is writing something to the console but not responding within
the given time window. What do you think?

Dawid

---------------------------------------------------------------------

We are currently investigating another hang. The next run also hung, it seems that a commit last night made the distributed tests hang forver.

Dawid was decoding the test runner files (JSON), this is printed to stdout all the time (at 0% CPU):

4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error while trying to recover. org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:59584/solr

Clearly a serious bug!
Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Friday, July 13, 2012 10:10 AM
> To: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>
> > This one is interesting, as the VM was hanging in one test, but
> randomizedrunner did not send heartbeats! It looks like this could be a bug in
> randomizedrunner. But it was also hanging in Solr, approx. at the same time
> like the otter one. Stack trace is in build log as Linux!
>
> Heartbeats are sent only if there are no events from the test -- it could be as
> well that the test is writing something to stdout/stderr periodically? Now that I
> think of it heartbeats should be sent even if the test is writing something to the
> console but not responding within the given time window. What do you think?
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

I'll post a full stderr log in a minute.
D.

On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> We are currently investigating another hang. The next run also hung, it seems that a commit last night made the distributed tests hang forver.
>
> Dawid was decoding the test runner files (JSON), this is printed to stdout all the time (at 0% CPU):
>
> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error while trying to recover. org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:59584/solr
>
> Clearly a serious bug!
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
>> -----Original Message-----
>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
>> Dawid Weiss
>> Sent: Friday, July 13, 2012 10:10 AM
>> To: [EMAIL PROTECTED]
>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>>
>> > This one is interesting, as the VM was hanging in one test, but
>> randomizedrunner did not send heartbeats! It looks like this could be a bug in
>> randomizedrunner. But it was also hanging in Solr, approx. at the same time
>> like the otter one. Stack trace is in build log as Linux!
>>
>> Heartbeats are sent only if there are no events from the test -- it could be as
>> well that the test is writing something to stdout/stderr periodically? Now that I
>> think of it heartbeats should be sent even if the test is writing something to the
>> console but not responding within the given time window. What do you think?
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
>> commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip

Uh, these tests emit a lot of logs...

Dawid

On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
> I'll post a full stderr log in a minute.
> D.
>
> On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
>> We are currently investigating another hang. The next run also hung, it seems that a commit last night made the distributed tests hang forver.
>>
>> Dawid was decoding the test runner files (JSON), this is printed to stdout all the time (at 0% CPU):
>>
>> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error while trying to recover. org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:59584/solr
>>
>> Clearly a serious bug!
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: [EMAIL PROTECTED]
>>
>>
>>> -----Original Message-----
>>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
>>> Dawid Weiss
>>> Sent: Friday, July 13, 2012 10:10 AM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>>>
>>> > This one is interesting, as the VM was hanging in one test, but
>>> randomizedrunner did not send heartbeats! It looks like this could be a bug in
>>> randomizedrunner. But it was also hanging in Solr, approx. at the same time
>>> like the otter one. Stack trace is in build log as Linux!
>>>
>>> Heartbeats are sent only if there are no events from the test -- it could be as
>>> well that the test is writing something to stdout/stderr periodically? Now that I
>>> think of it heartbeats should be sent even if the test is writing something to the
>>> console but not responding within the given time window. What do you think?
>>>
>>> Dawid
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
>>> commands, e-mail: [EMAIL PROTECTED]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>

---------------------------------------------------------------------

That's the build Dawid and me were analyzing live.... (open heart surgery)

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Friday, July 13, 2012 12:43 PM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 570 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-
> 64/570/
>
> All tests passed
>
> Build Log:
> [...truncated 17393 lines...]
> [junit4:junit4] 2012-07-13 09:30:46
> [junit4:junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21
> mixed mode):
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994158800
> nid=0x766e waiting on condition [0x00007f1a70ca5000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> )
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994377000
> nid=0x766c waiting for monitor entry [0x00007f1a2dbda000]
> [junit4:junit4]    java.lang.Thread.State: BLOCKED (on object monitor)
> [junit4:junit4] at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1017)
> [junit4:junit4] - waiting to lock <0x00000000ea3a1c08> (a
> java.util.LinkedHashMap)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:195)
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f199435e000
> nid=0x766b waiting on condition [0x00007f1a2e2e1000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> )
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> [junit4:junit4]
> [junit4:junit4] "Thread-463" daemon prio=10 tid=0x00007f1994382000
> nid=0x7667 waiting on condition [0x00007f1a2cdcc000]
> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for  <0x00000000dc508208> (a
> java.util.concurrent.CountDownLatch$Sync)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> AbstractQueuedSynchronizer.java:834)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> ptibly(AbstractQueuedSynchronizer.java:994)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> bly(AbstractQueuedSynchronizer.java:1303)
> [junit4:junit4] at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQ
> ueue.java:185)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:291)
> [junit4:junit4] at
> org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProc
> essor.java:81)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]
> [junit4:junit4] "Thread-462" daemon prio=10 tid=0x00007f1994362000
> nid=0x7666 waiting on condition [0x00007f1a79706000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.Overseer$CloudStateUpdater.run(Overseer.java:153)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]
> [junit4:junit4] "qtp439061382-874" prio=10 tid=0x00007f19943ab000

I'll correct the runner to emit heartbeats even if stdout messages are
emitted from a test. I have been working on a major overhaul of test
timeouting too and it's looking good -- it should solve the problem of
hanging tests once and for all. It'll take me a few more days to get
it tested properly.

Dawid

On Fri, Jul 13, 2012 at 12:49 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> That's the build Dawid and me were analyzing live.... (open heart surgery)
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
>> -----Original Message-----
>> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, July 13, 2012 12:43 PM
>> To: [EMAIL PROTECTED]
>> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 570 - Failure!
>>
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-
>> 64/570/
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 17393 lines...]
>> [junit4:junit4] 2012-07-13 09:30:46
>> [junit4:junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21
>> mixed mode):
>> [junit4:junit4]
>> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994158800
>> nid=0x766e waiting on condition [0x00007f1a70ca5000]
>> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
>> [junit4:junit4]       at java.lang.Thread.sleep(Native Method)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
>> )
>> [junit4:junit4]       at
>> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
>> [junit4:junit4]
>> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994377000
>> nid=0x766c waiting for monitor entry [0x00007f1a2dbda000]
>> [junit4:junit4]    java.lang.Thread.State: BLOCKED (on object monitor)
>> [junit4:junit4]       at
>> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1017)
>> [junit4:junit4]       - waiting to lock <0x00000000ea3a1c08> (a
>> java.util.LinkedHashMap)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:195)
>> [junit4:junit4]
>> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f199435e000
>> nid=0x766b waiting on condition [0x00007f1a2e2e1000]
>> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
>> [junit4:junit4]       at java.lang.Thread.sleep(Native Method)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
>> )
>> [junit4:junit4]       at
>> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
>> [junit4:junit4]
>> [junit4:junit4] "Thread-463" daemon prio=10 tid=0x00007f1994382000
>> nid=0x7667 waiting on condition [0x00007f1a2cdcc000]
>> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
>> [junit4:junit4]       at sun.misc.Unsafe.park(Native Method)
>> [junit4:junit4]       - parking to wait for  <0x00000000dc508208> (a
>> java.util.concurrent.CountDownLatch$Sync)
>> [junit4:junit4]       at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
>> [junit4:junit4]       at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
>> AbstractQueuedSynchronizer.java:834)
>> [junit4:junit4]       at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
>> ptibly(AbstractQueuedSynchronizer.java:994)
>> [junit4:junit4]       at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
>> bly(AbstractQueuedSynchronizer.java:1303)
>> [junit4:junit4]       at
>> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQ
>> ueue.java:185)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:291)
>> [junit4:junit4]       at
>> org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProc

Thanks!

> I'll correct the runner to emit heartbeats even if stdout messages are
> emitted from a test. I have been working on a major overhaul of test
> timeouting too and it's looking good -- it should solve the problem of
> hanging tests once and for all. It'll take me a few more days to get
> it tested properly.

This helps in timeouting the tests, but it does not fix the issue of some tests itself going amok and hanging! :-)

> On Fri, Jul 13, 2012 at 12:49 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> > That's the build Dawid and me were analyzing live.... (open heart surgery)
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: [EMAIL PROTECTED]
> >
> >
> >> -----Original Message-----
> >> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> >> Sent: Friday, July 13, 2012 12:43 PM
> >> To: [EMAIL PROTECTED]
> >> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 570 - Failure!
> >>
> >> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-
> >> 64/570/
> >>
> >> All tests passed
> >>
> >> Build Log:
> >> [...truncated 17393 lines...]
> >> [junit4:junit4] 2012-07-13 09:30:46
> >> [junit4:junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-
> b21
> >> mixed mode):
> >> [junit4:junit4]
> >> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994158800
> >> nid=0x766e waiting on condition [0x00007f1a70ca5000]
> >> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> >> [junit4:junit4]       at java.lang.Thread.sleep(Native Method)
> >> [junit4:junit4]       at
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> >> )
> >> [junit4:junit4]       at
> >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> >> [junit4:junit4]
> >> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f1994377000
> >> nid=0x766c waiting for monitor entry [0x00007f1a2dbda000]
> >> [junit4:junit4]    java.lang.Thread.State: BLOCKED (on object monitor)
> >> [junit4:junit4]       at
> >> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1017)
> >> [junit4:junit4]       - waiting to lock <0x00000000ea3a1c08> (a
> >> java.util.LinkedHashMap)
> >> [junit4:junit4]       at
> >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:195)
> >> [junit4:junit4]
> >> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f199435e000
> >> nid=0x766b waiting on condition [0x00007f1a2e2e1000]
> >> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> >> [junit4:junit4]       at java.lang.Thread.sleep(Native Method)
> >> [junit4:junit4]       at
> >>
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:415
> >> )
> >> [junit4:junit4]       at
> >> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:210)
> >> [junit4:junit4]
> >> [junit4:junit4] "Thread-463" daemon prio=10 tid=0x00007f1994382000
> >> nid=0x7667 waiting on condition [0x00007f1a2cdcc000]
> >> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
> >> [junit4:junit4]       at sun.misc.Unsafe.park(Native Method)
> >> [junit4:junit4]       - parking to wait for  <0x00000000dc508208> (a
> >> java.util.concurrent.CountDownLatch$Sync)
> >> [junit4:junit4]       at
> >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> >> [junit4:junit4]       at
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> >> AbstractQueuedSynchronizer.java:834)
> >> [junit4:junit4]       at
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> >> ptibly(AbstractQueuedSynchronizer.java:994)
> >> [junit4:junit4]       at
> >>
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> >> bly(AbstractQueuedSynchronizer.java:1303)
> >> [junit4:junit4]       at
> >> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> >> [junit4:junit4]       at

> This helps in timeouting the tests, but it does not fix the issue of some tests itself going amok and hanging! :-)

Oh, absolutely. I'm the infrastructure guy -- I want tests to run and
fail with as much background information as possible so that folks who
know the rest of the system can fix them properly. :)

Dawid

---------------------------------------------------------------------

Maybe thats the problem? That its logging too much and causing OOM?
Just an idea.

On Fri, Jul 13, 2012 at 6:36 AM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
> http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
>
> Uh, these tests emit a lot of logs...
>
> Dawid
>
> On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
> <[EMAIL PROTECTED]> wrote:
>> I'll post a full stderr log in a minute.
>> D.
>>
>> On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
>>> We are currently investigating another hang. The next run also hung, it seems that a commit last night made the distributed tests hang forver.
>>>
>>> Dawid was decoding the test runner files (JSON), this is printed to stdout all the time (at 0% CPU):
>>>
>>> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error while trying to recover. org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:59584/solr
>>>
>>> Clearly a serious bug!
>>> Uwe
>>>
>>> -----
>>> Uwe Schindler
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>> http://www.thetaphi.de
>>> eMail: [EMAIL PROTECTED]
>>>
>>>
>>>> -----Original Message-----
>>>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
>>>> Dawid Weiss
>>>> Sent: Friday, July 13, 2012 10:10 AM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>>>>
>>>> > This one is interesting, as the VM was hanging in one test, but
>>>> randomizedrunner did not send heartbeats! It looks like this could be a bug in
>>>> randomizedrunner. But it was also hanging in Solr, approx. at the same time
>>>> like the otter one. Stack trace is in build log as Linux!
>>>>
>>>> Heartbeats are sent only if there are no events from the test -- it could be as
>>>> well that the test is writing something to stdout/stderr periodically? Now that I
>>>> think of it heartbeats should be sent even if the test is writing something to the
>>>> console but not responding within the given time window. What do you think?
>>>>
>>>> Dawid
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
>>>> commands, e-mail: [EMAIL PROTECTED]
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

--
lucidimagination.com

---------------------------------------------------------------------

It is possible.

Dawid

On Fri, Jul 13, 2012 at 1:45 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> Maybe thats the problem? That its logging too much and causing OOM?
> Just an idea.
>
> On Fri, Jul 13, 2012 at 6:36 AM, Dawid Weiss
> <[EMAIL PROTECTED]> wrote:
>> http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
>>
>> Uh, these tests emit a lot of logs...
>>
>> Dawid
>>
>> On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
>> <[EMAIL PROTECTED]> wrote:
>>> I'll post a full stderr log in a minute.
>>> D.
>>>
>>> On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
>>>> We are currently investigating another hang. The next run also hung, it seems that a commit last night made the distributed tests hang forver.
>>>>
>>>> Dawid was decoding the test runner files (JSON), this is printed to stdout all the time (at 0% CPU):
>>>>
>>>> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error while trying to recover. org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:59584/solr
>>>>
>>>> Clearly a serious bug!
>>>> Uwe
>>>>
>>>> -----
>>>> Uwe Schindler
>>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>> http://www.thetaphi.de
>>>> eMail: [EMAIL PROTECTED]
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
>>>>> Dawid Weiss
>>>>> Sent: Friday, July 13, 2012 10:10 AM
>>>>> To: [EMAIL PROTECTED]
>>>>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>>>>>
>>>>> > This one is interesting, as the VM was hanging in one test, but
>>>>> randomizedrunner did not send heartbeats! It looks like this could be a bug in
>>>>> randomizedrunner. But it was also hanging in Solr, approx. at the same time
>>>>> like the otter one. Stack trace is in build log as Linux!
>>>>>
>>>>> Heartbeats are sent only if there are no events from the test -- it could be as
>>>>> well that the test is writing something to stdout/stderr periodically? Now that I
>>>>> think of it heartbeats should be sent even if the test is writing something to the
>>>>> console but not responding within the given time window. What do you think?
>>>>>
>>>>> Dawid
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
>>>>> commands, e-mail: [EMAIL PROTECTED]
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
>
>
> --
> lucidimagination.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

Then we should see some OOM somewhere in a stack trace!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Friday, July 13, 2012 1:56 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>
> It is possible.
>
> Dawid
>
> On Fri, Jul 13, 2012 at 1:45 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> > Maybe thats the problem? That its logging too much and causing OOM?
> > Just an idea.
> >
> > On Fri, Jul 13, 2012 at 6:36 AM, Dawid Weiss
> > <[EMAIL PROTECTED]> wrote:
> >> http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
> >>
> >> Uh, these tests emit a lot of logs...
> >>
> >> Dawid
> >>
> >> On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
> >> <[EMAIL PROTECTED]> wrote:
> >>> I'll post a full stderr log in a minute.
> >>> D.
> >>>
> >>> On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> >>>> We are currently investigating another hang. The next run also hung, it
> seems that a commit last night made the distributed tests hang forver.
> >>>>
> >>>> Dawid was decoding the test runner files (JSON), this is printed to stdout
> all the time (at 0% CPU):
> >>>>
> >>>> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE
> Error while trying to recover. org.apache.solr.client.solrj.SolrServerException:
> Server refused connection at: http://127.0.0.1:59584/solr
> >>>>
> >>>> Clearly a serious bug!
> >>>> Uwe
> >>>>
> >>>> -----
> >>>> Uwe Schindler
> >>>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>>> http://www.thetaphi.de
> >>>> eMail: [EMAIL PROTECTED]
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
> Behalf Of
> >>>>> Dawid Weiss
> >>>>> Sent: Friday, July 13, 2012 10:10 AM
> >>>>> To: [EMAIL PROTECTED]
> >>>>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 -
> Failure!
> >>>>>
> >>>>> > This one is interesting, as the VM was hanging in one test, but
> >>>>> randomizedrunner did not send heartbeats! It looks like this could be a
> bug in
> >>>>> randomizedrunner. But it was also hanging in Solr, approx. at the same
> time
> >>>>> like the otter one. Stack trace is in build log as Linux!
> >>>>>
> >>>>> Heartbeats are sent only if there are no events from the test -- it could
> be as
> >>>>> well that the test is writing something to stdout/stderr periodically? Now
> that I
> >>>>> think of it heartbeats should be sent even if the test is writing something
> to the
> >>>>> console but not responding within the given time window. What do you
> think?
> >>>>>
> >>>>> Dawid
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] For
> additional
> >>>>> commands, e-mail: [EMAIL PROTECTED]
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >>>> For additional commands, e-mail: [EMAIL PROTECTED]
> >>>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >> For additional commands, e-mail: [EMAIL PROTECTED]
> >>
> >
> >
> >
> > --
> > lucidimagination.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

It is hanging again! Same test same issue! Everybody should review his commits from yesterday!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Friday, July 13, 2012 12:37 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>
> http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
>
> Uh, these tests emit a lot of logs...
>
> Dawid
>
> On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss <[EMAIL PROTECTED]>
> wrote:
> > I'll post a full stderr log in a minute.
> > D.
> >
> > On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> >> We are currently investigating another hang. The next run also hung, it
> seems that a commit last night made the distributed tests hang forver.
> >>
> >> Dawid was decoding the test runner files (JSON), this is printed to stdout all
> the time (at 0% CPU):
> >>
> >> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE Error
> >> while trying to recover.
> >> org.apache.solr.client.solrj.SolrServerException: Server refused
> >> connection at: http://127.0.0.1:59584/solr
> >>
> >> Clearly a serious bug!
> >> Uwe
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: [EMAIL PROTECTED]
> >>
> >>
> >>> -----Original Message-----
> >>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
> Behalf
> >>> Of Dawid Weiss
> >>> Sent: Friday, July 13, 2012 10:10 AM
> >>> To: [EMAIL PROTECTED]
> >>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 -
> Failure!
> >>>
> >>> > This one is interesting, as the VM was hanging in one test, but
> >>> randomizedrunner did not send heartbeats! It looks like this could
> >>> be a bug in randomizedrunner. But it was also hanging in Solr,
> >>> approx. at the same time like the otter one. Stack trace is in build log as
> Linux!
> >>>
> >>> Heartbeats are sent only if there are no events from the test -- it
> >>> could be as well that the test is writing something to stdout/stderr
> >>> periodically? Now that I think of it heartbeats should be sent even
> >>> if the test is writing something to the console but not responding within the
> given time window. What do you think?
> >>>
> >>> Dawid
> >>>
> >>> --------------------------------------------------------------------
> >>> - To unsubscribe, e-mail: [EMAIL PROTECTED] For
> >>> additional commands, e-mail: [EMAIL PROTECTED]
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [EMAIL PROTECTED] For
> >> additional commands, e-mail: [EMAIL PROTECTED]
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

> Then we should see some OOM somewhere in a stack trace!

We should. I mean should have. I've written a simple test that OOMs
and it does show up:

test:
    [mkdir] Created dir: c:\Work\lucene-solr\lucene\build\core\test
[junit4:junit4] <JUnit4> says g'day! Master seed: D82ABE551D56D0E3
[junit4:junit4] Your console's encoding may not display certain
unicode glyphs: windows-1252
[junit4:junit4] Executing 1 suite with 1 JVM.
[junit4:junit4] Suite: org.apache.lucene.TestOOM
[junit4:junit4] ERROR   0.32s | TestOOM.testOOM
[junit4:junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4:junit4]    >    at
__randomizedtesting.SeedInfo.seed([D82ABE551D56D0E3:330E9B19A0F2DFCD]:0)
[junit4:junit4]    >    at org.apache.lucene.TestOOM.testOOM(TestOOM.java:28)

I can't predict what's going to happen when the memory is nearly
consumed (and kept) from leaked background threads though -- if this
is the case then nothing's short of preallocating static data
structures is going to help...

Dawid

---------------------------------------------------------------------

...and I wanted to inform you it is hanging again in Solr! Who can help to solve this issue?
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java7-64/400/console

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
> Sent: Friday, July 13, 2012 2:12 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 - Failure!
>
> It is hanging again! Same test same issue! Everybody should review his commits
> from yesterday!
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
> > -----Original Message-----
> > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf
> > Of Dawid Weiss
> > Sent: Friday, July 13, 2012 12:37 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 -
> Failure!
> >
> > http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
> >
> > Uh, these tests emit a lot of logs...
> >
> > Dawid
> >
> > On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
> > <[EMAIL PROTECTED]>
> > wrote:
> > > I'll post a full stderr log in a minute.
> > > D.
> > >
> > > On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> > >> We are currently investigating another hang. The next run also
> > >> hung, it
> > seems that a commit last night made the distributed tests hang forver.
> > >>
> > >> Dawid was decoding the test runner files (JSON), this is printed to
> > >> stdout all
> > the time (at 0% CPU):
> > >>
> > >> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE
> > >> Error while trying to recover.
> > >> org.apache.solr.client.solrj.SolrServerException: Server refused
> > >> connection at: http://127.0.0.1:59584/solr
> > >>
> > >> Clearly a serious bug!
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> > >> eMail: [EMAIL PROTECTED]
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
> > Behalf
> > >>> Of Dawid Weiss
> > >>> Sent: Friday, July 13, 2012 10:10 AM
> > >>> To: [EMAIL PROTECTED]
> > >>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 -
> > Failure!
> > >>>
> > >>> > This one is interesting, as the VM was hanging in one test, but
> > >>> randomizedrunner did not send heartbeats! It looks like this could
> > >>> be a bug in randomizedrunner. But it was also hanging in Solr,
> > >>> approx. at the same time like the otter one. Stack trace is in build log as
> > Linux!
> > >>>
> > >>> Heartbeats are sent only if there are no events from the test -- it
> > >>> could be as well that the test is writing something to stdout/stderr
> > >>> periodically? Now that I think of it heartbeats should be sent even
> > >>> if the test is writing something to the console but not responding within
> the
> > given time window. What do you think?
> > >>>
> > >>> Dawid
> > >>>
> > >>> --------------------------------------------------------------------
> > >>> - To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > >>> additional commands, e-mail: [EMAIL PROTECTED]
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > >> additional commands, e-mail: [EMAIL PROTECTED]
> > >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> > commands, e-mail: [EMAIL PROTECTED]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
---------

See https://issues.apache.org/jira/browse/SOLR-3620

On Fri, Jul 13, 2012 at 11:46 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:

> ...and I wanted to inform you it is hanging again in Solr! Who can help to
> solve this issue?
>
> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java7-64/400/console
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
> > -----Original Message-----
> > From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, July 13, 2012 2:12 PM
> > To: [EMAIL PROTECTED]
> > Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240 -
> Failure!
> >
> > It is hanging again! Same test same issue! Everybody should review his
> commits
> > from yesterday!
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: [EMAIL PROTECTED]
> >
> >
> > > -----Original Message-----
> > > From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf
> > > Of Dawid Weiss
> > > Sent: Friday, July 13, 2012 12:37 PM
> > > To: [EMAIL PROTECTED]
> > > Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1240
> -
> > Failure!
> > >
> > > http://ophelia.cs.put.poznan.pl/~dweiss/lucene/output.zip
> > >
> > > Uh, these tests emit a lot of logs...
> > >
> > > Dawid
> > >
> > > On Fri, Jul 13, 2012 at 12:10 PM, Dawid Weiss
> > > <[EMAIL PROTECTED]>
> > > wrote:
> > > > I'll post a full stderr log in a minute.
> > > > D.
> > > >
> > > > On Fri, Jul 13, 2012 at 12:06 PM, Uwe Schindler <[EMAIL PROTECTED]>
> wrote:
> > > >> We are currently investigating another hang. The next run also
> > > >> hung, it
> > > seems that a commit last night made the distributed tests hang forver.
> > > >>
> > > >> Dawid was decoding the test runner files (JSON), this is printed to
> > > >> stdout all
> > > the time (at 0% CPU):
> > > >>
> > > >> 4476606 T886 C82 P42953 oasc.RecoveryStrategy.doRecovery SEVERE
> > > >> Error while trying to recover.
> > > >> org.apache.solr.client.solrj.SolrServerException: Server refused
> > > >> connection at: http://127.0.0.1:59584/solr
> > > >>
> > > >> Clearly a serious bug!
> > > >> Uwe
> > > >>
> > > >> -----
> > > >> Uwe Schindler
> > > >> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> > > >> eMail: [EMAIL PROTECTED]
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On
> > > Behalf
> > > >>> Of Dawid Weiss
> > > >>> Sent: Friday, July 13, 2012 10:10 AM
> > > >>> To: [EMAIL PROTECTED]
> > > >>> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build #
> 1240 -
> > > Failure!
> > > >>>
> > > >>> > This one is interesting, as the VM was hanging in one test, but
> > > >>> randomizedrunner did not send heartbeats! It looks like this could
> > > >>> be a bug in randomizedrunner. But it was also hanging in Solr,
> > > >>> approx. at the same time like the otter one. Stack trace is in
> build log as
> > > Linux!
> > > >>>
> > > >>> Heartbeats are sent only if there are no events from the test -- it
> > > >>> could be as well that the test is writing something to
> stdout/stderr
> > > >>> periodically? Now that I think of it heartbeats should be sent even
> > > >>> if the test is writing something to the console but not responding
> within
> > the
> > > given time window. What do you think?
> > > >>>
> > > >>> Dawid
> > > >>>
> > > >>>
> --------------------------------------------------------------------
> > > >>> - To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > > >>> additional commands, e-mail: [EMAIL PROTECTED]
> > > >>
> > > >>
> > > >>
> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > > >> additional commands, e-mail: [EMAIL PROTECTED]
> > > >>
> > >
> > > ----------------------------------------------------

- Mark

http://www.lucidimagination.com

I just committed what should hopefully fix this error in the solr
distrib test framework.
An AIOOB resulted when the last element of an array was skipped for comparison.

-Yonik
http://lucidimagination.com
On Fri, Jul 13, 2012 at 4:54 PM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java8-64/15/
>
> 1 tests failed.
> REGRESSION:  org.apache.solr.handler.component.DistributedQueryElevationComponentTest.testDistribSearch
>
> Error Message:
> Index: 18, Size: 18
>
> Stack Trace:
> java.lang.IndexOutOfBoundsException: Index: 18, Size: 18
>         at __randomizedtesting.SeedInfo.seed([5629B043F56D9EA8:D7CF3E5B8232FE94]:0)
>         at java.util.ArrayList.rangeCheck(ArrayList.java:603)
>         at java.util.ArrayList.get(ArrayList.java:381)
>         at org.apache.solr.common.util.NamedList.getName(NamedList.java:127)
>         at org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearchTestCase.java:466)
>         at org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearchTestCase.java:612)
>         at org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearchTestCase.java:500)
>         at org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearchTestCase.java:612)
>         at org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearchTestCase.java:500)
>         at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:667)
>         at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:392)
>         at org.apache.solr.handler.component.DistributedQueryElevationComponentTest.doTest(DistributedQueryElevationComponentTest.java:81)
>         at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:474)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)

Thanks for taking care. Interestingly this test was mostly failing in Java 8
EA builds. Strange, maybe there is something different with Java 8!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Yonik
> Seeley
> Sent: Saturday, July 14, 2012 12:29 PM
> To: Policeman Jenkins Server
> Cc: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 15 -
Failure!
>
> I just committed what should hopefully fix this error in the solr
> distrib test framework.
> An AIOOB resulted when the last element of an array was skipped for
> comparison.
>
> -Yonik
> http://lucidimagination.com
>
>
> On Fri, Jul 13, 2012 at 4:54 PM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
> > Build:
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java8-
> 64/15/
> >
> > 1 tests failed.
> > REGRESSION:
> org.apache.solr.handler.component.DistributedQueryElevationComponentTest.t
> estDistribSearch
> >
> > Error Message:
> > Index: 18, Size: 18
> >
> > Stack Trace:
> > java.lang.IndexOutOfBoundsException: Index: 18, Size: 18
> >         at
> __randomizedtesting.SeedInfo.seed([5629B043F56D9EA8:D7CF3E5B8232FE94]:
> 0)
> >         at java.util.ArrayList.rangeCheck(ArrayList.java:603)
> >         at java.util.ArrayList.get(ArrayList.java:381)
> >         at
> org.apache.solr.common.util.NamedList.getName(NamedList.java:127)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:466)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:612)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:500)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:612)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compare(BaseDistributedSearc
> hTestCase.java:500)
> >         at
> org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistrib
> utedSearchTestCase.java:667)
> >         at
>
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTe
> stCase.java:392)
> >         at
> org.apache.solr.handler.component.DistributedQueryElevationComponentTest.
> doTest(DistributedQueryElevationComponentTest.java:81)
> >         at
>
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistribu
t
> edSearchTestCase.java:679)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
> >         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:474)
> >         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1995)
> >         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
> >         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:818)
> >         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:877)
> >         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:891)
> >         at
>
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.eval
> uate(SystemPropertiesRestoreRule.java:53)
> >         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
> >         at
>
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
> >         at
>
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> >         at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.eval
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClass
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRule
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAsserti
o
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTes
t
java.util.ArrayList.rangeCheck(ArrayList.java:603)
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTe
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistribu
t
sun.reflect.NativeMethodAccessorImpl.invoke0(Native
java.lang.reflect.Method.invoke(Method.java:474)
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.eval
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
com.carrotsearch.randomizedtesting.rules.SystemProper

On Sat, Jul 14, 2012 at 6:37 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Thanks for taking care. Interestingly this test was mostly failing in Java 8
> EA builds. Strange, maybe there is something different with Java 8!

Yeah, there still could be something else wrong with the test or
elevate component itself - I just fixed the most obvious top-level
bug.
Or it could just be that something in Java8 causes orderings of some
container (like a Map) to be different.

-Yonik
http://lucidimagination.com

---------------------------------------------------------------------

> Or it could just be that something in Java8 causes orderings of some
container (like a Map) to be different.

That's definitely the case, also (partly in Java 7 for
HashSet/HashMaps/ConcurrentHashMap > 512 entries). They have now a better
has algorithm enabled by default. I already fixed some Solr tests doing
Map.toString() and compare against a hardcoded String (which is of course
wrong for HashMaps or HashSets because order is undefined).

After reading the mail below we were already thinking about setting the
threshold in Java 7 randomly :-)

Here the Mail on OpenJDK list:

---------- Forwarded message ----------
From: Mike Duigou <[EMAIL PROTECTED]>
Date: Sat, Jul 14, 2012 at 1:09 AM
Subject: Update on alternative hashing for String keys with hash-based Maps
To: core-libs-dev Libs <[EMAIL PROTECTED]>,
[EMAIL PROTECTED]
Hello all;

About a month ago a significant change was made to the Java SE 7 and 8 hash
based Map implementations. This change was previously proposed and discussed
on this list[1]. The change affects all of the hashing based Map
implementations (HashMap, Hashtable, WeakHashMap, LinkedHashMap,
ConcurrentHashMap), the derived Set implementations (HashSet, LinkedHashSet,
etc.) and other classes which depend upon these classes (Properties,
UIDefaults). The change to hash based Maps improves performance when a large
number of key hash code collisions are encountered. This is accomplished by
altering the handling of String keys to use a different (better) hash
function.

As initially proposed, the alternative hashing behaviour was planned to
apply to all hash based Maps larger than 512 elements. Smaller maps would
continue to use the existing hashing approach.
ConcurrentHashMap, because reasons related to the complexity of it's
implementation, would always use the improved approach regardless of map
capacity. Providing capacity based triggering of the alternative hashing is
intended to improve compatibility by ensuring that the order in which
elements are iterated does not change. Specifically, at less than the
threshold capacity Map elements will be iterated in the same order as today.
At or above the threshold, the iteration order will be different than the
current order. Testing and evaluation of existing Java applications has
shown us that some applications have explicit or implicit dependence upon
the order that keys, values or entries will be iterated. The vast majority
of iteration order dependent cases involve small maps--once a map contains
hundreds of elements generally incorrect assumptions about iteration order
will have already been found and resolved. We believed that triggering the
alternative hashing behaviour at 512 element capacity would protect
iteration order in cases which depended upon the existing iteration order.
Java SE 7 and Java SE 8 have different policies. Java SE 8 is intended to
always use alternative hashing of String keys regardless of the capacity of
the Map.

After  integration a number of cases of iteration order dependence were
encountered within the OpenJDK code itself, in tests and in user code. Some
of these faults were easily diagnosed and correct. Some other cases, because
iteration order is not consistent from run-to-run under alternative hashing,
proved difficult to isolate and resolve.

Following this testing and and consultation with Java licensees and
developers it was decided disable the alternative hashing behaviour for Java
SE 7.  To ensure the greatest degree of compatibility with existing
applications it seems best to only enable alternative hashing by explicit
control. In Java SE 7u6 it will be necessary to set the system property
jdk.map.althashing.threshold in order to enable alternative hashing. It is
also still possible that the feature may be enabled by default in future
Java SE 7 releases but this will only happen if further testing indicates
compatibility can be reasonably assured.

Because Java 8 is unreleased and we still wish to shake out iteration order
dependencies alternative hashing remains enabled in Java SE 8.
Alternative hashing is very likely to be enabled by default in Java SE 8.
Doug Lea has been investigating further improvements to handling of key hash
code collisions and his design is extremely likely to be the basis for all
Java SE 8 hash based Map implementations.[2]

Developers and deployers are strongly encouraged to test their applications
by enabling the alternative String key hashing feature in Java SE 7u6 or
later and/or testing with Java SE 8 builds. BE WARNED:
it will probably not be possible to disable alternative hashing in Java SE
8. Applications MUST remove dependencies upon iteration order before they
can be deployed with Java SE 8.

Thanks,

Mike

TL;DR:
- Java SE 7 and 8 both now support alternative hashing for String keys with
hash based Maps
- Alternative hashing improves performance when many String key hash codes
collide
- Alternative hashing impacts key, value and element iteration order
- Alternative hashing is currently DISABLED by default for Java SE 7
- Future Java SE 7 releases may enable alternative hashing for "large"
(>512 capacity) maps
- Developers can enable the eature in Java SE 7 for testing and deployment
with a system property
- Alternative hashing is ENABLED for all maps in Java SE 8
- It will probably not be possible to disable alternative hashing in Java SE
8
- Hash map key, value and element iteration order WILL be different and
unpredictable in Java SE 8
- Different implementation approaches are still being investigated for Java
SE 8 and remain subject to change

[1]
http://mail.openjdk.java.net/pipermail/core-libs-dev/2012-May/010238.html
[2]
http://cs.oswego.edu/pipermail/concurrency-interest/2012-June/009505.html

Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
Failure!
8!
container

"... tests doing Map.toString() and compare against a hardcoded String
(which is of course wrong for HashMaps or HashSets because order is
undefined)."

Map.toString sounds like a great operation to forbid in Solr/Lucene,
especially in tests. Although Map.toString is still helpful for
debug/logging, what would really be helpful are two things: 1) display the
map sorted/ordered by key, and 2) compare two maps for equality (build a map
to compare against rather than using a presumed toString value.) And, an
"assertMapEquals" method as well. Maybe even a "assertMapKeys" method that
simply verifies that the keys of a map are "equal" to a list of keys (set
equality but not order.)

-- Jack Krupansky

-----Original Message-----
From: Uwe Schindler
Sent: Saturday, July 14, 2012 5:51 AM
To: [EMAIL PROTECTED] ; [EMAIL PROTECTED]
Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 15 -
Failure!

> Or it could just be that something in Java8 causes orderings of some
container (like a Map) to be different.

That's definitely the case, also (partly in Java 7 for
HashSet/HashMaps/ConcurrentHashMap > 512 entries). They have now a better
has algorithm enabled by default. I already fixed some Solr tests doing
Map.toString() and compare against a hardcoded String (which is of course
wrong for HashMaps or HashSets because order is undefined).

After reading the mail below we were already thinking about setting the
threshold in Java 7 randomly :-)

Here the Mail on OpenJDK list:

---------- Forwarded message ----------
From: Mike Duigou <[EMAIL PROTECTED]>
Date: Sat, Jul 14, 2012 at 1:09 AM
Subject: Update on alternative hashing for String keys with hash-based Maps
To: core-libs-dev Libs <[EMAIL PROTECTED]>,
[EMAIL PROTECTED]
Hello all;

About a month ago a significant change was made to the Java SE 7 and 8 hash
based Map implementations. This change was previously proposed and discussed
on this list[1]. The change affects all of the hashing based Map
implementations (HashMap, Hashtable, WeakHashMap, LinkedHashMap,
ConcurrentHashMap), the derived Set implementations (HashSet, LinkedHashSet,
etc.) and other classes which depend upon these classes (Properties,
UIDefaults). The change to hash based Maps improves performance when a large
number of key hash code collisions are encountered. This is accomplished by
altering the handling of String keys to use a different (better) hash
function.

As initially proposed, the alternative hashing behaviour was planned to
apply to all hash based Maps larger than 512 elements. Smaller maps would
continue to use the existing hashing approach.
ConcurrentHashMap, because reasons related to the complexity of it's
implementation, would always use the improved approach regardless of map
capacity. Providing capacity based triggering of the alternative hashing is
intended to improve compatibility by ensuring that the order in which
elements are iterated does not change. Specifically, at less than the
threshold capacity Map elements will be iterated in the same order as today.
At or above the threshold, the iteration order will be different than the
current order. Testing and evaluation of existing Java applications has
shown us that some applications have explicit or implicit dependence upon
the order that keys, values or entries will be iterated. The vast majority
of iteration order dependent cases involve small maps--once a map contains
hundreds of elements generally incorrect assumptions about iteration order
will have already been found and resolved. We believed that triggering the
alternative hashing behaviour at 512 element capacity would protect
iteration order in cases which depended upon the existing iteration order.
Java SE 7 and Java SE 8 have different policies. Java SE 8 is intended to
always use alternative hashing of String keys regardless of the capacity of
the Map.

After  integration a number of cases of iteration order dependence were
encountered within the OpenJDK code itself, in tests and in user code. Some
of these faults were easily diagnosed and correct. Some other cases, because
iteration order is not consistent from run-to-run under alternative hashing,
proved difficult to isolate and resolve.

Following this testing and and consultation with Java licensees and
developers it was decided disable the alternative hashing behaviour for Java
SE 7.  To ensure the greatest degree of compatibility with existing
applications it seems best to only enable alternative hashing by explicit
control. In Java SE 7u6 it will be necessary to set the system property
jdk.map.althashing.threshold in order to enable alternative hashing. It is
also still possible that the feature may be enabled by default in future
Java SE 7 releases but this will only happen if further testing indicates
compatibility can be reasonably assured.

Because Java 8 is unreleased and we still wish to shake out iteration order
dependencies alternative hashing remains enabled in Java SE 8.
Alternative hashing is very likely to be enabled by default in Java SE 8.
Doug Lea has been investigating further improvements to handling of key hash
code collisions and his design is extremely likely to be the basis for all
Java SE 8 hash based Map implementations.[2]

Developers and deployers are strongly encouraged to test their applications
by enabling the alternative String key hashing feature in Java SE 7u6 or
later and/or testing with Java SE 8 builds. BE WARNED:
it will probably not be possible to disable alternative hashing in Java SE
8. Applications MUST remove dependencies upon iteration order before they
can be deployed with Java SE 8.

Thanks,

Mike

TL;DR:
- Java SE 7 and 8 both now support alternative hashing for String keys with
hash based Maps
- Alternative hashing improves performance when many String key hash codes
collide
- Alternative hashing impacts key, value and element iteration order
- Alternative hashing is currently DISABL

Hi,

 
> "... tests doing Map.toString() and compare against a hardcoded String
(which
> is of course wrong for HashMaps or HashSets because order is undefined)."
>
> Map.toString sounds like a great operation to forbid in Solr/Lucene,
especially
> in tests. Although Map.toString is still helpful for debug/logging, what
would
> really be helpful are two things: 1) display the map sorted/ordered by
key, and
> 2) compare two maps for equality (build a map to compare against rather
than
> using a presumed toString value.) And, an "assertMapEquals" method as
well.
> Maybe even a "assertMapKeys" method that simply verifies that the keys of
a
> map are "equal" to a list of keys (set equality but not order.)

assertMapEquals can be done with assertEquals easily, you just have to pass
a full map as comparison base:

assertEquals("map differs", new HashMap() {{ map.put(...);....}},
someMapToTest);

For Sets it is much easier:

assertEquals("set differs", new HashSet(Arrays.asList(<items>)),
someSetToTest);

The semantics of Map/Set.equals() (see interface docs) explicitely specify
that any type of Map or Set must compare against another one, so you can
compare a TreeMap against a HashMap for equality.

If you want to compare by String and you only have a HashMap/HashSet, the
trick is:

assertEquals("map differs", "{items....}", new
TreeMap(someMapToTest));

By that you enforce order (with cloning the map, but that's not a perf
problem in tests).

Uwe
---------------------------------------------------------------------

On Sat, Jul 14, 2012 at 6:51 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:>
> That's definitely the case, also (partly in Java 7 for
> HashSet/HashMaps/ConcurrentHashMap > 512 entries). They have now a better
> has algorithm enabled by default. I already fixed some Solr tests doing
> Map.toString() and compare against a hardcoded String

Yeah, certainly a bug unless the map is ordered.  I wasn't aware of
anything like that anywhere.

-Yonik
http://lucidimagination.com

---------------------------------------------------------------------

Some typo in last example (toString() missing):

> If you want to compare by String and you only have a HashMap/HashSet, the
> trick is:
>
>  assertEquals("map differs", "{items....}", new
TreeMap(someMapToTest).toString());

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, July 14, 2012 2:20 PM
> To: [EMAIL PROTECTED]
> Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 15 -
Failure!
>
> Hi,
>
>
> > "... tests doing Map.toString() and compare against a hardcoded String
> (which
> > is of course wrong for HashMaps or HashSets because order is
undefined)."
> >
> > Map.toString sounds like a great operation to forbid in Solr/Lucene,
> especially
> > in tests. Although Map.toString is still helpful for debug/logging,
> > what
> would
> > really be helpful are two things: 1) display the map sorted/ordered by
> key, and
> > 2) compare two maps for equality (build a map to compare against
> > rather
> than
> > using a presumed toString value.) And, an "assertMapEquals" method as
> well.
> > Maybe even a "assertMapKeys" method that simply verifies that the keys
> > of
> a
> > map are "equal" to a list of keys (set equality but not order.)
>
> assertMapEquals can be done with assertEquals easily, you just have to
pass a
> full map as comparison base:
>
> assertEquals("map differs", new HashMap() {{ map.put(...);....}},
> someMapToTest);
>
> For Sets it is much easier:
>
> assertEquals("set differs", new HashSet(Arrays.asList(<items>)),
> someSetToTest);
>
> The semantics of Map/Set.equals() (see interface docs) explicitely specify
that
> any type of Map or Set must compare against another one, so you can
compare
> a TreeMap against a HashMap for equality.
>
> If you want to compare by String and you only have a HashMap/HashSet, the
> trick is:
>
> assertEquals("map differs", "{items....}", new
> TreeMap(someMapToTest));
>
> By that you enforce order (with cloning the map, but that's not a perf
problem
> in tests).
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

We use (and like) fest asserts. The syntax is nice but what's even
nicer are formatted exception messages for arrays, maps, etc. so you
get the idea what the mismatch was.

http://code.google.com/p/fest/

see fluid assertions; or here a new 2.x line of development:
https://github.com/alexruiz/fest-assert-2.x/wiki

Dawid

On Sat, Jul 14, 2012 at 2:20 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Hi,
>
>
>> "... tests doing Map.toString() and compare against a hardcoded String
> (which
>> is of course wrong for HashMaps or HashSets because order is undefined)."
>>
>> Map.toString sounds like a great operation to forbid in Solr/Lucene,
> especially
>> in tests. Although Map.toString is still helpful for debug/logging, what
> would
>> really be helpful are two things: 1) display the map sorted/ordered by
> key, and
>> 2) compare two maps for equality (build a map to compare against rather
> than
>> using a presumed toString value.) And, an "assertMapEquals" method as
> well.
>> Maybe even a "assertMapKeys" method that simply verifies that the keys of
> a
>> map are "equal" to a list of keys (set equality but not order.)
>
> assertMapEquals can be done with assertEquals easily, you just have to pass
> a full map as comparison base:
>
>         assertEquals("map differs", new HashMap() {{ map.put(...);....}},
> someMapToTest);
>
> For Sets it is much easier:
>
>         assertEquals("set differs", new HashSet(Arrays.asList(<items>)),
> someSetToTest);
>
> The semantics of Map/Set.equals() (see interface docs) explicitely specify
> that any type of Map or Set must compare against another one, so you can
> compare a TreeMap against a HashMap for equality.
>
> If you want to compare by String and you only have a HashMap/HashSet, the
> trick is:
>
>         assertEquals("map differs", "{items....}", new
> TreeMap(someMapToTest));
>
> By that you enforce order (with cloning the map, but that's not a perf
> problem in tests).
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

I like that, nice builder pattern *duck* ! (running away)

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Saturday, July 14, 2012 2:44 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 15 - Failure!
>
> We use (and like) fest asserts. The syntax is nice but what's even nicer are
> formatted exception messages for arrays, maps, etc. so you get the idea what
> the mismatch was.
>
> http://code.google.com/p/fest/
>
> see fluid assertions; or here a new 2.x line of development:
> https://github.com/alexruiz/fest-assert-2.x/wiki
>
> Dawid
>
> On Sat, Jul 14, 2012 at 2:20 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> >
> >> "... tests doing Map.toString() and compare against a hardcoded
> >> String
> > (which
> >> is of course wrong for HashMaps or HashSets because order is undefined)."
> >>
> >> Map.toString sounds like a great operation to forbid in Solr/Lucene,
> > especially
> >> in tests. Although Map.toString is still helpful for debug/logging,
> >> what
> > would
> >> really be helpful are two things: 1) display the map sorted/ordered
> >> by
> > key, and
> >> 2) compare two maps for equality (build a map to compare against
> >> rather
> > than
> >> using a presumed toString value.) And, an "assertMapEquals" method as
> > well.
> >> Maybe even a "assertMapKeys" method that simply verifies that the
> >> keys of
> > a
> >> map are "equal" to a list of keys (set equality but not order.)
> >
> > assertMapEquals can be done with assertEquals easily, you just have to
> > pass a full map as comparison base:
> >
> >         assertEquals("map differs", new HashMap() {{
> > map.put(...);....}}, someMapToTest);
> >
> > For Sets it is much easier:
> >
> >         assertEquals("set differs", new
> > HashSet(Arrays.asList(<items>)), someSetToTest);
> >
> > The semantics of Map/Set.equals() (see interface docs) explicitely
> > specify that any type of Map or Set must compare against another one,
> > so you can compare a TreeMap against a HashMap for equality.
> >
> > If you want to compare by String and you only have a HashMap/HashSet,
> > the trick is:
> >
> >         assertEquals("map differs", "{items....}", new
> > TreeMap(someMapToTest));
> >
> > By that you enforce order (with cloning the map, but that's not a perf
> > problem in tests).
> >
> > Uwe
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED] For
> > additional commands, e-mail: [EMAIL PROTECTED]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------

Formatted exception messages for non-scalar values is highly desirable, if
not essential. Not just that a map/list does not match, but how it doesn't
match - missing keys, extra keys, which key values differ (which can be a
recursive process).

-- Jack Krupansky

-----Original Message-----
From: Dawid Weiss
Sent: Saturday, July 14, 2012 7:43 AM
To: [EMAIL PROTECTED]
Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux-Java8-64 - Build # 15 -
Failure!

We use (and like) fest asserts. The syntax is nice but what's even
nicer are formatted exception messages for arrays, maps, etc. so you
get the idea what the mismatch was.

http://code.google.com/p/fest/

see fluid assertions; or here a new 2.x line of development:
https://github.com/alexruiz/fest-assert-2.x/wiki

Dawid

On Sat, Jul 14, 2012 at 2:20 PM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Hi,
>
>
>> "... tests doing Map.toString() and compare against a hardcoded String
> (which
>> is of course wrong for HashMaps or HashSets because order is undefined)."
>>
>> Map.toString sounds like a great operation to forbid in Solr/Lucene,
> especially
>> in tests. Although Map.toString is still helpful for debug/logging, what
> would
>> really be helpful are two things: 1) display the map sorted/ordered by
> key, and
>> 2) compare two maps for equality (build a map to compare against rather
> than
>> using a presumed toString value.) And, an "assertMapEquals" method as
> well.
>> Maybe even a "assertMapKeys" method that simply verifies that the keys of
> a
>> map are "equal" to a list of keys (set equality but not order.)
>
> assertMapEquals can be done with assertEquals easily, you just have to
> pass
> a full map as comparison base:
>
>         assertEquals("map differs", new HashMap() {{ map.put(...);....}},
> someMapToTest);
>
> For Sets it is much easier:
>
>         assertEquals("set differs", new HashSet(Arrays.asList(<items>)),
> someSetToTest);
>
> The semantics of Map/Set.equals() (see interface docs) explicitely specify
> that any type of Map or Set must compare against another one, so you can
> compare a TreeMap against a HashMap for equality.
>
> If you want to compare by String and you only have a HashMap/HashSet, the
> trick is:
>
>         assertEquals("map differs", "{items....}", new
> TreeMap(someMapToTest));
>
> By that you enforce order (with cloning the map, but that's not a perf
> problem in tests).
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------
---------------------------------------------------------------------

Mark,

It seems that since your last Solr commit we are back at hangs (16 hrs doing nothing)!

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Sunday, July 15, 2012 7:48 PM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 588 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-
> 64/588/
>
> All tests passed
>
> Build Log:
> [...truncated 16896 lines...]
> [junit4:junit4] 2012-07-15 17:47:22
> [junit4:junit4] Full thread dump Java HotSpot(TM) 64-Bit Server VM (23.0-b21
> mixed mode):
> [junit4:junit4]
> [junit4:junit4] "RecoveryThread" daemon prio=10 tid=0x00007f90f0622800
> nid=0x606c waiting for monitor entry [0x00007f90d9e1f000]
> [junit4:junit4]    java.lang.Thread.State: BLOCKED (on object monitor)
> [junit4:junit4] at
> org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1018)
> [junit4:junit4] - waiting to lock <0x00000000de8c1330> (a
> java.util.LinkedHashMap)
> [junit4:junit4] at
> org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:195)
> [junit4:junit4]
> [junit4:junit4] "Thread-453" daemon prio=10 tid=0x00007f90f0213000
> nid=0x6064 waiting on condition [0x00007f90d8a0b000]
> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for  <0x00000000e63347d8> (a
> java.util.concurrent.CountDownLatch$Sync)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> AbstractQueuedSynchronizer.java:834)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> ptibly(AbstractQueuedSynchronizer.java:994)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> bly(AbstractQueuedSynchronizer.java:1303)
> [junit4:junit4] at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQ
> ueue.java:185)
> [junit4:junit4] at
> org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:291)
> [junit4:junit4] at
> org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProc
> essor.java:81)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]
> [junit4:junit4] "Thread-452" daemon prio=10 tid=0x00007f90f0072800
> nid=0x6063 waiting on condition [0x00007f90d8708000]
> [junit4:junit4]    java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at
> org.apache.solr.cloud.Overseer$CloudStateUpdater.run(Overseer.java:153)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]
> [junit4:junit4] "Thread-451" daemon prio=10 tid=0x00007f90f05ce800
> nid=0x6059 waiting on condition [0x00007f90d65e7000]
> [junit4:junit4]    java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for  <0x00000000e6334938> (a
> java.util.concurrent.CountDownLatch$Sync)
> [junit4:junit4] at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
> AbstractQueuedSynchronizer.java:834)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterru
> ptibly(AbstractQueuedSynchronizer.java:994)
> [junit4:junit4] at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterrupti
> bly(AbstractQueuedSynchronizer.java:1303)
> [junit4:junit4] at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)

Sorry, my fault!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Sunday, July 15, 2012 7:54 PM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java6-64 - Build # 1267 - Still
> Failing!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-
> 64/1267/
>
> No tests ran.
>
> Build Log:
> [...truncated 244 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-
> 64/checkout/build.xml:55: The following error occurred while executing this
> line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-
> 64/checkout/lucene/build.xml:170: The following error occurred while
> executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java6-
> 64/checkout/lucene/tools/custom-tasks.xml:26: License check failed. Check the
> logs.
>
> Total time: 4 seconds
> Build step 'Execute shell' marked build as failure Archiving artifacts Recording
> test results Email was triggered for: Failure Sending email for trigger: Failure
>
>

---------------------------------------------------------------------

This looks like a gremlin in DirectPostingsFormat (LUCENE-4227), its
retrieving the wrong bytes for the payload

On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>
> Error Message:
> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>
> Stack Trace:
> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at org.junit.Assert.failNotEquals(Assert.java:647)
>         at org.junit.Assert.assertEquals(Assert.java:128)
>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)

lucidimagination.com

I'll dig.

Mike McCandless

http://blog.mikemccandless.com
On Sat, Jul 21, 2012 at 6:42 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> This looks like a gremlin in DirectPostingsFormat (LUCENE-4227), its
> retrieving the wrong bytes for the payload
>
> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>
>> Error Message:
>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>
>> Stack Trace:
>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>>         at org.junit.Assert.fail(Assert.java:93)
>>         at org.junit.Assert.failNotEquals(Assert.java:647)
>>         at org.junit.Assert.assertEquals(Assert.java:128)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)

Test passes if i do this:

    this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;

If i wire it to Integer.MAX_VALUE, easier tests fail like
TestPayloads.testPayloadsEncoding

On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>
> Error Message:
> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>
> Stack Trace:
> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at org.junit.Assert.failNotEquals(Assert.java:647)
>         at org.junit.Assert.assertEquals(Assert.java:128)
>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)

lucidimagination.com

OK that helps, thanks!

Mike McCandless

http://blog.mikemccandless.com

On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> Test passes if i do this:
>
>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>
> If i wire it to Integer.MAX_VALUE, easier tests fail like
> TestPayloads.testPayloadsEncoding
>
> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>
>> Error Message:
>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>
>> Stack Trace:
>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>>         at org.junit.Assert.fail(Assert.java:93)
>>         at org.junit.Assert.failNotEquals(Assert.java:647)
>>         at org.junit.Assert.assertEquals(Assert.java:128)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>

We should see if the new TestPostingsFormat fails with that set to
Integer.MAX_VALUE too!

On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
<[EMAIL PROTECTED]> wrote:
> OK that helps, thanks!
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>> Test passes if i do this:
>>
>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>
>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>> TestPayloads.testPayloadsEncoding
>>
>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
>> <[EMAIL PROTECTED]> wrote:
>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>>
>>> 1 tests failed.
>>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>>
>>> Error Message:
>>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>
>>> Stack Trace:
>>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>>>         at org.junit.Assert.fail(Assert.java:93)
>>>         at org.junit.Assert.failNotEquals(Assert.java:647)
>>>         at org.junit.Assert.assertEquals(Assert.java:128)
>>>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>>>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>>>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)

lucidimagination.com

I want to do that!  It lives only on the block branch for now ...

Mike McCandless

http://blog.mikemccandless.com

On Sat, Jul 21, 2012 at 7:13 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> We should see if the new TestPostingsFormat fails with that set to
> Integer.MAX_VALUE too!
>
> On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
> <[EMAIL PROTECTED]> wrote:
>> OK that helps, thanks!
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>> Test passes if i do this:
>>>
>>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>>
>>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>>> TestPayloads.testPayloadsEncoding
>>>
>>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
>>> <[EMAIL PROTECTED]> wrote:
>>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>>>
>>>> 1 tests failed.
>>>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>>>
>>>> Error Message:
>>>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>>
>>>> Stack Trace:
>>>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>>>>         at org.junit.Assert.fail(Assert.java:93)
>>>>         at org.junit.Assert.failNotEquals(Assert.java:647)
>>>>         at org.junit.Assert.assertEquals(Assert.java:128)
>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>>>>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>>>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)

Maybe after this bug gets resolved we can merge it over? I just svn
copied it to trunk as a test, and it compiles and seems to work.

Seems like a good test to have sooner than later...

On Sat, Jul 21, 2012 at 7:17 AM, Michael McCandless
<[EMAIL PROTECTED]> wrote:
> I want to do that!  It lives only on the block branch for now ...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sat, Jul 21, 2012 at 7:13 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>> We should see if the new TestPostingsFormat fails with that set to
>> Integer.MAX_VALUE too!
>>
>> On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
>> <[EMAIL PROTECTED]> wrote:
>>> OK that helps, thanks!
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>> Test passes if i do this:
>>>>
>>>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>>>
>>>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>>>> TestPayloads.testPayloadsEncoding
>>>>
>>>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>>>>
>>>>> 1 tests failed.
>>>>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>>>>
>>>>> Error Message:
>>>>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>>>
>>>>> Stack Trace:
>>>>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>>>         at __randomizedtesting.SeedInfo.seed([F217680F0F1AA27:B4E251B3EC8FA25]:0)
>>>>>         at org.junit.Assert.fail(Assert.java:93)
>>>>>         at org.junit.Assert.failNotEquals(Assert.java:647)
>>>>>         at org.junit.Assert.assertEquals(Assert.java:128)
>>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertDocsAndPositionsEnum(TestDuelingCodecs.java:416)
>>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTermsEnum(TestDuelingCodecs.java:334)
>>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertTerms(TestDuelingCodecs.java:213)
>>>>>         at org.apache.lucene.index.TestDuelingCodecs.assertFields(TestDuelingCodecs.java:182)
>>>>>         at org.apache.lucene.index.TestDuelingCodecs.testEquals(TestDuelingCodecs.java:143)
>>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>>>>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>
lucidimagination.com

I give up on this thing for now. but the nextDoc() in LowFreqD&PEnum
looks bogus if payloads are present.

I tried this (doesnt work), but i still feel like this is on the right path:

Index: src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java
==================================================================--- src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java (revision
1364061)
+++ src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java (working
copy)
@@ -95,7 +95,7 @@
   public DirectPostingsFormat(int minSkipCount, int lowFreqCutoff) {
     super("Direct");
     this.minSkipCount = minSkipCount;
-    this.lowFreqCutoff = lowFreqCutoff;
+    this.lowFreqCutoff = Integer.MAX_VALUE; // nocommit lowFreqCutoff;
   }

   @Override
@@ -1741,7 +1741,17 @@
             skipPositions = freq;
             return docID;
           }
-          upto += posMult * freq;
+          if (hasPayloads) {
+            for(int i=0;i<freq;i++) {
+              upto++;
+              if (hasOffsets) {
+                upto += 2;
+              }
+              payloadOffset += postings[upto++];
+            }
+          } else {
+            upto += posMult * freq;
+          }
         }
       }
On Sat, Jul 21, 2012 at 7:19 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> Maybe after this bug gets resolved we can merge it over? I just svn
> copied it to trunk as a test, and it compiles and seems to work.
>
> Seems like a good test to have sooner than later...
>
> On Sat, Jul 21, 2012 at 7:17 AM, Michael McCandless
> <[EMAIL PROTECTED]> wrote:
>> I want to do that!  It lives only on the block branch for now ...
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Sat, Jul 21, 2012 at 7:13 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>> We should see if the new TestPostingsFormat fails with that set to
>>> Integer.MAX_VALUE too!
>>>
>>> On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
>>> <[EMAIL PROTECTED]> wrote:
>>>> OK that helps, thanks!
>>>>
>>>> Mike McCandless
>>>>
>>>> http://blog.mikemccandless.com
>>>>
>>>> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>>> Test passes if i do this:
>>>>>
>>>>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>>>>
>>>>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>>>>> TestPayloads.testPayloadsEncoding
>>>>>
>>>>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>>>>>
>>>>>> 1 tests failed.
>>>>>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>>>>>
>>>>>> Error Message:
>>>>>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>
>>>>>>
>>>>>> Stack Trace:
>>>>>> java.lang.AssertionError: left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>

lucidimagination.com

OK: actually my fix  is correct, there are 2 bugs

1. this bug fixes scanning over deleted documents for the payloads
case: it fixes TestDuelingCodecs
2. the second bug (TestPayloads) is this line:

                      // Just to ensure all codecs can
                      // handle a caller that mucks with the
                      // returned payload:
                      if (rarely()) {
                        br.bytes = new byte[random().nextInt(5)];
                      }

DirectPF is returning a BytesRef accessing its inner byte[], so this
totally screws it up. It needs to arraycopy if this is how the
behavior should be.

On Sat, Jul 21, 2012 at 7:50 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> I give up on this thing for now. but the nextDoc() in LowFreqD&PEnum
> looks bogus if payloads are present.
>
> I tried this (doesnt work), but i still feel like this is on the right path:
>
> Index: src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java
> ==================================================================> --- src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java  (revision
> 1364061)
> +++ src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java  (working
> copy)
> @@ -95,7 +95,7 @@
>    public DirectPostingsFormat(int minSkipCount, int lowFreqCutoff) {
>      super("Direct");
>      this.minSkipCount = minSkipCount;
> -    this.lowFreqCutoff = lowFreqCutoff;
> +    this.lowFreqCutoff = Integer.MAX_VALUE; // nocommit lowFreqCutoff;
>    }
>
>    @Override
> @@ -1741,7 +1741,17 @@
>              skipPositions = freq;
>              return docID;
>            }
> -          upto += posMult * freq;
> +          if (hasPayloads) {
> +            for(int i=0;i<freq;i++) {
> +              upto++;
> +              if (hasOffsets) {
> +                upto += 2;
> +              }
> +              payloadOffset += postings[upto++];
> +            }
> +          } else {
> +            upto += posMult * freq;
> +          }
>          }
>        }
>
>
> On Sat, Jul 21, 2012 at 7:19 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>> Maybe after this bug gets resolved we can merge it over? I just svn
>> copied it to trunk as a test, and it compiles and seems to work.
>>
>> Seems like a good test to have sooner than later...
>>
>> On Sat, Jul 21, 2012 at 7:17 AM, Michael McCandless
>> <[EMAIL PROTECTED]> wrote:
>>> I want to do that!  It lives only on the block branch for now ...
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>> On Sat, Jul 21, 2012 at 7:13 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>> We should see if the new TestPostingsFormat fails with that set to
>>>> Integer.MAX_VALUE too!
>>>>
>>>> On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> OK that helps, thanks!
>>>>>
>>>>> Mike McCandless
>>>>>
>>>>> http://blog.mikemccandless.com
>>>>>
>>>>> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>>>> Test passes if i do this:
>>>>>>
>>>>>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>>>>>
>>>>>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>>>>>> TestPayloads.testPayloadsEncoding
>>>>>>
>>>>>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/655/
>>>>>>>
>>>>>>> 1 tests failed.
>>>>>>> REGRESSION:  org.apache.lucene.index.TestDuelingCodecs.testEquals
>>>>>>>
>>>>>>> Error Message:
>>>>>>> left: SimpleText / right: Lucene40: {docid=MockVariableIntBlock(baseBlockSize=116), body=PostingsFormat(name=Direct), title=PostingsFormat(name=NestedPulsing), titleTokenized=PostingsFormat(name=MockSep), date=PostingsFormat(name=Direct)} expected:<[c6 f9 d6 13 53 c2 83 62 83 bf 59 e7 a2 f2 95 e7 f2 cf 9f 73 96 f4 6d e1 b7 bc 74 47 b5 a6 90 65 e3 69 43 f7 77 69 b8 d4 9f 35 27 24 2d f5 e 7b da b 4e 5d 9c f8 1c cc b3 26 a3]> but was:<[cf 79 c fd fb e4 49 ea 16 a6 87 4e 6b 3d 7 31 20 93 42 1a 9f d5 63 f8 59 30 96 8f 35 6c d1 60 a3 f ba 38 a6 bb c4 f2 31 8b ba a7 57 f7 47 f5 aa db 94 49 7 94 2e 6e 93 6c 25]>

lucidimagination.com

I committed fixes for this, but things we should do:

1. Add checks to TestPostingsFormat that payloads work correctly when
scanning over deleted documents, and change the values in the returned
BytesRef itself from getPayload (bytes/offset/length) to ensure
callers actually set all 3 of these in getPayload.
2. we should probably also document in DocumentsAndPositionsEnum that
you should not change the actual *bytes* pointed to by the payloads
bytesref (unless we want to add arraycopies in these PFs that return
slices of some inner datastructure).

On Sat, Jul 21, 2012 at 8:01 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> OK: actually my fix  is correct, there are 2 bugs
>
> 1. this bug fixes scanning over deleted documents for the payloads
> case: it fixes TestDuelingCodecs
> 2. the second bug (TestPayloads) is this line:
>
>                       // Just to ensure all codecs can
>                       // handle a caller that mucks with the
>                       // returned payload:
>                       if (rarely()) {
>                         br.bytes = new byte[random().nextInt(5)];
>                       }
>
> DirectPF is returning a BytesRef accessing its inner byte[], so this
> totally screws it up. It needs to arraycopy if this is how the
> behavior should be.
>
> On Sat, Jul 21, 2012 at 7:50 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>> I give up on this thing for now. but the nextDoc() in LowFreqD&PEnum
>> looks bogus if payloads are present.
>>
>> I tried this (doesnt work), but i still feel like this is on the right path:
>>
>> Index: src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java
>> ==================================================================>> --- src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java  (revision
>> 1364061)
>> +++ src/java/org/apache/lucene/codecs/memory/DirectPostingsFormat.java  (working
>> copy)
>> @@ -95,7 +95,7 @@
>>    public DirectPostingsFormat(int minSkipCount, int lowFreqCutoff) {
>>      super("Direct");
>>      this.minSkipCount = minSkipCount;
>> -    this.lowFreqCutoff = lowFreqCutoff;
>> +    this.lowFreqCutoff = Integer.MAX_VALUE; // nocommit lowFreqCutoff;
>>    }
>>
>>    @Override
>> @@ -1741,7 +1741,17 @@
>>              skipPositions = freq;
>>              return docID;
>>            }
>> -          upto += posMult * freq;
>> +          if (hasPayloads) {
>> +            for(int i=0;i<freq;i++) {
>> +              upto++;
>> +              if (hasOffsets) {
>> +                upto += 2;
>> +              }
>> +              payloadOffset += postings[upto++];
>> +            }
>> +          } else {
>> +            upto += posMult * freq;
>> +          }
>>          }
>>        }
>>
>>
>> On Sat, Jul 21, 2012 at 7:19 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>> Maybe after this bug gets resolved we can merge it over? I just svn
>>> copied it to trunk as a test, and it compiles and seems to work.
>>>
>>> Seems like a good test to have sooner than later...
>>>
>>> On Sat, Jul 21, 2012 at 7:17 AM, Michael McCandless
>>> <[EMAIL PROTECTED]> wrote:
>>>> I want to do that!  It lives only on the block branch for now ...
>>>>
>>>> Mike McCandless
>>>>
>>>> http://blog.mikemccandless.com
>>>>
>>>> On Sat, Jul 21, 2012 at 7:13 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>>> We should see if the new TestPostingsFormat fails with that set to
>>>>> Integer.MAX_VALUE too!
>>>>>
>>>>> On Sat, Jul 21, 2012 at 7:12 AM, Michael McCandless
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> OK that helps, thanks!
>>>>>>
>>>>>> Mike McCandless
>>>>>>
>>>>>> http://blog.mikemccandless.com
>>>>>>
>>>>>> On Sat, Jul 21, 2012 at 7:08 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
>>>>>>> Test passes if i do this:
>>>>>>>
>>>>>>>     this.lowFreqCutoff = 1; // nocommit lowFreqCutoff;
>>>>>>>
>>>>>>> If i wire it to Integer.MAX_VALUE, easier tests fail like
>>>>>>> TestPayloads.testPayloadsEncoding
>>>>>>>
>>>>>>> On Sat, Jul 21, 2012 at 5:43 AM, Policeman Jenkins Server

lucidimagination.com

On Sat, Jul 21, 2012 at 8:15 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> I committed fixes for this, but things we should do:
>
> 1. Add checks to TestPostingsFormat that payloads work correctly when
> scanning over deleted documents, and change the values in the returned
> BytesRef itself from getPayload (bytes/offset/length) to ensure
> callers actually set all 3 of these in getPayload.

OK I confirmed (after some silly fixes) TestPostingsFormat would have
caught this bug (separately it also caught a reuse bug in SimpleText)
... so we'll have that coverage once we merge back to trunk.

> 2. we should probably also document in DocumentsAndPositionsEnum that
> you should not change the actual *bytes* pointed to by the payloads
> bytesref (unless we want to add arraycopies in these PFs that return
> slices of some inner datastructure).

I committed a fix to D&PEnum.getPayload stating this.

Thanks Robert!

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------

#666... Couldn't be any different.
On Jul 22, 2012 10:08 AM, "Policeman Jenkins Server" <
[EMAIL PROTECTED]> wrote:

> Build:
> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/666/
>
> 8 tests failed.
> REGRESSION:
>  org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_replace_nodelete
>
> Error Message:
> Exception during query
>
> Stack Trace:
> java.lang.RuntimeException: Exception during query
>         at
> __randomizedtesting.SeedInfo.seed([35621805D30986AD:B81829D4E867C647]:0)
>         at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:486)
>         at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:453)
>         at
> org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.add1document(TestSqlEntityProcessorDelta2.java:85)
>         at
> org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_replace_nodelete(TestSqlEntityProcessorDelta2.java:178)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)

Disable the test. Clearly no one cares about fixing it
On Jul 22, 2012 5:01 AM, "Dawid Weiss" <[EMAIL PROTECTED]> wrote:

> #666... Couldn't be any different.
> On Jul 22, 2012 10:08 AM, "Policeman Jenkins Server" <
> [EMAIL PROTECTED]> wrote:
>
>> Build:
>> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/666/
>>
>> 8 tests failed.
>> REGRESSION:
>>  org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_replace_nodelete
>>
>> Error Message:
>> Exception during query
>>
>> Stack Trace:
>> java.lang.RuntimeException: Exception during query
>>         at
>> __randomizedtesting.SeedInfo.seed([35621805D30986AD:B81829D4E867C647]:0)
>>         at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:486)
>>         at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:453)
>>         at
>> org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.add1document(TestSqlEntityProcessorDelta2.java:85)
>>         at
>> org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta2.testCompositePk_DeltaImport_replace_nodelete(TestSqlEntityProcessorDelta2.java:178)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:601)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>>         at
>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>>         at
>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>         at
>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>>         at
>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>         at
>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>>         at
>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>>         at
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>         at
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
>>         at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>>         at
>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>>         at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)

maybe java8 does not like minimizeSchindler... :)

On Tue, Jul 24, 2012 at 2:35 AM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java8-64/25/
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.util.automaton.TestMinimize.testMinimizeHuge
>
> Error Message:
> Java heap space
>
> Stack Trace:
> java.lang.OutOfMemoryError: Java heap space
>         at __randomizedtesting.SeedInfo.seed([DBD5BB4010DA2DDB:E72F6468824539BA]:0)
>         at java.util.ArrayList.<init>(ArrayList.java:132)
>         at java.util.ArrayList.<init>(ArrayList.java:139)
>         at org.apache.lucene.util.automaton.MinimizationOperations.minimizeHopcroft(MinimizationOperations.java:91)
>         at org.apache.lucene.util.automaton.MinimizationOperations.minimize(MinimizationOperations.java:54)
>         at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:502)
>         at org.apache.lucene.util.automaton.RegExp.toAutomatonAllowMutate(RegExp.java:478)
>         at org.apache.lucene.util.automaton.RegExp.toAutomaton(RegExp.java:428)
>         at org.apache.lucene.util.automaton.TestMinimize.testMinimizeHuge(TestMinimize.java:55)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:474)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>
>
>
>
> Build Log:
> [...truncated 758 lines...]
> [junit4:junit4] Suite: org.apache.lucene.util.automaton.TestMinimize
> [junit4:junit4] ERROR   9.68s J1 | TestMinimize.testMinimizeHuge
> [junit4:junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
> [junit4:junit4]    >    at __randomizedtesting.SeedInfo.seed([DBD5BB4010DA2DDB:E72F6468824539BA]:0)

lucidimagination.com

Fix coming shortly.

On Jul 24, 2012, at 2:55 AM, Policeman Jenkins Server wrote:

> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java6-64/1385/
>
> 1 tests failed.
> REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch
>
> Error Message:
>
>
> Stack Trace:
> java.lang.NullPointerException
> at __randomizedtesting.SeedInfo.seed([F98C6C87F6501642:786AE29F810F767E]:0)
> at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:162)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
> at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)

Shit ... my bad .. i forgot about those nocommit refactors.

on it.

: Date: Fri, 27 Jul 2012 00:03:37 +0000 (UTC)
: From: Policeman Jenkins Server <[EMAIL PROTECTED]>
: Reply-To: [EMAIL PROTECTED]
: To: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED],
:     [EMAIL PROTECTED], [EMAIL PROTECTED]
: Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 715 - Failure!
:
: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/715/
:
: No tests ran.
:
: Build Log:
: [...truncated 166 lines...]
:
:
:

-Hoss

---------------------------------------------------------------------

Sorry, fixed this. The problem in test was that path was not absolute in this configuration.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Saturday, July 28, 2012 9:56 PM
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 737 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-
> 64/737/
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.analysis.util.TestFilesystemResourceLoader.testBaseDir
>
> Error Message:
> Resource not found: ./fsResourceLoaderBase4682246822tmp/template.txt
>
> Stack Trace:
> java.io.IOException: Resource not found:
> ./fsResourceLoaderBase4682246822tmp/template.txt
> at
> __randomizedtesting.SeedInfo.seed([AD5802E6257518AC:9771B46C53531694]:
> 0)
> at
> org.apache.lucene.analysis.util.ClasspathResourceLoader.openResource(Classp
> athResourceLoader.java:67)
> at
> org.apache.lucene.analysis.util.FilesystemResourceLoader.openResource(Filesy
> stemResourceLoader.java:86)
> at
> org.apache.lucene.analysis.util.TestFilesystemResourceLoader.testBaseDir(Test
> FilesystemResourceLoader.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1995)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:818)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:877)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:891)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea
> dAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
> oreAfterMaxFailures.java:70)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
> va:48)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando
> mizedRunner.java:825)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(Randomiz
> edRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(Randomized
> Runner.java:671)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Randomiz
> edRunner.java:697)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Randomiz
> edRunner.java:736)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Randomiz
> edRunner.java:747)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClass
> Name.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev

I disabled Javadocs checking with IBM J9 Java 6 edition (the problem is: We compile Javadocs against the Oracle docs but those are incompatible to IBMs variant)

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Monday, July 30, 2012 9:28 PM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk6) - Build # 37 -
> Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/37/
> Java: 64bit/ibm-j9-jdk6
>
> All tests passed
>
> Build Log:
> [...truncated 10223 lines...]
> javadocs-lint:
>
> [...truncated 1688 lines...]
> javadocs-lint:
>      [exec]
>      [exec] Crawl/parse...
>      [exec]
>      [exec]   build/docs/core/org/apache/lucene/store/package-use.html
>      [exec]     WARNING: anchor "../../../../org/apache/lucene/store/subclasses"
> appears more than once
>      [exec]
>      [exec] Verify...
>      [exec]
>      [exec] build/docs/misc/org/apache/lucene/misc/class-
> use/SweetSpotSimilarity.html
>      [exec]   BROKEN LINK: build/docs/misc/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/misc/serialized-form.html
>      [exec]
>      [exec]
> build/docs/grouping/org/apache/lucene/search/grouping/term/TermDistinctVa
> luesCollector.html
>      [exec]   BROKEN LINK: build/docs/grouping/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/grouping/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-common/org/tartarus/snowball/ext/class-
> use/HungarianStemmer.html
>      [exec]   BROKEN LINK: build/docs/analyzers-common/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-common/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-
> common/org/apache/lucene/analysis/util/CharArrayMap.EntryIterator.html
>      [exec]   BROKEN LINK: build/docs/analyzers-common/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-common/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-
> phonetic/org/apache/lucene/analysis/phonetic/package-use.html
>      [exec]   BROKEN LINK: build/docs/analyzers-phonetic/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-phonetic/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-icu/overview-summary.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]
>      [exec]
> build/docs/suggest/org/apache/lucene/search/suggest/fst/Sort.ByteSequences
> Reader.html
>      [exec]   BROKEN LINK: build/docs/suggest/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/suggest/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-
> uima/org/apache/lucene/analysis/uima/ae/AEProvider.html
>      [exec]   BROKEN LINK: build/docs/analyzers-uima/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-uima/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-icu/org/apache/lucene/collation/package-
> summary.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-
> icu/org/apache/lucene/analysis/icu/ICUTransformFilter.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/analyzers-icu/serialized-form.html
>      [exec]
>      [exec] build/docs/grouping/org/apache/lucene/search/grouping/class-
> use/GroupingSearch.html
>      [exec]   BROKEN LINK: build/docs/grouping/serialized-form.html
>      [exec]   BROKEN LINK: build/docs/grouping/serialized-form.html
>      [exec]
>      [exec] build/docs/analyzers-
> stempel/org/apache/lucene/analysis/stempel/package-use.html

fixed: toLowerCase() -> toLowerCase(Locale.ROOT)

: Date: Tue, 31 Jul 2012 00:45:45 +0000 (UTC)
: From: Policeman Jenkins Server <[EMAIL PROTECTED]>
: Reply-To: [EMAIL PROTECTED]
: To: [EMAIL PROTECTED], [EMAIL PROTECTED]
: Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_05) - Build # 43 -
:     Failure!
:
: Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/43/
: Java: 32bit/jdk1.7.0_05 -server -XX:+UseG1GC
:
: All tests passed
:
: Build Log:
: [...truncated 19566 lines...]
: BUILD FAILED
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/checkout/build.xml:55: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/checkout/solr/build.xml:197: Check for forbidden API calls failed, see log.
:
: Total time: 29 seconds
: Build step 'Execute shell' marked build as failure
: Archiving artifacts
: Recording test results
: Email was triggered for: Failure
: Sending email for trigger: Failure
:
:
:

-Hoss

---------------------------------------------------------------------

We have again JRockit JVM bug, I will wait for the 32 bit version - just to see if happens there, too. Robert tested this on his CPU, no failure, on this Intel(R) Xeon(R) X3440  @ 2.53GHz it is 100% reproducible with any seed with no repetitions (in 64 bit JRockit JVM, 32 bit not yet tested). I will later try on Windows locally with other CPU. If it passes there, it looks like a processor-specific buggy optimization in JRockit, which is altogether slower on Lucene/Solr tests than Hotspot.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, July 31, 2012 6:18 AM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jrockit-jdk1.6.0_33-R28.2.4-
> 4.1.0) - Build # 47 - Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/47/
> Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0
>
> 1 tests failed.
> REGRESSION:
> org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
> at
> __randomizedtesting.SeedInfo.seed([E06157DBE3C945D:7B16FD3A0BDAE520]:
> 0)
> at
> org.apache.lucene.index.FreqProxTermsWriterPerField.writeOffsets(FreqProxT
> ermsWriterPerField.java:157)
> at
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTerm
> sWriterPerField.java:241)
> at
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
> at
> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField
> .java:167)
> at
> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcess
> or.java:305)
> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(Docum
> entsWriterPerThread.java:243)
> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter
> .java:375)
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1357)
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1108)
> at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> er.java:186)
> at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> er.java:146)
> at
> org.apache.lucene.index.TestPostingsOffsets.checkTokens(TestPostingsOffsets.j
> ava:491)
> at
> org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets(TestPostings
> Offsets.java:436)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1995)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:819)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:891)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea

Hi,

I did some further testing with JRockit:

- I was wrong, the problem is not easy to reproduce on my server; it behaves identical to what Robert tested on his machine. It might be some lucky time when I tried for the last time. So I don't think it's processor specific.
- You have to run the whole lucene-core test suite, otherwise it does not fail. It is also important to limit to 1 or 2 parallel JVMs, otherwise the running JDKs do too less to trigger the bug
- Use -Dtests.multiplier=3, otherwise it does not always reproduce
- The test fails with 32 and 64 bit.
- It cannot be verified without optimizer, as JRockit has no -Xint or any other way to disable optimizer

To reproduce, use the following (in lucene/core):

$ JAVA_HOME=/path/to/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 ant -Dtests.jvms=2 -Dtests.multiplier=3 -Dtests.seed=E0BB421A03A7B634 test

It also fails without this seed; this is just to be sure!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, July 31, 2012 8:38 AM
> To: [EMAIL PROTECTED]
> Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jrockit-jdk1.6.0_33-
> R28.2.4-4.1.0) - Build # 47 - Failure!
>
> We have again JRockit JVM bug, I will wait for the 32 bit version - just to see if
> happens there, too. Robert tested this on his CPU, no failure, on this Intel(R)
> Xeon(R) X3440  @ 2.53GHz it is 100% reproducible with any seed with no
> repetitions (in 64 bit JRockit JVM, 32 bit not yet tested). I will later try on
> Windows locally with other CPU. If it passes there, it looks like a processor-
> specific buggy optimization in JRockit, which is altogether slower on
> Lucene/Solr tests than Hotspot.
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
> > -----Original Message-----
> > From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, July 31, 2012 6:18 AM
> > To: [EMAIL PROTECTED]
> > Subject: [JENKINS] Lucene-Solr-trunk-Linux
> > (64bit/jrockit-jdk1.6.0_33-R28.2.4-
> > 4.1.0) - Build # 47 - Failure!
> >
> > Build:
> > http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/47/
> > Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0
> >
> > 1 tests failed.
> > REGRESSION:
> > org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets
> >
> > Error Message:
> >
> >
> > Stack Trace:
> > java.lang.AssertionError
> > at
> >
> __randomizedtesting.SeedInfo.seed([E06157DBE3C945D:7B16FD3A0BDAE520]:
> > 0)
> > at
> > org.apache.lucene.index.FreqProxTermsWriterPerField.writeOffsets(FreqP
> > roxT
> > ermsWriterPerField.java:157)
> > at
> > org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTe
> > rm
> > sWriterPerField.java:241)
> > at
> >
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
> > at
> > org.apache.lucene.index.DocInverterPerField.processFields(DocInverterP
> > erField
> > .java:167)
> > at
> > org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProc
> > ess
> > or.java:305)
> > at
> >
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(Docum
> > entsWriterPerThread.java:243)
> > at
> >
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter
> > .java:375)
> > at
> >
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1357)
> > at
> > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1108)
> > at
> >
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> > er.java:186)
> > at
> >
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> > er.java:146)
> > at
> > org.apache.lucene.index.TestPostingsOffsets.checkTokens(TestPostingsOf
> > fsets.j
> > ava:491)
> > at
> > org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets(TestP

I was interested in reproducing this, but i never could.

I don't think its too big a deal as people can just "not use jrockit"
and use a regular jdk instead.

We should come up with a set of patterns for broken JVMS that can
cause index corruption, detect them in e.g. Constants.java and throw
ThreadDeath and refuse to execute at all!

On Tue, Jul 31, 2012 at 4:56 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I did some further testing with JRockit:
>
> - I was wrong, the problem is not easy to reproduce on my server; it behaves identical to what Robert tested on his machine. It might be some lucky time when I tried for the last time. So I don't think it's processor specific.
> - You have to run the whole lucene-core test suite, otherwise it does not fail. It is also important to limit to 1 or 2 parallel JVMs, otherwise the running JDKs do too less to trigger the bug
> - Use -Dtests.multiplier=3, otherwise it does not always reproduce
> - The test fails with 32 and 64 bit.
> - It cannot be verified without optimizer, as JRockit has no -Xint or any other way to disable optimizer
>
> To reproduce, use the following (in lucene/core):
>
> $ JAVA_HOME=/path/to/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 ant -Dtests.jvms=2 -Dtests.multiplier=3 -Dtests.seed=E0BB421A03A7B634 test
>
> It also fails without this seed; this is just to be sure!
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: [EMAIL PROTECTED]
>
>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
>> Sent: Tuesday, July 31, 2012 8:38 AM
>> To: [EMAIL PROTECTED]
>> Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jrockit-jdk1.6.0_33-
>> R28.2.4-4.1.0) - Build # 47 - Failure!
>>
>> We have again JRockit JVM bug, I will wait for the 32 bit version - just to see if
>> happens there, too. Robert tested this on his CPU, no failure, on this Intel(R)
>> Xeon(R) X3440  @ 2.53GHz it is 100% reproducible with any seed with no
>> repetitions (in 64 bit JRockit JVM, 32 bit not yet tested). I will later try on
>> Windows locally with other CPU. If it passes there, it looks like a processor-
>> specific buggy optimization in JRockit, which is altogether slower on
>> Lucene/Solr tests than Hotspot.
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: [EMAIL PROTECTED]
>>
>>
>> > -----Original Message-----
>> > From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
>> > Sent: Tuesday, July 31, 2012 6:18 AM
>> > To: [EMAIL PROTECTED]
>> > Subject: [JENKINS] Lucene-Solr-trunk-Linux
>> > (64bit/jrockit-jdk1.6.0_33-R28.2.4-
>> > 4.1.0) - Build # 47 - Failure!
>> >
>> > Build:
>> > http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/47/
>> > Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0
>> >
>> > 1 tests failed.
>> > REGRESSION:
>> > org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets
>> >
>> > Error Message:
>> >
>> >
>> > Stack Trace:
>> > java.lang.AssertionError
>> >     at
>> >
>> __randomizedtesting.SeedInfo.seed([E06157DBE3C945D:7B16FD3A0BDAE520]:
>> > 0)
>> >     at
>> > org.apache.lucene.index.FreqProxTermsWriterPerField.writeOffsets(FreqP
>> > roxT
>> > ermsWriterPerField.java:157)
>> >     at
>> > org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTe
>> > rm
>> > sWriterPerField.java:241)
>> >     at
>> >
>> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
>> >     at
>> > org.apache.lucene.index.DocInverterPerField.processFields(DocInverterP
>> > erField
>> > .java:167)
>> >     at
>> > org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProc
>> > ess
>> > or.java:305)
>> >     at
>> >
>> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(Docum
>> > entsWriterPerThread.java:243)
>> >     at
>> >
>> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter
>> > .java:375)

lucidimagination.com

On Tue, Jul 31, 2012 at 7:24 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> We should come up with a set of patterns for broken JVMS that can
> cause index corruption, detect them in e.g. Constants.java and throw
> ThreadDeath and refuse to execute at all!

That seems extreme.  It's also likely to get out of date (i.e. if a
vendor fixes a bug, we can't go back and "fix" an earlier release to
not refuse to execute.)

Documentation seems like the best option.

-Yonik
http://lucidimagination.com

---------------------------------------------------------------------

> On Tue, Jul 31, 2012 at 7:24 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> > We should come up with a set of patterns for broken JVMS that can
> > cause index corruption, detect them in e.g. Constants.java and throw
> > ThreadDeath and refuse to execute at all!
>
> That seems extreme.  It's also likely to get out of date (i.e. if a vendor
fixes a
> bug, we can't go back and "fix" an earlier release to not refuse to
execute.)

I agree with Yonik. That was the first thing that came in my mind when
thinking about a hard failure.

On the other hand, we know for sure that every Java 7 before/including
1.7.0-147 is broken with any version of Lucene, we can deny execution. We
just need some patterns on sysprops and maybe a external config file in
resources folder with pattern + message to display.

> Documentation seems like the best option.

I agree, but some well known *exact versions* could still be denied to
execute.
---------------------------------------------------------------------

Hi,
 
> Hi,
>
> I did some further testing with JRockit:
>
> - I was wrong, the problem is not easy to reproduce on my server; it behaves
> identical to what Robert tested on his machine. It might be some lucky time
> when I tried for the last time. So I don't think it's processor specific.
> - You have to run the whole lucene-core test suite, otherwise it does not fail. It
> is also important to limit to 1 or 2 parallel JVMs, otherwise the running JDKs do
> too less to trigger the bug
> - Use -Dtests.multiplier=3, otherwise it does not always reproduce
> - The test fails with 32 and 64 bit.
> - It cannot be verified without optimizer, as JRockit has no -Xint or any other
> way to disable optimizer
>
> To reproduce, use the following (in lucene/core):
>
> $ JAVA_HOME=/path/to/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 ant -Dtests.jvms=2 -
> Dtests.multiplier=3 -Dtests.seed=E0BB421A03A7B634 test
>
> It also fails without this seed; this is just to be sure!

Dawid and I found another option that slows down JRockit only a little bit, but fixes the bug (tests with below command take 15 instead of 13 minutes): -XnoOpt

$ JAVA_HOME=/path/to/jrockit-jdk1.6.0_33-R28.2.4-4.1.0 ant -Dtests.jvms=2 -Dtests.multiplier=3 -Dtests.seed=E0BB421A03A7B634 -Dargs="-XnoOpt" test

The slowdown is minimal, as noOpt only disables second level optimizations (like in Hotspot). -Xint does not work because the code is compiled to assembly, but with -XnoOpt not optimized. JRockit has no interpreter at all.

Uwe

> > -----Original Message-----
> > From: Uwe Schindler [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, July 31, 2012 8:38 AM
> > To: [EMAIL PROTECTED]
> > Subject: RE: [JENKINS] Lucene-Solr-trunk-Linux
> > (64bit/jrockit-jdk1.6.0_33-
> > R28.2.4-4.1.0) - Build # 47 - Failure!
> >
> > We have again JRockit JVM bug, I will wait for the 32 bit version -
> > just to see if happens there, too. Robert tested this on his CPU, no
> > failure, on this Intel(R)
> > Xeon(R) X3440  @ 2.53GHz it is 100% reproducible with any seed with no
> > repetitions (in 64 bit JRockit JVM, 32 bit not yet tested). I will
> > later try on Windows locally with other CPU. If it passes there, it
> > looks like a processor- specific buggy optimization in JRockit, which
> > is altogether slower on Lucene/Solr tests than Hotspot.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: [EMAIL PROTECTED]
> >
> >
> > > -----Original Message-----
> > > From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> > > Sent: Tuesday, July 31, 2012 6:18 AM
> > > To: [EMAIL PROTECTED]
> > > Subject: [JENKINS] Lucene-Solr-trunk-Linux
> > > (64bit/jrockit-jdk1.6.0_33-R28.2.4-
> > > 4.1.0) - Build # 47 - Failure!
> > >
> > > Build:
> > > http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/47/
> > > Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0
> > >
> > > 1 tests failed.
> > > REGRESSION:
> > > org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets
> > >
> > > Error Message:
> > >
> > >
> > > Stack Trace:
> > > java.lang.AssertionError
> > > at
> > >
> >
> __randomizedtesting.SeedInfo.seed([E06157DBE3C945D:7B16FD3A0BDAE520]:
> > > 0)
> > > at
> > > org.apache.lucene.index.FreqProxTermsWriterPerField.writeOffsets(Fre
> > > qP
> > > roxT
> > > ermsWriterPerField.java:157)
> > > at
> > > org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProx
> > > Te
> > > rm
> > > sWriterPerField.java:241)
> > > at
> > >
> > org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:2
> > 35)
> > > at
> > > org.apache.lucene.index.DocInverterPerField.processFields(DocInverte
> > > rP
> > > erField
> > > .java:167)
> > > at
> > > org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldPr
> > > oc
> > > ess
> > > or.java:305)
> > > at
> > >
> >
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(Docum
> > > entsWriterPerThread.java:243)

I tried to reproduce the whole test run with my IBM JRE (which is not
the java7 one Uwe has, but still):

java version "1.6.0"
Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
J9VM - 20111207_096808
JIT  - r9_20111107_21307ifx1
GC   - 20110519_AA)
JCL  - 20111104_02

This just caused out of open file handles... I think mine has other issues :)

I cant reproduce the problems with any other JRE, it seems sketchy
this long is negative.

On Tue, Jul 31, 2012 at 7:24 PM, Policeman Jenkins Server
<[EMAIL PROTECTED]> wrote:
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/67/
> Java: 32bit/ibm-j9-jdk7
>
> 7 tests failed.
> FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestFieldCache
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([D686D14F0587EBAE]:0)
>         at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:70)
>         at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:83)
>         at org.apache.lucene.util.fst.FST.pack(FST.java:1505)
>         at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:273)
>         at org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:550)
>         at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
>         at org.apache.lucene.index.TermsHash.flush(TermsHash.java:117)
>         at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
>         at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:82)
>         at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:481)
>         at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:420)
>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:552)
>         at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2563)
>         at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2699)
>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2679)
>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2663)
>         at org.apache.lucene.index.RandomIndexWriter.maybeCommit(RandomIndexWriter.java:266)
>         at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
>         at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)
>         at org.apache.lucene.search.TestFieldCache.beforeClass(TestFieldCache.java:100)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
>         at java.lang.reflect.Method.invoke(Method.java:613)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:731)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>         at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)

lucidimagination.com

I looked into this more, including downloading the latest IBM JRE that
Uwe is using.

The test that always tends runs me out of file handles (only with ibm
jre) is TestShardSearching. I think this is because it opens several
directories. one for each index shard.

some experiments (r1367800), FAIL indicates fail due to too-many open
files, PASS indicates test passes:
FAIL: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
-Dtests.directory=NIOFSDirectory -Dtests.seed=4F026DAD227E08DB
FAIL: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
-Dtests.directory=SimpleFSDirectory -Dtests.seed=4F026DAD227E08DB
PASS: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
-Dtests.directory=MMapDirectory -Dtests.seed=4F026DAD227E08DB

in fact if i just pick seeds at random each time, with -Dtests.multiplier=3:
5 runs of MMAP: pass 5/5
5 runs of SimpleFS: pass 1/5
5 runs of NIOFS: pass 2/5

I can't help but think something is up here with SimpleFS/NIOFS: ive
had this problem with lucene and IBM jres for a very long time: see
https://issues.apache.org/jira/browse/LUCENE-2873 open a year and a
half ago.

I'm going to look into it and see if I can find anything

On Tue, Jul 31, 2012 at 9:10 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> I tried to reproduce the whole test run with my IBM JRE (which is not
> the java7 one Uwe has, but still):
>
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
> jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> J9VM - 20111207_096808
> JIT  - r9_20111107_21307ifx1
> GC   - 20110519_AA)
> JCL  - 20111104_02
>
> This just caused out of open file handles... I think mine has other issues :)
>
> I cant reproduce the problems with any other JRE, it seems sketchy
> this long is negative.
>
> On Tue, Jul 31, 2012 at 7:24 PM, Policeman Jenkins Server
> <[EMAIL PROTECTED]> wrote:
>> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/67/
>> Java: 32bit/ibm-j9-jdk7
>>
>> 7 tests failed.
>> FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestFieldCache
>>
>> Error Message:
>>
>>
>> Stack Trace:
>> java.lang.AssertionError
>>         at __randomizedtesting.SeedInfo.seed([D686D14F0587EBAE]:0)
>>         at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:70)
>>         at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:83)
>>         at org.apache.lucene.util.fst.FST.pack(FST.java:1505)
>>         at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:273)
>>         at org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:550)
>>         at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
>>         at org.apache.lucene.index.TermsHash.flush(TermsHash.java:117)
>>         at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
>>         at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:82)
>>         at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:481)
>>         at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:420)
>>         at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:552)
>>         at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2563)
>>         at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2699)
>>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2679)
>>         at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2663)
>>         at org.apache.lucene.index.RandomIndexWriter.maybeCommit(RandomIndexWriter.java:266)
>>         at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
>>         at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:145)

lucidimagination.com

Hi Robert,

I checked Jenkin's settings and the "ulimit -n" is 8192 by default for jenkins. To prevent this problem I raised this to 32768.
The thing with IBM J9 is that is has several caches for class files (so compiled class files can be cached in shared memory for parallel JVMs using the same classes), but I assume this needs more file handles.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Robert Muir [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, August 01, 2012 4:11 AM
> To: [EMAIL PROTECTED]
> Subject: Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build # 67 -
> Failure!
>
> I looked into this more, including downloading the latest IBM JRE that
> Uwe is using.
>
> The test that always tends runs me out of file handles (only with ibm
> jre) is TestShardSearching. I think this is because it opens several
> directories. one for each index shard.
>
> some experiments (r1367800), FAIL indicates fail due to too-many open
> files, PASS indicates test passes:
> FAIL: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
> -Dtests.directory=NIOFSDirectory -Dtests.seed=4F026DAD227E08DB
> FAIL: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
> -Dtests.directory=SimpleFSDirectory -Dtests.seed=4F026DAD227E08DB
> PASS: ant -Dtests.multiplier=3 test -Dtestcase=TestShardSearching
> -Dtests.directory=MMapDirectory -Dtests.seed=4F026DAD227E08DB
>
> in fact if i just pick seeds at random each time, with -Dtests.multiplier=3:
> 5 runs of MMAP: pass 5/5
> 5 runs of SimpleFS: pass 1/5
> 5 runs of NIOFS: pass 2/5
>
> I can't help but think something is up here with SimpleFS/NIOFS: ive
> had this problem with lucene and IBM jres for a very long time: see
> https://issues.apache.org/jira/browse/LUCENE-2873 open a year and a
> half ago.
>
> I'm going to look into it and see if I can find anything
>
> On Tue, Jul 31, 2012 at 9:10 PM, Robert Muir <[EMAIL PROTECTED]> wrote:
> > I tried to reproduce the whole test run with my IBM JRE (which is not
> > the java7 one Uwe has, but still):
> >
> > java version "1.6.0"
> > Java(TM) SE Runtime Environment (build pxi3260sr10-20111208_01(SR10))
> > IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
> > jvmxi3260sr10-20111207_96808 (JIT enabled, AOT enabled)
> > J9VM - 20111207_096808
> > JIT  - r9_20111107_21307ifx1
> > GC   - 20110519_AA)
> > JCL  - 20111104_02
> >
> > This just caused out of open file handles... I think mine has other issues :)
> >
> > I cant reproduce the problems with any other JRE, it seems sketchy
> > this long is negative.
> >
> > On Tue, Jul 31, 2012 at 7:24 PM, Policeman Jenkins Server
> > <[EMAIL PROTECTED]> wrote:
> >> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/67/
> >> Java: 32bit/ibm-j9-jdk7
> >>
> >> 7 tests failed.
> >> FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestFieldCache
> >>
> >> Error Message:
> >>
> >>
> >> Stack Trace:
> >> java.lang.AssertionError
> >>         at __randomizedtesting.SeedInfo.seed([D686D14F0587EBAE]:0)
> >>         at
> org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWrite
> r.java:70)
> >>         at
> org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:83)
> >>         at org.apache.lucene.util.fst.FST.pack(FST.java:1505)
> >>         at
> org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish
> (MemoryPostingsFormat.java:273)
> >>         at
> org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWr
> iterPerField.java:550)
> >>         at
> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java
> :85)
> >>         at org.apache.lucene.index.TermsHash.flush(TermsHash.java:117)
> >>         at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
> >>         at
> org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:82)

I enabled -XnoOpt for now, so we can still run Jenkins tests with this JVM. But I would have a bad feeling with this JVM in production - poor WebLogic / Oracle Fusion Middleware users!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]

> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, August 01, 2012 11:33 AM
> To: [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jrockit-jdk1.6.0_33-R28.2.4-
> 4.1.0) - Build # 79 - Still Failing!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/79/
> Java: 64bit/jrockit-jdk1.6.0_33-R28.2.4-4.1.0
>
> 1 tests failed.
> REGRESSION:
> org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
> at
> __randomizedtesting.SeedInfo.seed([C068769794D3F462:B5789ED02135851F]:
> 0)
> at
> org.apache.lucene.index.FreqProxTermsWriterPerField.writeOffsets(FreqProxT
> ermsWriterPerField.java:157)
> at
> org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm(FreqProxTerm
> sWriterPerField.java:241)
> at
> org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:235)
> at
> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField
> .java:167)
> at
> org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcess
> or.java:305)
> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(Docum
> entsWriterPerThread.java:243)
> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter
> .java:374)
> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1357)
> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1108)
> at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> er.java:186)
> at
> org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWrit
> er.java:146)
> at
> org.apache.lucene.index.TestPostingsOffsets.checkTokens(TestPostingsOffsets.j
> ava:491)
> at
> org.apache.lucene.index.TestPostingsOffsets.testBackwardsOffsets(TestPostings
> Offsets.java:436)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI
> mpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu
> nner.java:1995)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random
> izedRunner.java:132)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz
> edRunner.java:819)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz
> edRunner.java:878)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz
> edRunner.java:891)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet
> upTeardownChained.java:50)
> at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach
> eSanity.java:32)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte
> rRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev
> aluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule
> ReportUncaughtExceptions.java:68)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea
> dAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn
> oreAfterMaxFailures.java:70)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja
> va:48)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando

On Wed, Aug 1, 2012 at 4:35 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
> Hi Robert,
>
> I checked Jenkin's settings and the "ulimit -n" is 8192 by default for jenkins. To prevent this problem I raised this to 32768.
> The thing with IBM J9 is that is has several caches for class files (so compiled class files can be cached in shared memory for parallel JVMs using the same classes), but I assume this needs more file handles.
>

Right I bumped this on charlie cron too and forgot about it, until i
installed this IBM jre on this machine.

IBM J9 using a few extra files compared to sun doesnt seem to explain
to me why SimpleFS/NIOFS use more filehandles than mmap though? And
that this problem never happens with other JREs....

I feel like something might be wrong here.

--
lucidimagination.com

---------------------------------------------------------------------

On Wed, Aug 1, 2012 at 9:17 AM, Robert Muir <[EMAIL PROTECTED]> wrote:
> On Wed, Aug 1, 2012 at 4:35 AM, Uwe Schindler <[EMAIL PROTECTED]> wrote:
>> Hi Robert,
>>
>> I checked Jenkin's settings and the "ulimit -n" is 8192 by default for jenkins. To prevent this problem I raised this to 32768.
>> The thing with IBM J9 is that is has several caches for class files (so compiled class files can be cached in shared memory for parallel JVMs using the same classes), but I assume this needs more file handles.
>>
>
> Right I bumped this on charlie cron too and forgot about it, until i
> installed this IBM jre on this machine.
>
> IBM J9 using a few extra files compared to sun doesnt seem to explain
> to me why SimpleFS/NIOFS use more filehandles than mmap though? And
> that this problem never happens with other JREs....
>
> I feel like something might be wrong here.

Rob and I dug a bit on this ... the hard open-file limit was 4096 and
the soft limit was 1024, and curiously, it looks like Oracle JVMs
"allow" themselves to go up to the hard limit (likely change the soft
limit on startup), while the IBM JVM uses the soft limit.  Has anyone
heard of JVMs doing this (increasing the soft limit to the hard limit
for open files) before...?  I see this curious "-XX:+MaxFDLimit"
option here:

    http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

But it says it's Solaris only ...

If I use ulimit to set the hard and soft limit to 1024 then both
Oracle and IBM JVMs fail TestShardSearching with NIOFSDir due to too
many open files.

But, for some reason if you run the test with MMapDir, far fewer file
descriptors are consumed ...

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------

This is failing a lot with different fails lately - I've got a list of things already I need to look into - but I'll add this to it.

On Aug 2, 2012, at 5:31 PM, Policeman Jenkins Server <[EMAIL PROTECTED]> wrote:

> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/121/
> Java: 32bit/jdk1.8.0-ea-b49 -server -XX:+UseParallelGC
>
> 1 tests failed.
> REGRESSION:  org.apache.solr.handler.component.DistributedQueryElevationComponentTest.testDistribSearch
>
> Error Message:
> .responseHeader.params.size()==9,9skipped=4,5
>
> Stack Trace:
> junit.framework.AssertionFailedError: .responseHeader.params.size()==9,9skipped=4,5
> at __randomizedtesting.SeedInfo.seed([B6FB0D16AB34F85F:371D830EDC6B9863]:0)
> at junit.framework.Assert.fail(Assert.java:50)
> at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:670)
> at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:392)
> at org.apache.solr.handler.component.DistributedQueryElevationComponentTest.doTest(DistributedQueryElevationComponentTest.java:81)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:474)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)

- Mark Miller
lucidimagination.com

Test uses "new Random()" without seed or usage of LuceneTestCase.random():

[forbidden-apis] Forbidden method invocation: java.util.Random#<init>()
[forbidden-apis]   in org.apache.solr.search.TestSolrJ (TestSolrJ.java:133)

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, August 09, 2012 9:54 PM
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.6.0_33) - Build # 311 -
> Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/311/
> Java: 32bit/jdk1.6.0_33 -client -XX:+UseConcMarkSweepGC
>
> All tests passed
>
> Build Log:
> [...truncated 10242 lines...]
> javadocs-lint:
>
> [...truncated 1691 lines...]
> javadocs-lint:
>      [exec]
>      [exec] Crawl/parse...
>      [exec]
>      [exec]   build/docs/core/org/apache/lucene/store/package-use.html
>      [exec]     WARNING: anchor "../../../../org/apache/lucene/store/subclasses"
> appears more than once
>      [exec]
>      [exec] Verify...
>
> [...truncated 563 lines...]
> javadocs-lint:
>      [exec]
>      [exec] Crawl/parse...
>      [exec]
>      [exec] Verify...
>
> [...truncated 8532 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/checkout/build.xml:55:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-
> Linux/checkout/solr/build.xml:213: Check for forbidden API calls failed, see log.
>
> Total time: 32 seconds
> Build step 'Execute shell' marked build as failure Archiving artifacts Recording
> test results Email was triggered for: Failure Sending email for trigger: Failure
>

---------------------------------------------------------------------

How comes that this MemoryPoolMXBean thread is only created by this test? Do we need to add this JVM specific thread to the allowed threads list?

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]
> -----Original Message-----
> From: Policeman Jenkins Server [mailto:[EMAIL PROTECTED]]
> Sent: Sunday, August 12, 2012 11:48 PM
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk6) - Build # 396 -
> Failure!
>
> Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/396/
> Java: 32bit/ibm-j9-jdk6
>
> 4 tests failed.
> FAILED:
> junit.framework.TestSuite.org.apache.lucene.analysis.uima.UIMABaseAnalyzer
> Test
>
> Error Message:
> 1 thread leaked from SUITE scope at
> org.apache.lucene.analysis.uima.UIMABaseAnalyzerTest:     1) Thread[id=19,
> name=MemoryPoolMXBean notification dispatcher, state=RUNNABLE,
> group=TGRP-UIMABaseAnalyzerTest]         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from
> SUITE scope at org.apache.lucene.analysis.uima.UIMABaseAnalyzerTest:
>    1) Thread[id=19, name=MemoryPoolMXBean notification dispatcher,
> state=RUNNABLE, group=TGRP-UIMABaseAnalyzerTest]
>         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)
>         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
> at __randomizedtesting.SeedInfo.seed([61647F0CD380E273]:0)
>
>
> FAILED:
> junit.framework.TestSuite.org.apache.lucene.analysis.uima.UIMABaseAnalyzer
> Test
>
> Error Message:
> There are still zombie threads that couldn't be terminated:    1) Thread[id=19,
> name=MemoryPoolMXBean notification dispatcher, state=RUNNABLE,
> group=TGRP-UIMABaseAnalyzerTest]         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
> threads that couldn't be terminated:
>    1) Thread[id=19, name=MemoryPoolMXBean notification dispatcher,
> state=RUNNABLE, group=TGRP-UIMABaseAnalyzerTest]
>         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)
>         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
> at __randomizedtesting.SeedInfo.seed([61647F0CD380E273]:0)
>
>
> FAILED:
> junit.framework.TestSuite.org.apache.lucene.analysis.uima.UIMATypeAwareAn
> alyzerTest
>
> Error Message:
> 1 thread leaked from SUITE scope at
> org.apache.lucene.analysis.uima.UIMATypeAwareAnalyzerTest:     1)
> Thread[id=20, name=MemoryPoolMXBean notification dispatcher,
> state=RUNNABLE, group=TGRP-UIMATypeAwareAnalyzerTest]         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from
> SUITE scope at org.apache.lucene.analysis.uima.UIMATypeAwareAnalyzerTest:
>    1) Thread[id=20, name=MemoryPoolMXBean notification dispatcher,
> state=RUNNABLE, group=TGRP-UIMATypeAwareAnalyzerTest]
>         at
> com.ibm.lang.management.MemoryNotificationThread.processNotificationLoo
> p(Native Method)
>         at
> com.ibm.lang.management.MemoryNotificationThread.run(MemoryNotificatio
> nThread.java:56)
> at __randomizedtesting.SeedInfo.seed([61647F0CD380E273]:0)
>
>
> FAILED:
> junit.framework.TestSuite.org.apache.lucene.analysis.uima.UIMATypeAwareAn