Overriding toString on a Thread is not a good idea. Can I remove it or
at least make it simpler in ConcurrentMergeScheduler? This override
caused a fantastic deadlock -- an interesting possibility I didn't
think of -- again, when dumping threads (for the exception string)
Thread.toString was invoked from what I thought was an isolated
monitor (and it was); only toString had its own monitors underneath
and here's what happened (simplified):

"Lucene Merge Thread #1":
        at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:3764)
        - waiting to lock <L5> (a org.apache.lucene.index.IndexWriter)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.toString(ConcurrentMergeScheduler.java:499)
        ...
        at org.apache.lucene.util.LuceneTestCase.getRandom(LuceneTestCase.java:276)
        at org.apache.lucene.index.TestTransactions.access$100(TestTransactions.java:33)
        at org.apache.lucene.index.TestTransactions$RandomFailure.eval(TestTransactions.java:40)
        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:688)
        - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:415)
        - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
        at org.apache.lucene.codecs.lucene40.Lucene40FieldInfosWriter.write(Lucene40FieldInfosWriter.java:56)
        at org.apache.lucene.index.SegmentMerger.mergeFieldInfos(SegmentMerger.java:194)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3623)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3257)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
"Lucene Merge Thread #0":
        at org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:695)
        - waiting to lock <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
        at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:345)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3272)
        - locked <L5> (a org.apache.lucene.index.IndexWriter)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)

A classic, isn't it?

Dawid

---------------------------------------------------------------------

On Thu, Mar 1, 2012 at 5:20 PM, Dawid Weiss <[EMAIL PROTECTED]> wrote:
> Overriding toString on a Thread is not a good idea. Can I remove it or
> at least make it simpler in ConcurrentMergeScheduler? This override
> caused a fantastic deadlock -- an interesting possibility I didn't
> think of -- again, when dumping threads (for the exception string)
> Thread.toString was invoked from what I thought was an isolated
> monitor (and it was); only toString had its own monitors underneath
> and here's what happened (simplified):

Ouch!
Now I've got to go think if we've done anything like that in Solr...

-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10

---------------------------------------------------------------------

Scary!  I think remove it?

Though it is nice to see what segments are being merged by the
thread... but this risk is awful.  App can turn on IW's infoStream to
see it too...

Mike McCandless

http://blog.mikemccandless.com

On Thu, Mar 1, 2012 at 5:20 PM, Dawid Weiss <[EMAIL PROTECTED]> wrote:
> Overriding toString on a Thread is not a good idea. Can I remove it or
> at least make it simpler in ConcurrentMergeScheduler? This override
> caused a fantastic deadlock -- an interesting possibility I didn't
> think of -- again, when dumping threads (for the exception string)
> Thread.toString was invoked from what I thought was an isolated
> monitor (and it was); only toString had its own monitors underneath
> and here's what happened (simplified):
>
> "Lucene Merge Thread #1":
>        at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:3764)
>        - waiting to lock <L5> (a org.apache.lucene.index.IndexWriter)
>        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.toString(ConcurrentMergeScheduler.java:499)
>        ...
>        at org.apache.lucene.util.LuceneTestCase.getRandom(LuceneTestCase.java:276)
>        at org.apache.lucene.index.TestTransactions.access$100(TestTransactions.java:33)
>        at org.apache.lucene.index.TestTransactions$RandomFailure.eval(TestTransactions.java:40)
>        at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:688)
>        - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
>        at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:415)
>        - locked <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
>        at org.apache.lucene.codecs.lucene40.Lucene40FieldInfosWriter.write(Lucene40FieldInfosWriter.java:56)
>        at org.apache.lucene.index.SegmentMerger.mergeFieldInfos(SegmentMerger.java:194)
>        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:109)
>        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3623)
>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3257)
>        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
>        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
> "Lucene Merge Thread #0":
>        at org.apache.lucene.store.MockDirectoryWrapper.listAll(MockDirectoryWrapper.java:695)
>        - waiting to lock <L4> (a org.apache.lucene.store.MockDirectoryWrapper)
>        at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:345)
>        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3272)
>        - locked <L5> (a org.apache.lucene.index.IndexWriter)
>        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382)
>        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451)
>
> A classic, isn't it?
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

---------------------------------------------------------------------

> Ouch!
> Now I've got to go think if we've done anything like that in Solr...

Yeah... I honestly never thought about such possibility and I don't
think any sane person would ;)

I think this qualifies as a hack similar to the solution to this puzzler:
http://wouter.coekaerts.be/2012/puzzle-clowns

Dawid

---------------------------------------------------------------------

> Though it is nice to see what segments are being merged by the
> thread... but this risk is awful.  App can turn on IW's infoStream to
> see it too...

This could be possible by updating a volatile string somewhere and
only exposing it in toString override, but I don't know if this is
worth the effort. Volatile will impose an additional happens-before,
etc.

Dawid

---------------------------------------------------------------------

> Now I've got to go think if we've done anything like that in Solr...

I did a quick check via Eclipse's Java search and it seems nothing
else overrides Thread#toString() or Thread#getName. Can't guarantee
anything, but 99% sure we're safe from this one.

Dawid

---------------------------------------------------------------------

On Thu, Mar 1, 2012 at 5:30 PM, Dawid Weiss
<[EMAIL PROTECTED]> wrote:
>> Ouch!
>> Now I've got to go think if we've done anything like that in Solr...
>
> Yeah... I honestly never thought about such possibility and I don't
> think any sane person would ;)
>
> I think this qualifies as a hack similar to the solution to this puzzler:
> http://wouter.coekaerts.be/2012/puzzle-clowns

Cute!  Hadn't seen that.  My first thought to a solution matched the
first comment on the solutions page (no spoilers!).
I missed the more elegant  official solution.

-Yonik
lucenerevolution.com - Lucene/Solr Open Source Search Conference.
Boston May 7-10

---------------------------------------------------------------------

I like the Clowns! Whenever I see IndexWriter internals I think about those clowns...

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [EMAIL PROTECTED]

> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Dawid Weiss
> Sent: Thursday, March 01, 2012 11:30 PM
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Subject: Re: toString on Thread
>
> > Ouch!
> > Now I've got to go think if we've done anything like that in Solr...
>
> Yeah... I honestly never thought about such possibility and I don't think any sane
> person would ;)
>
> I think this qualifies as a hack similar to the solution to this puzzler:
> http://wouter.coekaerts.be/2012/puzzle-clowns
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED] For additional
> commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------