On 6/13/2018 4:04 AM, Markus Jelsma wrote:
> You mentioned shard handler tweaks, thanks. I see we have an incorrect setting there for maximumPoolSize, way too high, but that doesn't account for the number of threads created. After reducing the number, for dubious reasons, twice the number of threads are created and the node dies.

The specific config in the shard handler I was thinking of was socket
timeouts.

The EofException in Jetty almost always indicates that the client
disconnected before the server responded.  A low socket timeout will
cause a client to disconnect if the server takes longer than the timeout
to finish its work.  When the server finally does finish and try to
respond, it throws the EofException because the connection it was
expecting to use is no longer there.

I believe that the default socket timeout in Solr's shard handler is 60
seconds.  Which is a relative eternity in most situations.  It would
take a particularly nasty GC pause problem to exceed that timeout.

> Regarding memory leaks, of course, the first that came to mind is that i made an error which only causes trouble on 7.3, but it is unreproducible so far, even if i fully replicate production in a test environment. Since it only leaks on commits, first suspect were URPs, and the URPs are the only things i can disable in production without affecting customers. Needless to say, it weren't the URPs.

Custom update processors could cause leaks, but it is not something that
I would expect from a typical URP implementation. If you've disabled
them and it's still happening, then that's probably not it.  It's
plugins for queries that have the most potential for not closing
resources correctly even when written by experienced programmers.  I'm
not sure what the potential for leaks is on index-time plugins, but I
suspect that it's less likely than problems with query plugins.

Thanks,
Shawn
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB