Home | About | Sematext search-lucene.com search-hadoop.com
 Search Lucene and all its subprojects:

Switch to Threaded View
Lucene, mail # dev - Low pause GC for java 1.6
Bill Bell
2012-06-30, 03:09
Bill Bell
2012-07-01, 01:48
Walter Underwood
2012-07-01, 03:56
Jack Krupansky
2012-07-01, 04:48
Bill Bell
2012-07-01, 06:28
Shawn Heisey
2012-07-01, 16:20
Jack Krupansky
2012-07-01, 16:37
Greg Bowyer
2012-07-02, 16:58
Otis Gospodnetic
2012-07-10, 06:26
Copy link to this message
-
RE: Low pause GC for java 1.6
Dyer, James 2012-07-01, 03:49
Bill,

As you know, it really depends on the size of your index combined with which features you're using.  There is really no substitute for having a good load test and monitoring tool and to run multiple tests while trying different settings.

My guess is that you're experiencing "full" gc's, even with CMS enabled.  This means either your tenured ("old") generation is too small or you have the "-XX:CMSInitiatingOccupancyFraction" set too high (it starts the CMS too late and runs out of memory before it can finish).  We've found that some of the defaults the JVM picks and/or the general advice out there doesn't apply to an app like Solr, which is just a different kind of animal than the typical web frontend you might run in a J2ee container.

Below are the settings i am using as a starting point for our development Solr 4.0 app.  These may or may not work for you but at least should give you a basic idea of how 1 other installation is configured.

Also, if you're using older grouping patches (I remember you worked on some of these), perhaps you're hitting some of the scalibility problems that were predicted for some of these?  I'm pretty sure the GA grouping features in 3.x solved these problems though.

Finally, you probably will get better responses on the users list than the dev list.  Also, other users might benefit from other answers you get, so perhaps you could cross-post your question.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311

# Basic JVM settings.  I thing the 3g new generation size is bigger than you'd normally have with a typical web app but for us it makes the old gen fill up slower and have fewer CMS gc's.  Minor ("parnew") gc's are still fast enough for us, even with a biggish new gen.
-XX:MaxNewSize=3000m
-XX:NewSize=3000m
-Xms20g
-Xmx20g
-XX:MaxPermSize=256m

# These are our CMS settings
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=85
-XX:+CMSParallelRemarkEnabled
-XX:CMSMaxAbortablePrecleanTime=15000

# trial and error found this to be the sweet spot for out 16-way machines.
-XX:ParallelGCThreads=8

# YOu want these so you can see in your logs what is going on.  There are some tutorials on the web on how to make sense of verbose garbage collection.  There's no problem using these in Production.
-verbose:gc
-XX:+PrintGCDetails
-XX:+PrintGCTimeStamps

# Use this on a 64-bit machine unless your jvm is too old to support it (on by default on newer jvms, I think)
-XX:+UseCompressedOops

# we found these save a little memory
-XX:+UseStringCache
-XX:+UseCompressedStrings

-----Original Message-----
From: Bill Bell [mailto:[EMAIL PROTECTED]]
Sent: Saturday, June 30, 2012 8:49 PM
To: Bill Bell
Cc: [EMAIL PROTECTED]
Subject: Re: Low pause GC for java 1.6

Nothing?

Bill Bell
Sent from mobile
On Jun 29, 2012, at 9:09 PM, Bill Bell <[EMAIL PROTECTED]> wrote:

> We are getting large Solr pauses on Java garbage collection in 1.6 Java.
>
> We have tried CMS. But we still have. 4 second wait on GC.
>
> What works well for Solr when using 16 GB of RAM?
>
> I have read lots of articles and now just looking for practical advise and examples.
>
> Sent from my Mobile device
> 720-256-8076

---------------------------------------------------------------------
---------------------------------------------------------------------