Hi Shawn,

Thanks for detail mail. Yes I am behind the IndexConfig only.

Regarding 5GB size of collection, it's not one document. It has almost 3M of
docs in that collection.

I am using the default configuration, as all solr experts say default one
suits for most of the cases and so following the defaults. We changed only
the commits part and using 15000 for hard commits and 1 sec for soft commit.
All other setting like locking, deleting policy, merge, directory, etc are
left to default ones.

One strange thing we noticed after moving from solr 5.x to solr 6.5.1 is
that CPU and RAM usage is increased drastically. We have two solr nodes, one
for data and another for replica. It's EC2 r4.xlarge machine. We have almost
6 collection  and each carries around 5GB of data in average and in couple
of collection we have frequent updates too.

In solr 5.x we didn't feel this much of RAM and CPU usages. CPU is always 80
to 90% even if we are trying to index or update some 50 docs at one shot and
RAM it occupies whatever we give. We started with 8GB of Heap. But its
always 8GB.  Initially we were using CMS GC and tried with G1 GC. Only
difference is that, In case of CMS, even after starting solr, cpu goes to
80%, where as in G1, after started solr it's around 10% and when load comes
( around 100 to 200 docs  in 5 mins ) it's goes 90% ( in both CMS and G1 )

When we tried to profile the solr process, we found that merging is keep on
happening.

This spikes of CPU and memory, we are seeing only after moving to 6.5.1. Is
that stable version ? moving to latest stable will solve this issue or we
miss something wrt configurations ? Do we need to change the solr default
config ?
Advice...
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB