Hi lubin,

the terms dictionary is using the "tim" and "tip" files. It should be approximately in the dimension of the FST.

Do you have all indexes running in the same JVM or is it 10 servers? Because then the numbers look correct. If you really want to have such an large index in a single machine using a single JVM, you should plan for more heap space. I'd start with 12 GiB of heap space to run this index.

A last recommendation: If you update your index during runtime, make sure that you correctly close the outdated IndexReaders/IndexSearchers (e.g. using SearcherManager), so you don't have orphaned instances of IndexReader consuming heap space and disk space, because the files can't be fully deleted as long as those are open!


Uwe Schindler
Achterdiek 19, D-28357 Bremen
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB