We are experiencing a strange problem. Everything with Elastic Search is fine but we get sudden deaths now and again. This may happen once a week or once a month or twice a day. During this time, I do not believe the index count or query count is unusual.
You can see the current rate of indexing and searching is not abnormal. However, if you look at the "rate of opened http connections", this rocketed at 10:20. The "search thread pool queue by size by node" also rocketed to > 1000 at this time. This caused all nodes to go offline and ES to become unresponsive.
Has anyone had this and do you know the cause of such issues?
Yes we do use Elastic Cloud. But for this particular client, we are using Qbox.
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext