I would probably start localizing the issue. You can use [preference](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-request-preference.html
) flags to try to search just on specific node. For example, if you are assuming that this is a transport - try to run this query on specific node with specific shard and call the query with `preference=_shards:2,3|_primary` as an example to exclude any routing. See if that will give you better result.
If it does - possible a bug in communication. Next options are look in sources, increase verbosity, submit a bug.