Shameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches.…
This is a good resource for the up to date resilience status in Elasticsearch https://www.elastic.co/guide/en/elasticsearch/resiliency/cur... They have gotten A LOT better since the 1.x days
We feel that ~300 nodes strikes a good balance in the cattle vs pets philosophy. Going up to i3en.12xlarge (or equivalent) would probably have worked as well. But after that the impact of loosing just one node would be…
One time, a few years ago a particularly nasty query was executed over and over again and it took a few hours to find it and then block it. And during that time so many nodes had became slow and unresponsive that…
Thank you for those kind words. And yes we have had our fair share of pain with the old cluster for sure. the new version (7.17) is still behaving a lot better so far and feels a lot more predictable.
Thank you. Yes it was a massive project. I don't want to spoil the other blog posts but we managed to solve almost all of our custom use cases without modifying elasticsearch itself. We still have one custom plugin but…
Thanks for this info! Doing a quick googling on elasticsearch and ZGC I just found this https://github.com/elastic/elasticsearch/issues/44321 where the official response from elastic is that only G1GC and CMS are tested…
Hi, do you have real use experience running elasticsearch with 64g+ heap? Is there any articles/benchmark/notes or anything that you would be willing to share? We have considered trying out 64g+ heaps for our cluster…
The social data are fairly small documents, but we index lots of other things as well that is larger in size. Another factor is our large replica count, especially for the hot data. We also have quite a lot of metadata.
We use datadog for monitoring our cluster. Shardonnay also produces lots of custom metrics that you don't get from the vanilla elasticsearch datadog integration.
We tried the hot/cold architecture as well, and used it before in our data centre architecture. But currently we have concluded that it is not worth the added complexity. But that might change again if/when we learn…
Hi. We are actually using an even older version, 1.7.6 for our cluster. This is off not an ideal set-up and we will upgrade as soon as we can. That is not an easy task in a cluster of this size and given our uptime…
Shameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches.…
This is a good resource for the up to date resilience status in Elasticsearch https://www.elastic.co/guide/en/elasticsearch/resiliency/cur... They have gotten A LOT better since the 1.x days
We feel that ~300 nodes strikes a good balance in the cattle vs pets philosophy. Going up to i3en.12xlarge (or equivalent) would probably have worked as well. But after that the impact of loosing just one node would be…
One time, a few years ago a particularly nasty query was executed over and over again and it took a few hours to find it and then block it. And during that time so many nodes had became slow and unresponsive that…
Thank you for those kind words. And yes we have had our fair share of pain with the old cluster for sure. the new version (7.17) is still behaving a lot better so far and feels a lot more predictable.
Thank you. Yes it was a massive project. I don't want to spoil the other blog posts but we managed to solve almost all of our custom use cases without modifying elasticsearch itself. We still have one custom plugin but…
Thanks for this info! Doing a quick googling on elasticsearch and ZGC I just found this https://github.com/elastic/elasticsearch/issues/44321 where the official response from elastic is that only G1GC and CMS are tested…
Hi, do you have real use experience running elasticsearch with 64g+ heap? Is there any articles/benchmark/notes or anything that you would be willing to share? We have considered trying out 64g+ heaps for our cluster…
The social data are fairly small documents, but we index lots of other things as well that is larger in size. Another factor is our large replica count, especially for the hot data. We also have quite a lot of metadata.
We use datadog for monitoring our cluster. Shardonnay also produces lots of custom metrics that you don't get from the vanilla elasticsearch datadog integration.
We tried the hot/cold architecture as well, and used it before in our data centre architecture. But currently we have concluded that it is not worth the added complexity. But that might change again if/when we learn…
Hi. We are actually using an even older version, 1.7.6 for our cluster. This is off not an ideal set-up and we will upgrade as soon as we can. That is not an easy task in a cluster of this size and given our uptime…