Operating Elasticsearch in Kubernetes

Operating stateful applications is a challenge and Elasticsearch clusters are no exception. At Zalando, the search department has a long-term history of operating large-scale Elasticsearch clusters consisting of hundreds of nodes.

In this talk Mikkel and Oliver present how Zalando utilizes Kubernetes and the operator pattern as introduced by CoreOS, to reliably and cost efficiently operate large-scale Elasticsearch clusters.

The talk will walk through how the Elasticsearch operator was designed, what problems it solves and how building it from the bottom up allowed getting it in production fast, gather more learnings and later extending the feature set to make it less manual to operate and reducing the infrastructure cost by introducing auto-scaling.

While the talk revolves around running Elasticsearch, the ideas presented also apply to running other types of stateful applications in Kubernetes. Additionally, the features of the Elasticsearch operator presented is compared to the new Kubernetes operator offering from Elastic.