Elasticsearch and the Elastic Stack¶

Overview¶

Elasticsearch provides full-text search and analytics; Kibana visualizes and manages indexed data. A logging pipeline may also use Elastic Agent or Fluent Bit for collection and Logstash for complex transformation. This family is commonly called ELK, although modern deployments do not always require Logstash.

Elasticsearch or Loki¶

Requirement	Better starting point
Kubernetes logs with low operational cost	Loki + Alloy + Grafana
Application full-text search	Elasticsearch
Rich log analytics and Elastic integrations	Elasticsearch + Kibana
Complex event transformation	Add Logstash only when needed

Do not deploy both Loki and Elasticsearch for the same logs without a clear reason.

VPS Capacity Warning¶

Elasticsearch and Kibana are memory-intensive. A single Elasticsearch node needs at least about 2 GiB of available memory before application data, filesystem cache, and Kibana are considered. On an 8 GB VPS already running PostgreSQL, Redis, Prometheus, and Kafka or Redpanda, use managed Elasticsearch or omit the broker/log stack.

GitOps Deployment¶

Use Elastic Cloud on Kubernetes (ECK) when Kubernetes-native lifecycle management is required. Keep the ECK operator Application separate from Elasticsearch and Kibana custom resources so CRDs sync first. Pin both the operator and Elastic Stack versions.

For a lab profile, use one Elasticsearch node, one Kibana instance, a bounded PVC, ILM retention, and ClusterIP services. Keep the default authentication and TLS enabled. Never publish port 9200 directly.

Verification¶

kubectl get elasticsearch,kibana,pod,pvc -n search
kubectl port-forward -n search svc/search-es-http 9200:9200
kubectl port-forward -n search svc/search-kb-http 5601:5601

Check cluster health, JVM heap, disk watermarks, indexing latency, rejected requests, shard count, and index retention. A green single-node cluster is still not highly available.