Kafka-Compatible Streaming on k3s¶

Overview¶

Provides an event-streaming platform for applications that need durable topics, consumer groups, replay, and Kafka client compatibility.

Kafka or Redpanda¶

Choice	Use when	VPS impact
Apache Kafka in KRaft mode	Exact Kafka behavior and ecosystem are required	Higher tuning and memory cost
Redpanda	Kafka API compatibility and simpler single-node operations are enough	Better fit for an 8 GB VPS
No broker	Only background jobs are needed	Prefer a PostgreSQL outbox or Redis queue

The template should make this component optional. A database, cache, observability stack, and broker together can exhaust an 8 GB node.

Single-Node Profile¶

Deploy the official Redpanda Helm chart through Argo CD with one replica, persistent storage, explicit CPU/memory limits, and external listeners disabled. Applications use the internal bootstrap service on port 9092.

Start with approximately 1 CPU, 1–2 GiB memory, and a 10–20 GiB volume. Validate exact values against the pinned chart version because its schema changes. Set topic retention by bytes and time so the VPS disk cannot fill indefinitely.

Verification¶

kubectl get pod,pvc,svc -n streaming
kubectl exec -n streaming redpanda-0 -c redpanda -- rpk cluster health
kubectl exec -n streaming redpanda-0 -c redpanda -- rpk topic create smoke-test
kubectl exec -n streaming redpanda-0 -c redpanda -- rpk topic list

Monitor under-replicated partitions, consumer lag, produce/fetch latency, disk usage, and broker restarts. One replica provides no broker or host redundancy; back up critical events at the application or data layer.