Kubernetes Services and Networking

The problem: Pods are disposable, their IPs are not stable

From Fundamentals: when a Pod dies, the ReplicaSet controller creates a replacement, not a resurrection — the new Pod gets a new IP address. Anything that hardcoded the old Pod's IP is now pointing at nothing. A Service solves this by giving a group of Pods (selected by label, the same selector mechanism a Deployment uses) one stable virtual IP that never changes, no matter how many times the underlying Pods are replaced.

graph TD client["Caller"] --> vip["Service: my-app<br/>stable ClusterIP: 10.96.0.5"] vip -.->|"kube-proxy rule"| p1["Pod (10.244.1.4)"] vip -.->|"kube-proxy rule"| p2["Pod (10.244.2.7)"] vip -.->|"kube-proxy rule"| p3["Pod (10.244.1.9)"] note["Pod IPs change on every restart<br/>Service IP never does"]
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

selector: app: my-app is the entire mechanism — the Service has no idea which specific Pods exist; it just continuously asks "which Pods right now have the label app: my-app," the same label-matching approach a ReplicaSet uses to count Pods. kube-proxy, on every node, watches the Service and its matching Pods, and programs local iptables (or IPVS) rules so that any packet sent to the Service's ClusterIP 10.96.0.5:80 gets rewritten to one of the current Pod IPs on port 8080 — load-balanced across whichever Pods currently match, with the rule set updated automatically as Pods come and go.

DNS: the same convenience as Compose, cluster-wide

Every Service automatically gets a DNS name via the cluster's internal DNS (CoreDNS): my-app.default.svc.cluster.local — or just my-app from within the same namespace. This is the cluster-scale version of the Docker Compose service-name DNS mechanism: application code connects to http://my-app/, never to a literal IP, and it keeps working through Pod restarts, rescheduling, and even Service recreation, as long as the Service name itself doesn't change.

Service types: how far outside the cluster the Service is exposed

graph TD subgraph internet["Outside the cluster"] u["User"] end subgraph cluster["Cluster"] subgraph clusterip["ClusterIP (default)"] cip["Only reachable from inside the cluster"] end subgraph nodeport["NodePort"] np["Reachable at every node's IP, on one static port (30000-32767)"] end subgraph lb["LoadBalancer"] elb["Cloud provider provisions an external load balancer<br/>that forwards to a NodePort under the hood"] end end u -.->|"blocked - internal only"| cip u -->|"http://any-node-ip:31234"| np u -->|"http://provisioned-lb-address"| elb
  • ClusterIP (the default) — only reachable from inside the cluster. Right for anything that's only ever called by other Pods (a database, an internal API) — there's no reason to expose it further, and not exposing it further is one less thing to secure.
  • NodePort — additionally opens a static port (in the 30000-32767 range) on every node's IP, forwarding to the Service. Usable directly, but crude — it exposes a high, unmemorable port on every node, and doesn't handle TLS or host/path-based routing.
  • LoadBalancer — on a cloud provider with Kubernetes integration (AWS, GCP, etc.), requesting this type provisions an actual external load balancer (an AWS NLB/ALB, a GCP Load Balancer) that forwards traffic in — under the hood, still ultimately routing to a NodePort. Each LoadBalancer Service typically provisions its own cloud load balancer, which is a real, billed cloud resource — creating one per microservice adds up in cost quickly, which is exactly the problem Ingress (below) exists to solve.

Ingress: one entry point, routed by host/path, instead of one LB per Service

graph TD client["Client"] --> lb["Single LoadBalancer / entry point"] lb --> ic["Ingress Controller<br/>(e.g. nginx-ingress — a Pod that reads Ingress rules)"] ic -->|"Host: api.example.com"| svcA["Service: api"] ic -->|"Host: app.example.com"| svcB["Service: web"] ic -->|"Path: /admin"| svcC["Service: admin"]
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app-ingress
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

Critical fact that surprises almost everyone new to this: an Ingress resource, by itself, does nothing. It's a declaration of routing rules — a config object sitting in etcd — with no built-in component reading and enforcing it. Something called an Ingress Controller (nginx-ingress, Traefik, cloud-provider-specific ones) must be separately installed in the cluster; it watches Ingress objects and actually configures a real reverse proxy to match. One Ingress Controller (behind one LoadBalancer Service) can front routing rules for dozens of internal Services, replacing "one expensive cloud load balancer per Service" with one shared entry point that fans out by host or path.

NetworkPolicy: Kubernetes networking is allow-all by default

Without any NetworkPolicy objects, every Pod can reach every other Pod in the cluster, across namespaces, with no restriction — the same "secure by default" gap noted for Docker's root-by-default containers. A NetworkPolicy switches specific Pods to a deny-by-default model, allowing only explicitly listed traffic:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: db-only-from-api
spec:
  podSelector:
    matchLabels:
      app: db
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: api
      ports:
        - port: 5432

Once this applies to Pods labeled app: db, only Pods labeled app: api can reach them on port 5432 — every other Pod in the cluster, which previously could reach the database with zero restriction, now cannot. NetworkPolicy enforcement itself depends on the cluster's CNI plugin supporting it (Calico, Cilium — some simpler CNI plugins silently ignore NetworkPolicy objects entirely, which is worth confirming, not assuming).

Common pitfall

A Service whose selector doesn't match any Pod's actual labels — a typo, or a label that was renamed on the Deployment but not updated on the Service — creates a Service with zero endpoints. This fails silently: no error, no warning, just a Service that accepts connections and then hangs or refuses, because kube-proxy has no Pod IP to route to. kubectl get endpoints my-app showing an empty list is the direct way to confirm this specific failure mode rather than guessing at network-level causes.