Kubernetes Architecture, Operations, and Troubleshooting Questions
How Kubernetes works, how to run it, and how to debug it. Covers control-plane and node components, the scheduler and API server, cluster design, high availability and multi-cluster topologies, and platform-level operations; the workload primitives (pods, deployments, services, controllers), cluster upgrades, and designing Kubernetes as an internal platform; and the operational depth inside a cluster including pod and service networking, ingress and the CNI model, service mesh, persistent volumes and storage classes, resource requests and limits, and systematically diagnosing scheduling, networking, and storage failures. The full architecture-through-day-two-operations span of Kubernetes.
Design how you would run a stateful, production-grade database within Kubernetes. Discuss operators, persistent volumes, storage classes, pod disruption budgets, backup/restore, and handling of node failures and scaling.
Sample Answer
Run the database through a purpose-built operator (a controller that encodes that specific database's operational knowledge as a CRD, Custom Resource Definition) rather than a hand-rolled StatefulSet, back it with topology-aware block storage, protect quorum with a PodDisruptionBudget (PDB) and anti-affinity, and treat backup/restore and failure recovery as first-class design requirements, not afterthoughts. The hardest part is not storage or scaling, it's guaranteeing zero data loss without a split-brain during failover, which needs an explicit fencing mechanism the platform doesn't give you for free.
Requirements first
Pin down RPO/RTO (Recovery Point/Time Objective) targets, expected throughput, latency budget, number of Availability Zones (AZs), and whether managed-DB-as-a-service is politically or contractually off the table; these decide almost everything below.
Three-way comparison: plain StatefulSet vs operator vs managed DB
| Plain StatefulSet (DIY) | Operator-managed StatefulSet | Managed DB service (e.g. RDS, Cloud SQL) | |
|---|---|---|---|
| Failover automation | None built in; you write it | Automated: leader election, promotion, replica rebuild | Fully automated by the provider |
| Upgrade safety | Manual, error-prone | Operator sequences safe rolling upgrades | Provider-managed, scheduled maintenance windows |
| Backup/restore | You build and test it | Operator typically integrates snapshot + WAL/binlog shipping | Built-in, provider-managed |
| Operational control | Full control, full responsibility | High control, less toil | Least control, least toil |
| Best fit | Learning/simple cases only; not recommended for production-grade multi-replica DBs | Teams that need in-cluster placement, cost control, or a DB the managed offering doesn't support well | Teams that want to trade control for reduced operational burden |
For production, an operator is almost always the right middle ground unless a managed service is available and acceptable; a bare StatefulSet without an operator means re-implementing failover, backup orchestration, and safe upgrades yourself, which is exactly the hard part.
Concrete examples of DB-specific operators: CloudNativePG or the Zalando operator for PostgreSQL, Strimzi for Kafka, ECK (Elastic Cloud on Kubernetes) for Elasticsearch. Worth noting for currency: Kafka 4.0 (released March 2025) dropped ZooKeeper support entirely and runs KRaft-only, so a current Strimzi-managed Kafka cluster is one StatefulSet-backed topology (the brokers, using the KRaft consensus protocol for metadata) rather than two separate stateful systems (brokers plus a ZooKeeper ensemble) as older deployments required.
Architecture
flowchart TB
OP[Database Operator / CRD controller]
SS[StatefulSet: ordered, stable Pod identities]
PDB[PodDisruptionBudget: protects quorum]
SC[Topology-aware StorageClass, CSI provisioner]
BK[Backup: snapshots + WAL/binlog shipping to object storage]
OP --> SS
OP --> PDB
SS --> SC
OP --> BK
SS -->|anti-affinity| AZ1[AZ 1: primary]
SS -->|anti-affinity| AZ2[AZ 2: sync replica]
SS -->|anti-affinity| AZ3[AZ 3: async replica / DR]
Storage layer
- CSI (Container Storage Interface)-backed StorageClass with
volumeBindingMode: WaitForFirstConsumer, so the volume is provisioned in the same zone the scheduler actually places the Pod, avoiding an unschedulable Pod stuck waiting on a volume in the wrong zone. reclaimPolicy: Retainon the StorageClass: an accidental PVC (PersistentVolumeClaim) deletion must not silently destroy the primary's data; recovery should require a deliberate, human step.- Block storage versus a distributed filesystem, explicitly. Zonal block storage (EBS/PD/Azure Disk-style volumes) gives the lowest latency per replica but is single-zone and single-writer; the database's own replication protocol is what provides cross-zone durability, not the storage layer. A distributed filesystem or storage system (Ceph, Portworx, and similar) instead replicates data itself across nodes/zones at the storage layer, trading some latency and operational complexity for a storage layer that doesn't rely entirely on the database's own replication being correctly configured. For a workload with a mature, well-tested internal replication protocol (most production RDBMS and Kafka), zonal block storage plus database-level replication is usually the better latency/complexity trade-off; a distributed storage layer is more attractive for workloads with weak or no built-in replication.
The zero-data-loss and fencing problem
This is the part a plain StatefulSet does not solve for you. If a primary becomes unreachable but is not actually dead (a network partition, not a crash), promoting a replica while the old primary is still accepting writes creates two primaries accepting conflicting writes, a split-brain, which is a correctness failure, not just an availability one. A correct design needs explicit fencing: before a new primary is promoted, the old one must be provably prevented from accepting further writes, whether by a storage-level fence (revoking its volume attachment), a network-level fence (an admission gate the operator controls), or a consensus-based lease/term mechanism where a promoted replica only becomes writable once it holds a lease the old primary cannot renew. Most mature operators build this in (Kafka's KRaft controller quorum and Postgres operators using a distributed lock like Patroni's approach are both fencing-aware); a hand-rolled StatefulSet failover script that just runs a promote command is not safe against a network partition unless it independently solves this same problem.
PodDisruptionBudget and placement
PodDisruptionBudgetwithminAvailableset so voluntary disruptions (node drains, cluster upgrades) can never take out enough replicas to lose quorum, for exampleminAvailable: 2on a 3-replica set.- Required (hard) pod anti-affinity across zones so replicas never land on the same node or, ideally, the same zone, or a single zone failure takes out more than one replica at once.
- Readiness probes gate traffic during recovery; liveness probes tuned to the database's actual health semantics, not a generic TCP check.
3-AZ replication topology
A common shape for regional durability without full cross-region cost: synchronous replication between the primary and one replica in a second AZ (so a confirmed write survives losing either AZ, at the cost of added write latency for the round trip), plus asynchronous replication to a third AZ or a separate region for disaster recovery, accepting a small, bounded RPO there in exchange for avoiding synchronous replication's latency cost across a third leg. The trade-off is explicit: every synchronous replica added tightens RPO but adds write latency; asynchronous replicas protect against a wider blast radius (a whole-region event) but carry a real, non-zero RPO if the primary is lost before the async replica catches up.
Backup and restore
- Application-consistent snapshots via CSI Volume Snapshots for fast full backups.
- Continuous WAL (write-ahead log) or binlog shipping for point-in-time recovery, usually built into the operator.
- Periodic backups to durable object storage (S3/GCS/Azure Blob-equivalent) with encryption and a lifecycle/retention policy.
- Restore drills on a schedule, not just documented in a runbook; an untested restore path is not a real recovery capability.
Handling node failures and scaling
- Node failure: the operator promotes a healthy replica (using the fencing mechanism above), Kubernetes reschedules the failed Pod elsewhere, and the storage layer either reattaches the same volume (single-zone block storage on a healthy node) or the operator rebuilds a fresh replica from backup/replication if the volume itself is unavailable.
- Vertical scaling: usually requires a coordinated, operator-sequenced resize or a planned failover, not a live in-place change.
- Horizontal scaling: read scaling via additional replicas behind a read-only Service; write scaling (if needed at all) requires sharding at the application or a purpose-built layer (e.g. Citus for Postgres, Vitess for MySQL), which is a materially different architecture, not a StatefulSet replica-count change.
Trade-offs and pitfalls
- Treating "add a PodDisruptionBudget and anti-affinity" as sufficient without addressing fencing is the most dangerous gap: it protects against voluntary disruption and Pod placement risk but says nothing about split-brain during an involuntary, ambiguous failure like a network partition.
- Choosing a distributed storage layer for its own replication when the database already replicates well is redundant complexity and extra latency for no real durability gain; verify what layer is actually providing the durability guarantee you're relying on.
- An untested backup is not a backup; the failure mode where a restore procedure is discovered to be broken happens exactly once, during a real incident.
- Running a fully DIY StatefulSet without an operator is a legitimate choice only for genuinely low-stakes or learning environments; for anything production-grade, the missing failover and fencing logic is a correctness gap, not just an operational inconvenience.
Explain Kubernetes networking end-to-end: pod IP model, service virtual IPs, kube-proxy modes (iptables vs IPVS), CNI plugins and how they implement pod networking, and practical debugging steps when pods on different nodes cannot reach each other on specific TCP ports. Include commands and sources of truth to inspect.
Sample Answer
Direct answer
Every pod gets a routable IP address from the CNI (Container Network Interface) plugin, with no NAT (network address translation) between pods by default. A Service layers a stable virtual IP and DNS name on top of that, and kube-proxy (or an eBPF-based replacement like Cilium; eBPF lets Linux run sandboxed programs inside the kernel to process packets directly) programs the actual rewriting rules that turn a Service IP into a real pod IP. When two pods on different nodes cannot reach each other on a specific port, the fault sits in exactly one of four layers, checked in this order: the pod's own listener, the kube-proxy/Service plumbing, the CNI's cross-node routing, or a host/cloud firewall. Testing each layer directly beats guessing from symptoms.
Structured elaboration
Pod IP model
The CNI plugin assigns each pod a unique IP from the cluster's pod address range and is responsible for making that IP reachable from every node, either by programming host routes directly (an "underlay" approach, for example Calico in BGP mode, Border Gateway Protocol, the routing protocol routers use to exchange reachability information) or by encapsulating pod traffic in an overlay (VXLAN, a tunneling protocol that wraps one Ethernet frame inside a UDP packet, used by Flannel and by Calico's IP-in-IP mode).
Service virtual IPs and kube-proxy modes
A ClusterIP Service is a stable virtual IP that never corresponds to a real listener; kube-proxy rewrites traffic to it into a real pod IP and port.
| Mode | Mechanism | Status |
|---|---|---|
| iptables | Programs Linux netfilter/iptables NAT rules per Service and endpoint | Still the default |
| IPVS | Uses the kernel's IPVS load balancer, built for large numbers of Services | GA, scales better than iptables (near-constant-time lookup versus iptables' linear rule matching) |
| nftables | Programs nftables rules, the modern successor to iptables | Alpha in 1.29, beta in 1.31, GA in Kubernetes 1.33; not the default even where available, but the direction the project is moving, since the iptables backend is legacy code |
Cilium and some other CNIs bypass kube-proxy entirely, attaching eBPF programs to kernel networking hooks to do the same rewriting job with lower per-packet overhead and without iptables' rule-count scaling problem.
CNI plugins in practice
- Calico: pure L3 routing via BGP (no overlay, lowest overhead, needs a BGP-capable network) or IP-in-IP/VXLAN encapsulation where BGP peering is not available.
- Flannel: a simple VXLAN overlay by default; fewer features than Calico or Cilium.
- Cilium: eBPF-based; can replace kube-proxy, adds L7-aware NetworkPolicy and traffic observability, at the cost of needing a reasonably current kernel.
flowchart LR
A[Pod A on Node 1] -->|packet to Service ClusterIP| KP1[kube-proxy or eBPF dataplane, Node 1]
KP1 -->|rewrite to real Pod IP| CNI1[CNI routing or overlay, Node 1]
CNI1 -->|cross-node route or tunnel| CNI2[CNI routing or overlay, Node 2]
CNI2 --> B[Pod B on Node 2]
FW[Host firewall or cloud security group] -.may block specific ports.-> CNI2
Debugging cross-node connectivity, layer by layer
Reproduce and localize before changing anything:
$ kubectl get pod -o wide -n app
NAME READY STATUS NODE IP
pod-a 1/1 Running node-1 10.244.1.7
pod-b 1/1 Running node-2 10.244.2.9
$ kubectl exec -n app pod-a -- nc -vz 10.244.2.9 8080
nc: connect to 10.244.2.9 port 8080 (tcp) failed: Connection timed out
A timeout, not "connection refused," means the packet is being dropped somewhere in transit rather than rejected by pod-b's listener; that already rules out "the app isn't listening" and points at routing, the CNI, or a firewall.
- Confirm the listener exists:
kubectl exec -n app pod-b -- ss -tlnpshould show something bound to 8080; if not, this was never a networking problem. - Check node-to-node routing for the pod's address range:
$ ip route get 10.244.2.9
10.244.2.9 via 10.0.1.2 dev eth0 src 10.0.1.5
A missing route, or one pointing at the wrong interface, means the CNI failed to program cross-node routing, usually because its daemonset is crashlooping:
$ kubectl get pods -n kube-system -l k8s-app=calico-node
NAME READY STATUS RESTARTS
calico-node-x7k2p 0/1 CrashLoopBackOff 14
- If the failure is specifically Service-IP traffic rather than direct pod-IP traffic, check kube-proxy's own health the same way (
kubectl get pods -n kube-system -l k8s-app=kube-proxy) and inspect its programmed rules (sudo iptables -t nat -S | grep <service-name>orsudo ipvsadm -Ln, depending on mode). - If routing and the CNI both look healthy, check host-level and cloud firewall rules for the specific port. CNI and kube-proxy problems tend to affect all ports uniformly; a single-port failure with everything else in that layer healthy is a strong signal for a firewall rule scoped to that port.
- Capture packets at the suspect boundary to confirm rather than infer:
sudo tcpdump -n -i any host 10.244.2.9 and port 8080on the destination node shows whether the packet arrives at the host at all, before deciding whether the problem is upstream (routing) or downstream (host firewall, pod network namespace).
Trade-offs and pitfalls
- iptables rule count grows linearly with Services and endpoints and becomes a real CPU cost past a few thousand Services; IPVS or nftables scale better, but changing kube-proxy mode cluster-wide is a disruptive change to plan and test, not to flip live.
- The distinction between "Connection timed out" and "Connection refused" is easy to skip past but changes where to look first: refused means the packet arrived and something rejected it (wrong port, no listener); timed out means the packet was dropped somewhere before that.
- MTU (maximum transmission unit) mismatches between overlay encapsulation and the underlying network are a classic cause of connections that work for small packets and hang for large ones. Confirm with a full-size, non-fragmenting ping rather than assuming a working small ping rules out MTU.
Describe the kubectl commands and rollout strategies you would use to perform a safe rolling restart of a Deployment, view rollout history, and rollback to a previous revision. Include examples using kubectl and explain how you would avoid causing cascading failures during a restart of a consumer‑facing service.
Sample Answer
A safe restart uses kubectl rollout restart, which recreates pods through the normal RollingUpdate strategy rather than deleting them directly, so the same availability guarantees that protect a routine deployment protect the restart too.
Commands
Trigger and watch a rolling restart:
kubectl rollout restart deployment my-app -n prod
kubectl rollout status deployment my-app -n prod --watch
View rollout history and inspect a specific revision:
kubectl rollout history deployment my-app -n prod
kubectl rollout history deployment my-app -n prod --revision=3
Roll back:
kubectl rollout undo deployment my-app -n prod --to-revision=3
kubectl rollout status deployment my-app -n prod
Ship an image change with a recorded reason (the --record flag some older references use for this is deprecated; annotate explicitly instead):
kubectl set image deployment/my-app my-app=registry/app:1.2.3 -n prod
kubectl annotate deployment my-app kubernetes.io/change-cause="bump to 1.2.3, ticket OPS-441" --overwrite -n prod
What keeps a restart from becoming a cascading failure
- RollingUpdate parameters:
maxUnavailableandmaxSurge(both default to 25% of desired replicas) bound how many old pods can be down and how many extra new pods can exist at once. For a consumer-facing service, a conservative setting (for examplemaxUnavailable: 0, maxSurge: 1) never drops capacity below the current replica count during the restart, at the cost of briefly running more pods than the steady-state count. - Readiness probes: a Service only sends traffic to pods that pass their readiness probe, so a newly restarted pod that's still initializing doesn't receive requests it can't yet handle. This is the single biggest lever against a restart-induced error spike; without a readiness probe, the rollout has no signal that a "new" pod is actually ready and can start routing traffic to it immediately.
- PodDisruptionBudget (PDB): guarantees a minimum number (or percentage) of replicas stay available throughout the restart, independent of the Deployment's own
maxUnavailablesetting, which matters when other voluntary disruptions (a node drain, a cluster upgrade) happen to overlap with the restart window. - Graceful shutdown: a
preStophook plus aterminationGracePeriodSecondslong enough for in-flight requests to finish, combined with the Service removing the pod's endpoint before the container actually stops, avoids dropping requests that were already in progress when the restart began. - Staged rollout for risk-sensitive services: restarting (or deploying) to a small subset first, watching error rate and latency, then proceeding, catches a bad new revision before it reaches full traffic; this is a general staged-rollout practice, not a specific traffic-splitting mechanism (traffic-splitting techniques like weighted canary routing are a load-balancing/ingress-layer concern, not something the Deployment object itself provides).
Trade-offs and pitfalls
kubectl rollout restartonly recreates pods; it does not change the Deployment's spec, sorollout historyrecords it as a new revision with the same template, which is easy to forget when later trying toundoyour way back past a restart that changed nothing.- Setting
maxUnavailable: 0guarantees no capacity loss but requires enough spare cluster capacity formaxSurgeextra pods to schedule; on a tightly packed cluster this can leave the rollout stuck Pending on the surge pods instead of proceeding. - A rollback only restores the pod template (image, env, resource requests, and so on). If the Deployment reads a ConfigMap or Secret by a fixed name and that ConfigMap was edited in place rather than replaced with a new name or hash-suffixed name, rolling the Deployment back does not restore the old configuration content, only the old pod template pointing at the same (already-mutated) ConfigMap. This is the most common way a rollback fails to actually roll back.
- The same gap applies to a PersistentVolumeClaim (PVC): a Deployment's rollback restores the pod template's volume mount references, not the data on the volume itself. If the new version wrote a schema migration or otherwise mutated data in place on that volume, rolling the Deployment back gives you the old code pointing at already-changed data, not the old data. Anything stateful needs its own restore path (a volume snapshot or application-level backup) alongside the Deployment rollback, not instead of thinking about it separately.
Describe Kubernetes Service types (ClusterIP, NodePort, LoadBalancer, ExternalName). Explain how each type routes traffic to pods, typical cloud provider integrations, operational limitations, and how kube-proxy implementation affects packet forwarding behavior.
Sample Answer
A Kubernetes Service is a stable virtual address in front of a changing set of pods. The four types differ in who can reach that address and how: ClusterIP is internal-only, NodePort adds a port opened on every node, LoadBalancer asks the cloud provider to provision an external load balancer in front of that, and ExternalName is different in kind: a DNS alias to something outside the cluster with no pod routing involved at all.
Service types
| Type | Reachable from | How traffic reaches pods | Typical cloud integration |
|---|---|---|---|
| ClusterIP | inside the cluster only | kube-proxy rewrites the Service's virtual IP to a pod IP | none needed |
| NodePort | any node's IP, on a fixed port in the 30000-32767 range | same in-cluster rewrite as ClusterIP, reached via the node's IP first | often sits behind a hand-rolled external load balancer |
| LoadBalancer | the internet, or a private network, via a provisioned load balancer's address | the provider's load balancer forwards to node ports, which kube-proxy then routes to pods | automatic provisioning on AWS, GCP, or Azure through the cloud-controller-manager |
| ExternalName | resolves to an external DNS name; no cluster-internal routing at all | none; CoreDNS returns a CNAME and the client connects directly | giving a managed external service (e.g., a managed database) a cluster-local name |
A fifth variant sits alongside these four rather than replacing any of them: a headless Service (clusterIP: None). It skips kube-proxy's virtual-IP rewriting entirely, so instead of one stable IP load-balancing across pods, CoreDNS returns the individual pod IPs behind the Service directly to the client (sourced from the same EndpointSlices kube-proxy would otherwise consume). This is why StatefulSets pair with a headless Service: each replica needs its own stable, individually addressable DNS name rather than one shared address in front of all of them.
kube-proxy: the mechanism underneath
kube-proxy is what actually implements the ClusterIP, NodePort, and LoadBalancer rewriting, and it has more than one implementation:
- iptables mode (the long-standing default): programs Linux netfilter rules to destination-NAT (DNAT, rewriting a packet's destination address) Service traffic to a pod IP. Simple, but rule evaluation is roughly linear in Service count, which shows up as latency at high Service counts.
- ipvs mode (IP Virtual Server): uses the kernel's IPVS load-balancing tables instead of a rule chain, giving better performance and more scheduling algorithm choices at scale.
- nftables mode: the newer replacement backend, built on the modern Linux nftables framework, which reached general availability in Kubernetes 1.33. It targets the same correctness as iptables mode with materially better performance at high Service counts, though iptables remains the cluster-wide default for now.
Older material sometimes still mentions a userspace mode; it was deprecated years ago and fully removed in Kubernetes 1.26, so it should not appear in any current design.
Worked example: tracing one request through a LoadBalancer Service
flowchart LR
Client --> ELB[Cloud load balancer]
ELB --> NP[Node: NodePort]
NP --> KP[kube-proxy DNAT rule]
KP --> PodA[Pod A]
KP --> PodB[Pod B]
The client hits the cloud load balancer's public address, which forwards to a NodePort on whichever node it picked, where kube-proxy's DNAT rule sends the packet to one of the pods listed in the Service's EndpointSlices. With externalTrafficPolicy left at its default Cluster, the node that first receives the packet can forward it on to a pod on a different node, an extra hop, but with even load spread across all pods. Setting it to Local skips that extra hop and preserves the client's real source IP, at the cost of only routing to pods that happen to already be on the node the load balancer chose, so an uneven pod distribution across nodes turns directly into uneven traffic.
Trade-offs and pitfalls
- ExternalName is often reached for as 'the DNS one' when someone actually wants a placeholder Service for pods that do not exist yet. Because it has no selector and no health checking, nothing about it verifies the external target is even up.
- NodePort's fixed 30000-32767 range and one-port-per-Service-per-node model does not scale past a modest number of exposed Services and puts every node's IP in the attack surface; it is a building block for an external load balancer, rarely the final production answer on its own.
- LoadBalancer provisions one cloud load balancer per Service by default, which gets expensive and slow to provision as the number of Services grows; that pressure is usually solved by putting an Ingress controller behind a single LoadBalancer Service, a platform decision distinct from Service type selection itself.
- kube-proxy's mode is a cluster-wide setting, not a per-Service choice; moving from iptables to ipvs or nftables needs a validated cluster-wide rollout plan, not a Service-by-service change.
Explain the differences between Deployment and StatefulSet. Provide a scenario (e.g., a database cluster) that requires StatefulSet semantics and explain which StatefulSet features (stable network IDs, ordinal indexes, persistent volumes) are necessary.
Sample Answer
Deployment and StatefulSet both manage a set of pods from one template, but a Deployment treats every pod as interchangeable, while a StatefulSet gives each pod a stable, ordinal identity and its own dedicated storage. Reach for StatefulSet only when the application itself needs to know which replica it is, not just how many replicas exist.
Direct comparison
| Deployment | StatefulSet | |
|---|---|---|
| Pod naming | random suffix, no ordering | ordinal suffix (pod-0, pod-1, ...), stable across restarts |
| Network identity | pod IP changes on recreation; normally fronted by a load-balancing Service | stable hostname per ordinal, usually via a headless Service (clusterIP: None), so pod-0.svc.namespace.svc.cluster.local always resolves to the same replica |
| Storage | usually shared, external, or none | one PersistentVolumeClaim (PVC, a request for storage) bound to a PersistentVolume (PV, the actual storage resource) per ordinal, via volumeClaimTemplates; deleting the StatefulSet does not delete these PVCs |
| Startup and scale order | no ordering guarantee | default OrderedReady policy starts and stops one ordinal at a time, lowest-to-highest on scale-up and highest-to-lowest on scale-down; a Parallel policy is available when the app does not need this ordering |
Worked example: a PostgreSQL primary-replica cluster
A database cluster with one primary and several read replicas needs exactly the guarantees a Deployment cannot give:
- Stable hostnames: replication configuration and client connection strings can target
postgres-0.postgres-svcby name, and that name always means the same replica, even across restarts. - Ordinal indexes: an init script can branch on the pod's ordinal (ordinal 0 initializes as primary, ordinals 1 and above join as replicas) and rely on ordered startup so replicas do not attempt to join before the primary exists.
- Per-ordinal persistent volumes:
volumeClaimTemplatesties each replica's data directory to its own ordinal, sopostgres-1always reattaches topostgres-1's data after a restart, never to another replica's.
Trade-offs and pitfalls
- Not everything with 'stateful' in its name needs a StatefulSet. Kafka brokers do: each broker owns local partition log segments on disk and needs a stable identity for replica assignment. Kafka Connect workers do not: a Connect worker's task state and offsets live in Kafka topics, not on local disk, so the worker pool is effectively stateless and runs fine as a plain Deployment. Confirm where the durable state actually lives before defaulting to StatefulSet.
- StatefulSet's ordered rollout is slower and more conservative than a Deployment's by design; if a stateful workload's replicas are genuinely independent (sharded, no leader election), that ordering guarantee adds rollout latency with no real benefit, and a
Parallelpod-management policy, or even a Deployment with per-shard volumes mounted individually, may fit better. - Deleting a StatefulSet leaves its PVCs behind on purpose, to protect data from an accidental delete. Teams that do not know this get surprised in both directions: unexpected data loss if they assumed cleanup was automatic, or orphaned storage cost if they never clean it up.
Unlock Full Question Bank
Get access to all Kubernetes Architecture, Operations, and Troubleshooting interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.