Google Cloud Platform Services and Architecture Questions
Google Cloud Platform's core services and architecture: Compute Engine, Cloud Run, GKE, Cloud Storage, VPC, managed databases (Cloud SQL, Spanner, Firestore, Bigtable), and BigQuery-adjacent data services. Covers GCP service selection, networking, IAM and security specifics, cost and quota management, and reference patterns for building on the platform. For provider-agnostic compute, storage, or networking concepts, see the cross-cloud entries.
You need to scale a GKE cluster to handle a seasonal peak that's 20 times the baseline traffic, then scale back down to control cost during the off-peak period. What would you do with node auto-provisioning, cluster autoscaler tuning, and surge upgrades, and how would you validate the scaling behavior ahead of the actual peak?
Sample Answer
Direct answer
Treat this as a steady-state problem with a seasonal problem layered on top, not one problem. First get the steady-state autoscaling right, a cluster comfortably serving a baseline load, say 1,000 requests per second per service at a 99.9 percent availability target, with pod disruption budgets protecting that availability during any voluntary disruption. Only then layer on the seasonal challenge: pre-provisioning enough headroom that the autoscaler isn't fighting a 20x step function on its own, using node auto-provisioning for capacity, careful cluster autoscaler tuning for how fast that capacity actually shows up, and surge upgrades so a routine node upgrade during the ramp doesn't compete with the scaling event for resources.
Building the plan
The steady-state foundation
A pod disruption budget (PDB, a policy that caps how many replicas of a service can be voluntarily taken down at once, for example during a node drain or cluster upgrade) protects the 99.9 percent availability target from your own maintenance actions, not just from traffic. At 1,000 requests per second with, say, 3 replicas required to stay available at all times, the PDB stops the cluster autoscaler or an upgrade from draining nodes in a way that would drop below that floor. Get this right before even thinking about the seasonal event, otherwise the 20x scale-up's own node churn can trip the exact availability floor you're trying to protect.
Node auto-provisioning
Node auto-provisioning (NAP) lets the cluster create entirely new node pools automatically, matching whatever machine shapes pending pods actually need, instead of forcing you to pre-create every node pool shape you might need for peak. At 20x scale, demand doesn't necessarily just mean "more of the same machine type," a spiky season might shift the mix toward, say, more memory-heavy replicas, and NAP adapts the node pool mix instead of forcing everything through pools sized for baseline.
Cluster autoscaler tuning
The default autoscaler reacts to pending pods, it does not anticipate them. For a 20x step function, tune scale-up responsiveness and consider a scheduled or metrics-driven pre-scale ahead of the known peak window rather than relying purely on reactive scale-up, because provisioning new nodes and pulling container images onto them takes real time that a purely reactive autoscaler spends after demand has already arrived. On the way down, tune scale-down delay and utilization thresholds conservatively enough that the cluster doesn't thrash, scaling down right before a secondary demand wave, then scaling back up, during a seasonal tail that doesn't drop instantly back to baseline.
Surge upgrades
A surge upgrade creates extra nodes before draining old ones during a cluster or node pool upgrade, so capacity never dips during the upgrade itself. During a scaling event this matters because a routine upgrade should never compete with the autoscaler for capacity right when every bit of it is needed, either avoid scheduling upgrades inside the peak window entirely, or size the surge generously enough that the upgrade adds temporary capacity instead of borrowing it from an already-stretched cluster.
Validating ahead of the actual peak
Load-test against a replica of the intended peak shape, not just the peak volume: the full 20x request rate, with the same request mix and downstream dependency call patterns as the real event, run far enough ahead of the actual peak that a discovered bottleneck can genuinely be fixed in time. Specifically test the scale-down behavior in this drill too, a cluster that scales up beautifully but flaps or drops capacity too early on the way back to baseline is still a production incident, just one that happens after everyone has stopped watching the dashboards as closely.
Worked example
A retail service running steady state at 1,000 requests per second across 3 baseline replicas per service needs to absorb a 20,000 requests-per-second seasonal peak, 20 times baseline. A drill 3 weeks out replays production-shaped traffic ramping from 1,000 to 20,000 requests per second over 30 minutes, matching the real event's expected ramp rate rather than an instant step, against a staging cluster running the same autoscaler and node auto-provisioning configuration as production. The drill surfaces that a downstream payment-processing dependency's connection pool caps out well before 20,000 requests per second is reached, a bottleneck that has nothing to do with GKE's autoscaler and would have caused an outage at the real peak regardless of how well the cluster itself scaled. Because this was caught 3 weeks ahead of the real event, the team had time to raise the connection pool limit, something no amount of cluster-autoscaler tuning would have fixed.
Trade-offs and pitfalls
Tuning the autoscaler and node auto-provisioning for the seasonal event while never revisiting the steady-state pod disruption budget is a common pitfall, teams then discover during the ramp-down that the availability floor was violated by the cluster's own scale-down actions, not by traffic. Load-testing raw request volume without matching request mix or ramp shape is another, a flat 20x step test can pass while a realistic 30-minute ramp with a different traffic mix fails, because real bottlenecks often live in a specific dependency the flat test never exercised. A third pitfall is leaving scale-down thresholds as aggressive as they'd be for a low-variance steady-state workload, which causes capacity flapping during a seasonal tail that doesn't taper cleanly. It's worth naming the cost trade-off explicitly: pre-provisioning ahead of a known peak costs real money for capacity that sits partly idle during the ramp-in window, that is the price of not discovering the autoscaler's reaction lag during the actual event instead.
Design a GKE platform that many engineering teams in a large enterprise will share. Would you isolate tenants with namespaces or separate clusters, and how would you handle resource quotas, admission control, multi-cluster ingress, and self-service CI/CD without teams stepping on each other?
Sample Answer
Direct answer
Default to namespace-based multi-tenancy on a shared cluster, or a small number of clusters, for most teams, it is dramatically cheaper to operate and scales to more teams without a linear growth in clusters to patch and upgrade. Reserve dedicated clusters for the specific teams whose requirements namespace isolation genuinely can't satisfy, most commonly a hard compliance boundary, a need for cluster-level configuration a shared cluster can't offer, or a security posture that treats sharing a control plane and kernel with other tenants as unacceptable regardless of role-based access control (RBAC, Kubernetes' system for governing which identities can perform which actions) and network policy.
Designing the shared platform
Namespaces versus separate clusters
| Isolation need | Namespace-based | Separate cluster |
|---|---|---|
| RBAC and quota isolation | Sufficient | Sufficient, but overkill |
| Compliance requiring separate infrastructure | Not sufficient, shared control plane and nodes | Required |
| Operational cost | Low, one cluster to upgrade and patch | High, N clusters to upgrade and patch |
| Blast radius of a cluster-level incident | Shared across all tenants | Contained to one tenant |
| A team needing a different Kubernetes version or node OS | Not possible in a shared cluster | Native fit |
Resource quotas
Kubernetes ResourceQuota objects per namespace cap aggregate CPU and memory requests and object counts, so one team's runaway deployment can't starve every other team's namespace on the same cluster. LimitRange objects set sane per-pod defaults, so a team that forgets to set requests or limits doesn't get scheduled as if it needs zero resources, which risks node oversubscription, or blocked outright.
Admission control
A policy engine, Google's built-in Policy Controller or an admission webhook you run yourself, enforces platform-wide rules at object-creation time: no privileged containers, images only from an approved registry, every namespace carrying required labels for cost allocation, resource requests and limits mandatory. This turns "please follow the platform's conventions" from a wiki page into something actually enforced, a manifest violating policy gets rejected before it ever reaches a node.
Multi-cluster ingress
Even with namespace-per-team on one cluster, larger platforms often still run more than one cluster for blast-radius or capacity reasons. Multi-cluster ingress gives every team one consistent way to expose a service regardless of which underlying cluster it landed on, so "which cluster is this team's service actually in" stops being something application teams need to know or route around themselves.
Self-service CI/CD without teams stepping on each other
Give each team its own namespace-scoped deploy permissions, their CI/CD (continuous integration and continuous delivery) service account can only touch resources in their own namespace, enforced by RBAC, not convention, so teams can ship on their own schedule without needing the platform team to review or gate every deploy. The admission-control and quota layers above are what keep one team's self-service deploy from being able to affect another team's namespace even if it tried, which is what makes "self-service" honest, self-service that still requires a platform-team approval on every deploy isn't really self-service.
Worked example
Team A's CI/CD pipeline pushes a manifest requesting an unbounded amount of memory per pod, a genuine mistake, not malice. Because Team A's namespace has a LimitRange with a sane default and a ResourceQuota capping total memory for that namespace, the deploy either gets a sane default applied automatically or is rejected outright, either way it cannot consume enough of the shared cluster's capacity to evict Team B's pods or block them from scheduling elsewhere in the same cluster. Team B never notices anything happened, which is exactly the point, their service's reliability didn't depend on Team A's manifest being correct.
Trade-offs and pitfalls
Treating namespaces as "enough" isolation for a team with an actual compliance requirement for separate infrastructure is a common pitfall, RBAC and network policy reduce risk within a shared cluster but do not eliminate the shared-control-plane and shared-kernel blast radius a compliance auditor may specifically be asking about. Rolling out admission-control policies platform-wide with no exemption process is another, the first team with a legitimate edge case either breaks or works around the platform, often by requesting their own separate cluster, defeating the purpose, build a reviewable exception path in from the start. A third pitfall is giving every team cluster-admin-equivalent access "to move fast," the same failure as sharing one CI/CD service account across every environment a pipeline touches, self-service has to mean self-service within a boundary, not without one.
Design a secure GKE cluster for an enterprise with many teams sharing it. Cover network policies, RBAC, namespace isolation, Workload Identity, and how you'd enforce that only scanned, trusted images get admitted. How would you explain this design to a security team without making developers' lives miserable?
Sample Answer
Direct answer
Four independent layers, each closing a different attack surface: network policies restrict pod-to-pod traffic to only what's declared necessary, containing lateral movement; role-based access control (RBAC) and namespace isolation restrict who or what can act on the Kubernetes API itself; Workload Identity replaces long-lived service account keys inside pods with short-lived, per-workload credentials; and image admission control ensures only scanned, trusted images get scheduled at all. To a security team, these map onto four plain questions, can an attacker move sideways, can an attacker escalate privilege through the API, can a leaked credential be used broadly, and can untrusted code run here in the first place, which is a much stronger pitch than a bare feature list.
The four layers
Network policies, for lateral movement
Kubernetes NetworkPolicy objects are deny-by-default once any policy exists in a namespace, define exactly which pods can talk to which other pods, on which ports, instead of the flat, fully-open pod network GKE (Google Kubernetes Engine) gives you by default. This stops a compromised pod in one namespace from freely reaching a database-tier pod in another namespace just because they share the same cluster network.
RBAC and namespace isolation, for API-level control
RBAC governs who, or which service account, can perform which verbs on which API resource types, scoped to a namespace via RoleBindings rather than cluster-wide via ClusterRoleBindings wherever possible. Combined with namespace isolation, each team or environment in its own namespace with its own RBAC bindings, this stops a developer or a compromised CI/CD identity in one team's namespace from reading secrets or modifying deployments in another team's namespace.
Workload Identity, for credential exposure
Workload Identity binds a Kubernetes service account to a Google Cloud service account, so a pod authenticates to Google Cloud APIs with short-lived, automatically rotated credentials tied to its specific Kubernetes service account, with no downloaded key living inside the pod or the container image. This closes the same leaked-key risk that makes downloaded service account keys dangerous in general, applied specifically to workloads running inside the cluster.
Admission control, for image trust
An admission controller, Policy Controller for general policy or Binary Authorization specifically for image provenance, enforces that a pod can only be scheduled if its image passes policy: signed by a trusted build pipeline, scanned with no unresolved critical vulnerabilities, sourced from an approved registry. This answers "how do we know nothing untrusted is even running," independent of how well-behaved that code turns out to be once it is running.
Presenting this without making developers miserable
Frame each control around what it does not require developers to do manually: Workload Identity means developers stop thinking about service account keys at all, strictly less toil, not more. Image scanning happens in the CI pipeline before a developer's normal merge and deploy flow, so a failure surfaces as a familiar pipeline check, not a mysterious runtime rejection. Network policies should ship with a sensible default, allow within your own namespace, deny cross-namespace unless declared, so most teams never need to hand-write one. The one place friction is real, and should be named honestly, is a new cross-namespace dependency: a team that legitimately needs to call another team's service now has to explicitly request that network policy, a deliberate, reviewable speed bump, not an accident, and it's worth saying that out loud rather than pretending there's zero cost anywhere in this design.
Worked example
A developer's pod is compromised through an application-level vulnerability, unrelated to any of these four controls, they don't prevent the initial compromise, only what happens next. Network policy stops the compromised pod from reaching the database tier in a different namespace it was never declared to talk to. Even if it could reach something, RBAC means the pod's own service account has no permission to read Secrets or list workloads outside its own namespace. If the attacker tries to pivot to Google Cloud resources, Workload Identity means there's no downloaded key to steal, only a scoped, short-lived credential tied to that workload's narrow IAM grant. And the image that was compromised had already passed Binary Authorization, meaning the vulnerability was likely a runtime or application flaw rather than a known catalogued vulnerability (a CVE, a publicly documented security issue) the scan should have caught, a useful, honest thing to tell a security team, these controls narrow the blast radius of a compromise, they don't claim to prevent one.
Trade-offs and pitfalls
Rolling out NetworkPolicy cluster-wide with a default-deny stance and no migration plan is a common pitfall, it breaks every existing cross-namespace dependency at once and is exactly the kind of change that gives security controls a reputation for breaking things, roll it out namespace by namespace with existing traffic mapped first. Using ClusterRoleBindings out of convenience, because scoping every binding to a namespace is more setup work, quietly reintroduces the cross-team blast radius these controls exist to eliminate. Binary Authorization policies that are so strict any dependency update blocks a deploy, with no fast, reviewable exception path, teach teams to route around the control entirely, which defeats it. Being honest with the security team about the real, small, controlled friction this design does introduce buys credibility for the large amount of protection that costs developers nothing.
Architect a GKE deployment across two or more regions that needs zero-downtime deployments, cross-cluster traffic management, and secure service-to-service communication, tolerating a full regional failure. What role would a service mesh play, and how would you handle certificate management and global load balancing?
Sample Answer
Direct answer
Build this in two layers: a baseline of independent GKE (Google Kubernetes Engine) clusters in two or more regions behind a single global external load balancer, with a content delivery network (CDN, a globally distributed cache for static and cacheable content) in front, and rolling updates within each cluster handling zero-downtime deploys. Then add what makes this the harder version of the design: a service mesh spanning the clusters for consistent mutual TLS (mTLS, a handshake where both sides of a connection present and verify a certificate, not just the server as in ordinary TLS)-secured service-to-service traffic and fine-grained cross-cluster routing, plus a certificate strategy that doesn't become the thing that breaks during a regional failover.
Designing the architecture
The baseline: topology and global load balancing
Run independent GKE clusters per region rather than one cluster stretched across regions, GKE clusters are regional or zonal constructs, not global ones. Put each behind Google's global external Application Load Balancer, which uses anycast (the same IP address is advertised from every region at once, and ordinary network routing delivers each client to whichever advertising location is topologically closest) so client traffic naturally enters the nearest healthy region without any client-side region-selection logic. A CDN in front absorbs static and cacheable traffic, so a regional failure has less live traffic to fail over in the first place. Zero-downtime deploys within a single region are handled by Kubernetes' own rolling update mechanism, new pods pass readiness checks before old ones are terminated, and that mechanism doesn't change just because a second region has been added.
Tolerating a full regional failure
The global load balancer's health checks deliver the actual failover: when a region's backends stop passing health checks, the load balancer stops routing new traffic there and shifts it to the healthy region, with no DNS change and no manual step required. Unlike a database's cross-region replica, which typically needs a human to promote it before it can serve traffic, GKE's global load-balancing layer is designed to fail over automatically. What still needs explicit design is state: if services are stateless, as this baseline assumes, failover is close to seamless, but any stateful component behind these services needs its own cross-region replication and failover story, that is a separate problem from the load-balancing layer itself.
What a service mesh adds
A service mesh, Istio, or GCP's managed Cloud Service Mesh, gives every service consistent mTLS without each team hand-rolling TLS, plus fine-grained traffic policy: weighted routing, retries, circuit breaking, and multi-cluster service discovery so a service in one region's cluster can call a service in another region's cluster through the mesh itself, rather than through the public internet or another round trip through the global load balancer. This is what turns "two independent regional stacks that happen to share a load balancer" into one logical multi-region system with secure, observable service-to-service traffic.
Certificate management
Two certificate surfaces need different treatment. The public-facing certificate on the global load balancer should use Google-managed certificates so renewal is automatic and never becomes a manual step someone forgets right before an expiry-triggered outage. The mesh's internal service-to-service certificates should be issued and rotated automatically by the mesh's own certificate authority as part of the mTLS setup, this should never be a manually managed, long-lived certificate per service, that would reintroduce exactly the operational risk a service mesh exists to remove.
Worked example
A checkout service runs as identical deployments in two regions behind the global load balancer. One region goes down entirely, a real outage, not just one bad pod. The load balancer's health checks fail for that region's backends within its configured check interval, and traffic shifts to the surviving region, users see at worst one retried request each, not an outage. Meanwhile a routine deploy is rolling out to the surviving region's checkout service via a standard rolling update at the same time, unaffected by the regional failure since it's a per-cluster mechanism. The mesh's mTLS certificates for the checkout service's calls to the inventory service in the surviving region were already short-lived and auto-rotated, so nothing about the failover required anyone to touch a certificate.
Trade-offs and pitfalls
A common pitfall is assuming "multi-region" solves availability for stateful components the same way it does for the stateless web tier, a database or cache behind these services needs its own explicit cross-region strategy, or the failover is cosmetic. A second is hand-managed, long-lived internal certificates instead of the mesh's automatic rotation, precisely the kind of manual dependency that causes an outage unrelated to the regional failure the design was built for. A third is adding a full service mesh purely to get "zero-downtime deploys and multi-region," when the load-balancer-plus-rolling-update baseline already delivers most of that for a purely stateless system, the mesh's real justification is the secure service-to-service traffic and fine-grained routing requirement, name that requirement explicitly before taking on the operational complexity of running a mesh.
Design a CI/CD pipeline for deploying containerized microservices to GKE using Cloud Build and Artifact Registry, and pick a deployment strategy, rolling, canary, or blue-green. What security controls and testing phases would you build in, and what does your rollback procedure look like?
Sample Answer
Direct answer
For a typical containerized microservice on GKE (Google Kubernetes Engine), I would default to a canary rollout over rolling or blue-green: it exposes a new version to a small slice of real traffic and lets automated checks catch a regression before it reaches everyone, without paying for a full duplicate environment. Cloud Build (GCP's managed CI/CD service) drives the pipeline; Artifact Registry (GCP's managed image registry) holds the built image by digest so every stage deploys the exact same artifact that was tested.
Pipeline design
- Build and test: Cloud Build runs unit tests, builds the container, and pushes it to Artifact Registry addressed by digest, not a mutable tag.
- Deploy to staging: the same digest deploys to a staging GKE cluster.
- Integration and smoke tests: automated tests exercise real dependencies (a staging database, downstream services) and a lightweight smoke test confirms the service actually starts and answers health checks.
- Canary in prod: the new digest takes a small, fixed slice of production traffic (for example 10 percent) behind the same Service, while the previous version continues serving the rest.
- Automated canary analysis: compare error rate and latency between the canary and the baseline over a defined window; only on a pass does traffic progressively shift to 100 percent.
flowchart LR
A[Git push] --> B[Cloud Build: build and test]
B --> C[Push image to Artifact Registry]
C --> D[Deploy to staging GKE]
D --> E[Integration and smoke tests]
E -->|pass| F[Canary: 10 percent of prod traffic]
E -->|fail| R[Rollback: redeploy last good image]
F --> G{Error rate and latency healthy?}
G -->|yes| H[Progressive rollout to 100 percent]
G -->|no| R
Security controls
Cloud Build should run under a dedicated, least-privilege service account rather than its default one, scoped only to the registries and clusters it needs to touch. Secrets used during build or deploy belong in Secret Manager, never as plaintext build-config variables. On the running side, pods should use Workload Identity Federation for GKE (GKE's mechanism for letting a Kubernetes service account impersonate a Google service account without any node-level key; commonly still called just Workload Identity), so a compromised pod can only reach what its own identity was granted, not whatever the node happens to hold. A vulnerability scan gate on the pushed image (or a Binary Authorization policy, GCP's admission-time image-signing control) before the deploy step is a cheap addition that catches known-bad base images before they ever reach staging.
Rollback procedure
Because every deploy is pinned to an immutable digest and GKE keeps Deployment revision history, rollback means pointing the Service back at the last known-good digest, either a kubectl rollout undo or, if using Cloud Deploy, its built-in rollback action. That is fast because the previous image is already sitting in Artifact Registry; nothing needs to be rebuilt. Trigger it automatically off the canary analysis step failing, rather than waiting on a human to notice a dashboard, since the whole point of canarying is catching the regression before most users see it.
Trade-offs and pitfalls
Blue-green needs a full second environment running in parallel during cutover, roughly double the compute cost for the transition window, in exchange for an instant, unambiguous rollback (just flip traffic back). Rolling updates are the cheapest option but give no traffic-based safety net: a broken version becomes 100 percent of traffic before anyone necessarily notices, so I would only pick it for a low-risk, easily-reversible internal service. Canary needs real infrastructure to weight traffic and, more importantly, automated analysis to compare canary against baseline; running two replica sets behind one Service with no explicit traffic split and no automated comparison is not actually canarying, it is an uncontrolled rolling update with extra steps and false confidence.
Unlock Full Question Bank
Get access to all Google Cloud Platform Services and Architecture interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.