Zero Trust, Segmentation, and Service-to-Service Security Questions

Direct answer: the three approaches sit on a spectrum from infrastructure-managed identity (mutual TLS via a mesh) to application-issued identity (signed tokens) to cloud-managed identity (native identity and access management, IAM, roles), and for a multi-team, multi-namespace environment the right choice usually is not picking one, it is using mesh-based mutual TLS for baseline service-to-service trust, then layering the other two in for specific needs like cross-cloud calls or user-context propagation.

Mutual TLS via service mesh with workload identities (for example SPIFFE/SPIRE, a standard and implementation for issuing cryptographic workload identities): trust is rooted in a shared certificate authority (CA) every workload and the mesh agree on, and each workload gets a short-lived certificate tied to its identity. Rotation is automatic and frequent, minutes to hours, handled transparently by sidecar proxies. Operational overhead is highest to stand up (control plane, CA infrastructure, sidecar injection) but lowest ongoing per call, since verification happens at the network layer with no app code changes.

Signed JSON Web Tokens (JWTs) from a central authority: a central issuer signs tokens, and any verifier trusting the issuer's public key validates a token locally, no callback needed. The issuer rotates its own signing key on a schedule, published via a key-set endpoint verifiers refresh; the tokens themselves are simply re-issued rather than "rotated." Operational overhead is low to start, but every service must correctly validate audience, expiry, and issuer itself, a per-service correctness burden instead of a centralized one.

Cloud-native IAM roles or client certificates: the cloud platform attests that a given piece of compute is who it claims to be, based on the platform's own control plane, and hands out short-lived credentials on request, fully managed rotation, no certificate files for engineers to touch. Overhead is lowest for workloads staying inside one cloud provider, but this does not natively extend across clouds or to on-premises workloads, and ties your identity model to that provider's semantics.

Deciding for a multi-team, multi-namespace environment: if most traffic is service-to-service inside one platform, mesh-based mutual TLS gives every team the same baseline identity and encryption without each team writing its own auth code, which matters more as team count grows. Add signed JWTs where a human user's identity or fine-grained claims need to travel through a call chain. Reach for native cloud IAM roles specifically at the boundary where a workload calls a CLOUD service, a storage bucket, a managed queue, not another one of your own services, since that is the boundary the cloud provider's identity model is actually built for.

Worked example (a cloud-IAM branch in practice): a nightly batch job running as a Kubernetes pod needs to read from a cloud storage bucket. Instead of embedding a static access key, the pod's service account is federated to a cloud IAM role, for example AWS's IAM Roles for Service Accounts (IRSA), and at startup the job exchanges a short-lived, automatically-rotated token for temporary credentials from the cloud's Security Token Service (STS), typically valid for under an hour. If the pod is compromised, the exposed credential expires within that window, rather than being a static key that works indefinitely until someone manually revokes it.

Trade-offs & pitfalls: mixing all three without a clear rule for which is used when is itself a common failure, teams pick whichever they found a tutorial for, and you end up auditing three inconsistent identity systems instead of one. Cloud-native IAM roles are the easiest of the three to reach for and the easiest to over-scope, since there is no mesh-level review step forcing a second set of eyes on the grant.

Direct answer

That claim does not hold up: zero trust reduces the frequency and blast radius of breaches, it does not eliminate them, because it still depends on identities, credentials, and policy that can themselves be compromised or simply wrong. If an attacker obtains a legitimate, currently-valid identity, a phished token, a stolen service-account key, a compromised build pipeline, every zero-trust check will honor that identity exactly as it should, because cryptographically it is authorized.

Structured elaboration

Where the gaps remain:

• Identity-based access is only as strong as identity issuance and lifecycle management. Credential theft, session-token replay, or a compromised identity provider defeats it at the root, since everything downstream trusts that identity.
• Mutual authentication between services proves which service is talking, not that the service's logic or the human behind a request is behaving correctly. A legitimate service with a compromised dependency can still make destructive calls using its own valid identity.
• Policy enforcement points are only correct if the policy behind them is complete and current. A gap nobody thought to write, an overly broad default scope, a stale rule left over from an old integration, is not something the architecture closes automatically; a human still has to author the policy correctly, and human authoring is fallible.
• Zero trust also does not protect against fully authorized insider misuse or a supply-chain compromise inside code the identity is entitled to run: the request looks legitimate at every checkpoint because, by the rules of the system, it is.

Migration friction, moving from a permissive environment to zero trust:

• Developer friction: engineers used to broad, standing access (a shared service account with wide database permissions, SSH (secure shell) access anywhere) now hit explicit denials for previously-invisible dependencies, which slows delivery until the missing flows are identified and granted. The common failure mode is routing around the friction with overly broad, "temporary" grants that never get revoked, quietly recreating the old permissive model.
• Trust bootstrapping: early in a migration, new identity and policy infrastructure has to be trusted by systems that have no independent way yet to verify it. A new policy decision point typically has to run in shadow mode against production traffic before anyone is comfortable making it the sole authority, and the very first workloads onboarded often have nothing established yet to authenticate their own dependencies against, which is why pilots usually start with a small, self-contained set of services and a manually managed root of trust before automation exists.

Worked example

A payroll service uses short-lived, cryptographically issued service identities, and every request is authorized per call by a policy engine, a textbook zero-trust setup. An attacker compromises the build pipeline's deployment credentials, a supply-chain attack rather than a network attack, and pushes a malicious build that, once deployed, carries the payroll service's own legitimate identity. Every request that malicious build makes to the database is mutually authenticated, matches the policy that the payroll service is supposed to read and write payroll records, and passes every zero-trust check, because the compromise happened upstream of all of them, in the build pipeline, not in the network or the request path. Zero trust here limits what the attacker can do, only what the payroll service's identity is scoped to touch, but it does not prevent the breach, that requires supply-chain controls entirely outside the access model.

Trade-offs and pitfalls

The common wrong turn is treating zero trust as a project with an end state, "we're zero trust now, we're safe," rather than one layer of defense-in-depth that still needs supply-chain security, credential hygiene, detection and response, and correctly authored policy behind it. The friction and bootstrapping costs above are real and frequently underestimated in migration timelines.

Direct answer

An API gateway should handle the checks that make sense to do once, at the edge, before a request reaches any backend service: validating the request's token, checking coarse-grained scopes or claims against a policy decision point, and applying input validation and rate limiting. It cannot be the only enforcement point, though; backend services still need to independently authorize the specific action being requested, because a gateway can't reason about business-logic-specific authorization that only the service itself understands.

Structured elaboration

• Token validation: the gateway verifies the request carries a valid, unexpired, correctly signed identity token before doing anything else, rejecting malformed or expired tokens immediately and cheaply at the edge, rather than every backend service redundantly re-implementing that same check.
• Scope or claim-based authorization integrated with a policy decision point (PDP): beyond confirming a token is valid, the gateway checks whether its claims and scopes are sufficient for the requested endpoint, in some designs by consulting the PDP directly, so coarse-grained access decisions, whether this identity can call this API at all, happen once at the gateway rather than being reimplemented per service.
• Input validation: the gateway rejects malformed requests, wrong types, oversized payloads, invalid characters, before they reach a backend service's own parsing logic, reducing the attack surface each service has to defend on its own.
• Rate limiting: the gateway throttles request volume per identity or client, protecting backend services from a misbehaving client or a deliberate abuse pattern, again more efficient to enforce once at a shared edge than duplicated everywhere.

What must still be enforced by the backend services themselves: the gateway can confirm identity and coarse access, that a token is allowed to call the orders API, for example, but it generally cannot know fine-grained, resource-specific, business-logic-dependent rules: whether this specific user may view this specific order they did not create, whether an account's current state permits a specific action, or anything that depends on data only the service has. Those checks have to happen in the service itself, at the point where it actually holds the relevant data. Otherwise you get a well-known class of bug where the gateway correctly confirms a request is allowed to call an endpoint, but the service forgets to check whether the specific resource in that request actually belongs to the caller, sometimes called broken object-level authorization, precisely because it's easy to assume the gateway already handled "authorization" when it only handled the coarser layer.

Worked example

A request arrives for a specific order's details, with a valid token belonging to user A. The gateway validates the token, confirms it is unexpired, checks that its scope includes reading orders, confirms the request is well-formed, and checks that user A hasn't exceeded their rate limit, then forwards the request to the orders service. All of that is legitimate and correctly enforced at the gateway. But the requested order actually belongs to user B, not user A. If the orders service assumes the gateway already handled authorization and simply returns whatever order matches the identifier in the request, user A can read user B's order despite every gateway-level check passing correctly, because the gateway had no way to know which specific order belongs to which specific user, only the orders service, with access to that data, can make that check.

Trade-offs and pitfalls

The most common and serious mistake is exactly the example above: treating gateway-level scope checks as equivalent to full authorization and skipping the service-level ownership check, which produces vulnerabilities that pass every gateway-level test because the gateway was never the layer that could have caught them.

Direct answer

At millions of authorization checks per second with a single-digit-millisecond latency budget, a per-request network round trip to a central decision point is off the table. The architecture has to push policy evaluation out to where the request already is, in-process or same-host, and treat the central policy decision point (PDP) as a policy distributor and source of truth rather than a per-request decision maker.

Structured elaboration

• Caching strategy: cache the compiled policy itself at each enforcement point, not just individual decisions, so every request is evaluated fully in-process, sub-millisecond, against a locally held policy set. No network hop sits on the request's critical path at all.
• Policy distribution: the central PDP computes and pushes policy changes out to every policy enforcement point (PEP) through a publish-subscribe or streaming mechanism rather than PEPs polling for updates, so updates propagate without adding per-request cost. Distribution should send deltas once a PEP already has a baseline, not the full policy set every time, to keep propagation fast as the policy set grows.
• Consistency trade-offs: this is an eventually-consistent system by design, there will always be a short window where different PEPs run slightly different policy versions during a rollout. You choose the size of that window against how urgently a change needs to take effect everywhere, and for something as urgent as revoking a compromised credential, a separate, higher-priority fast path (or a short-lived local cache specifically for revocation status) is usually layered on top of the bulk distribution mechanism, precisely because eventual consistency is not acceptable for that one case.
• Fault tolerance: each PEP keeps serving decisions from its last-known-good local policy if it loses contact with the distribution mechanism, failing static rather than failing open or failing entirely, and the distribution mechanism itself needs redundancy so one central PDP instance being down only delays the next update rather than stopping policy from ever reaching PEPs.
• Rolling out updates without violating the latency budget: since evaluation already happens locally and in-process, a policy update itself never touches per-request latency. The risk is only in how the update is applied at each PEP, so it should be deployed as an atomic swap to a newly compiled ruleset, not a live in-place edit of the structure currently being evaluated concurrently.

Worked example

As a capacity-planning illustration, not a benchmark claim, suppose the system must sustain 2,000,000 authorization checks per second in aggregate. If policy evaluation happens locally in-process at each of 500 service instances, each instance only needs to sustain 2,000,000 divided by 500, or 4,000 checks per second locally, comfortably within what an in-memory rule evaluation does in well under a millisecond, with no network call involved at all. Contrast that with routing all 2,000,000 checks per second to a central PDP cluster: even a well-provisioned cluster doing 50,000 evaluations per second per node would need 2,000,000 divided by 50,000, or 40 PDP replicas, just to keep up, and every one of those checks would still pay a network round trip, exactly the latency and infrastructure cost the local-cache design avoids.

Trade-offs and pitfalls

The biggest mistake is treating "a central PDP" and "a central point of decision for every request" as the same thing; at this scale they must not be. The second is under-investing in the revocation fast path and discovering during an incident that a compromised credential's access takes as long to stop working everywhere as your slowest, most conservative policy-propagation setting.

Comparing coarse VLANs and subnets to fine-grained microsegmentation is a trade-off between blast-radius control and operational overhead: microsegmentation is strictly more effective at limiting lateral movement, but costs more in policy volume, tooling, and day-to-day upkeep. For a fast-growing company with a small operations team, jumping straight to full microsegmentation everywhere usually isn't the right call; a staged path that reserves fine-grained policy for the highest-value assets gets most of the security benefit for a fraction of the operational load.

Comparing the two approaches

Recommendation for a fast-growing company with a small team

A staged path works best: first, segment coarsely by trust tier, public-facing, internal, data, so there's at least one real boundary an attacker has to cross. Second, identify the small set of highest-value assets, the customer data store, payment processing, admin or control-plane tooling, and apply fine-grained microsegmentation only there, since that's where a breach is most costly and the legitimate callers are usually few and well-known. Third, grow fine-grained coverage outward as the team and tooling mature, instead of trying to boil the ocean on day one. Going straight to full microsegmentation with a small team usually means the rules go stale (nobody has time to keep them current) or drift toward a de facto allow-all, which is worse than a well-maintained coarse boundary.

Worked example

A thirty-person startup runs forty microservices. Full pairwise microsegmentation for all forty could mean up to 40 x 39 = 1,560 ordered pairs to reason about. Instead, group the services into four or five zones by data sensitivity, public API, internal business logic, payments, data store, admin tooling, apply default-deny between zones, and add per-workload rules only inside the payments and data-store zones. That keeps the rule count in the dozens rather than the thousands, small enough for a small team to actually keep correct.

The catch

The biggest pitfall is rule sprawl without an owner: a policy asset nobody actively maintains eventually becomes either a security gap (stale, overly permissive) or a reliability problem (blocks a legitimate change nobody remembered to allow). Staging by risk means accepting a wider blast radius for lower-value assets in exchange for keeping the high-value ones tight and correct.