Direct answer
A load balancer is like a receptionist routing visitors to available staff. An L4 (transport-layer) balancer only reads the outside of the envelope, the source and destination IP address and port, and forwards traffic without opening it: fast and generic, but blind to what is actually being requested. An L7 (application-layer) balancer opens the envelope and reads the HTTP request itself (the path, headers, cookies) before deciding where to send it: more work per request, but it can make smart, content-aware routing decisions. For a multi-tenant API product, that difference determines whether we can treat tenants and endpoints differently or only treat raw connections the same.
Performance, in plain terms
L4 does less work per request because it never looks past the network envelope, so it scales to more concurrent connections on the same hardware and adds negligible processing overhead. L7 has to parse and often decrypt (TLS-terminate) every request, which costs more compute per request and adds a processing step before the request reaches a backend. The trade is: L4 is cheaper and simpler, L7 is more expensive but earns that cost back in flexibility.
Routing flexibility, in plain terms
L4 can only split traffic by connection-level facts: which IP called, which port. It cannot tell a /v1/search request from a /v1/billing request, or tenant Acme from tenant Globex, because it never reads that far into the message. L7 can route on any of that: by URL path, hostname, header, or cookie. That is what lets us send one tenant's traffic to a dedicated pool (for isolation or a different SLA), route search and billing to differently-scaled backend pools, or run a canary release that only certain requests hit.
Observability, in plain terms
L4 sees connections and bytes: how many connections are open, how much data moved, whether a TCP handshake succeeded. It cannot tell you whether the response was correct. L7 sees the actual application outcome: HTTP status codes (200 vs 500), per-endpoint latency, which tenant is generating errors. For a product team, L7-level dashboards are what let us say "tenant Acme is seeing elevated error rates on checkout" instead of just "some connections are slow."
What session affinity (sticky sessions) changes
Session affinity means the load balancer remembers which backend served a given user (usually via a cookie) and keeps sending that user back to the same backend. Enabling it requires L7 (the LB has to read the cookie), and it is a product decision with real engineering consequences, not just a checkbox:
- Engineering can no longer treat every backend instance as interchangeable. If a sticky user's backend is taken down for a deploy or crashes, that user needs somewhere safe to land, so the team needs a shared session store or a way to migrate that user's state.
- Autoscaling gets harder: removing an instance now means displacing every user pinned to it, not just draining idle connections.
- Deploys need a drain step: an instance being retired has to stop accepting new sticky users while letting existing ones finish, instead of just being killed.
Worked example
Suppose our multi-tenant API has three backend pools behind an L7 balancer, and one legacy endpoint keeps a large in-memory shopping cart per user (no external store yet). Without stickiness, a user's second request could land on a different instance that has never seen their cart, and the cart would appear empty. Enabling session affinity on just that endpoint fixes the symptom immediately: same user, same backend, cart stays intact. But it means that endpoint's backends are no longer freely interchangeable, so before shipping it engineering needs to document, for that endpoint specifically: how long the affinity cookie lasts, what happens to a user's session if their pinned instance dies (do they lose the cart, or is there a fallback), and how deploys will drain sticky users instead of dropping them.
Trade-offs and pitfalls
Sticky sessions are a workaround, not a design goal: they trade operational flexibility for a quick fix to statefulness. The stronger long-term answer is usually to move the state itself (to a shared cache or database) so any backend can serve any user, and reserve stickiness for cases where that migration is not worth it yet, such as a legacy endpoint scheduled for rework. As TPM, the artifact I would insist on before approving a sticky-session change is a short doc that states which endpoints are affected, the cookie's lifetime and fallback behavior, and the updated deploy and drain procedure, so this doesn't become an undocumented constraint engineering has to relearn during the next incident.
