Scalability & Capacity Planning Questions

Confirming the network really is the binding constraint

I wouldn't take the symptom (link saturation, rising query latency) at face value without ruling out other causes first. I'd check network interface utilization on both the app server and database side during the exact peak window, and correlate its onset with the onset of the latency increase, while also checking CPU and disk I/O on both tiers over the same window. For example: NIC (network interface) throughput at 950 Mbps sustained out of a 1 Gbps link (95% utilized), with TCP retransmissions (packets being resent because they weren't acknowledged in time, a sign the network link is congested) rising right when p99 latency (the 99th percentile response time, the slowest 1% of requests) crosses 300ms, while app server CPU sits at 40% and database CPU at 55%, both well under their own ceilings. That combination, network metric pegged and rising retransmits at the same moment latency spikes, while compute stays comfortable, is what actually confirms the network link as the binding constraint rather than assuming it from the topology alone.

Buying headroom quickly

Without touching the architecture, I'd reduce the bytes crossing that link: stop over-fetching (select only the columns actually needed instead of every column), compress payloads, batch queries to cut per-query overhead, and move some read traffic to a local cache or replica so it never has to cross the saturated link at all. These changes can ship in days and buy real headroom while a longer-term fix is planned.

The longer-term architectural fix

The quick fixes reduce load on the shared link, but the underlying issue is that traffic growth keeps colliding with a fixed-bandwidth path between two tiers that are architecturally coupled. Longer term I'd look at moving the app and database tiers onto a higher-bandwidth or lower-latency path (a dedicated link, tighter physical or network placement to cut hop count), and reducing chattiness structurally, colocating read replicas closer to the app servers, or adding a caching layer so most reads never round-trip to the database at all.

What happens next

Once the network stops being the ceiling, growth will eventually push the next resource, likely database CPU or the app tier itself, into becoming the new binding constraint. This isn't a one-time fix, it's the same identify-the-constraint exercise that needs to run again at the next growth checkpoint.