Cost Optimization at Scale Questions

Addresses cost conscious design and operational practices for systems operating at large scale and high volume. Candidates should discuss measuring and improving unit economics such as cost per request or cost per customer, multi tier storage strategies and lifecycle management, caching, batching and request consolidation to reduce resource use, data and model compression, optimizing network and input output patterns, and minimizing egress and transfer charges. Senior discussions include product level trade offs, prioritization of cost reductions versus feature velocity, instrumentation and observability for ongoing cost measurement, automation and runbook approaches to enforce cost controls, and organizational practices to continuously identify, quantify, and implement savings without compromising critical service level objectives. The topic emphasizes measurement, benchmarking, risk assessment, and communicating expected savings and operational impacts to stakeholders.

MediumSystem Design

43 practiced

Design a multi-region replication strategy for a read-heavy service that must keep read latency <100ms for 95% of global traffic and minimize cross-region egress charges. Provide options (active-active, active-passive, caching, geo-routing), their expected egress implications, and when each is appropriate.

Sample Answer

Requirements & constraints (clarify): read-heavy, 95th-percentile read latency <100ms globally, minimize cross-region egress charges, data consistency tolerance (eventual vs near‑real time), RPO/RTO, write throughput and write locality, number of regions and traffic distribution.

High-level options (summary, egress implications, when appropriate):

1) Active‑active multi-region replication- Description: Deploy read/write replicas in each region with conflict resolution or a primaryless datastore (e.g., DynamoDB global tables, CockroachDB, Cosmos DB).- Latency: Local reads <100ms for all regions.- Egress: Writes must replicate updates to other regions — cross-region egress proportional to write volume × (N‑1). Reads served locally so read egress minimal.- When to use: High global read+write traffic, strong availability needs, application tolerates eventual or merged conflict resolution. Use if write volume is low relative to read volume or you can batch/compact replication to reduce egress.

2) Active‑passive (primary in one region, read replicas elsewhere)- Description: Single primary handles writes; asynchronous replication to passive regions; reads served from read-only replicas.- Latency: Reads local <100ms if replicas warmed; writes from remote clients may incur higher latency if they must hit primary.- Egress: Replication from primary to other regions causes cross-region egress equal to write data × (N‑1); read egress negligible if reads stay local.- When to use: Lower write volume, strong single-writer consistency desired, simpler conflict model, cost-conscious when writes are infrequent.

3) Edge caching + origin replication- Description: Use CDN/Edge caches (CloudFront, Fastly) for reads; origin(s) updated by writes replicated between regions. TTLs and cache invalidation control staleness.- Latency: Reads to edge typically <<100ms globally.- Egress: CDN caches serve reads without egress from origin; egress is dominated by cache misses and origin updates. Cross-region egress limited to origin replication (can be reduced via centralized origin or origin in each region).- When to use: Extremely read-heavy, largely static or cacheable content, tolerant to short staleness. Best for minimizing egress costs for reads.

4) Geo-routing + regional primary for writes- Description: Route reads to nearest read replica; route writes to region determined by user geography (regional master per shard) or redirect to write region.- Latency: Reads local; writes optimized if users tied to a write region; cross-region write redirection adds latency otherwise.- Egress: Cross-region egress depends on replication strategy—if writes are sharded to region and replicated only for failover, egress low. If replication is full, egress equals write replication volume.- When to use: Multi-tenant, geo-partitionable workloads where users are naturally partitioned and writes are local to partitions.

Cost & egress optimizations (practical patterns)- Reduce replication payload: replicate deltas, compress, batch writes, or use change‑capture with compacted topics (Kafka MirrorMaker, CDC pipelines).- Configure replication topology: chain replication or hub‑and‑spoke to avoid full mesh (reduces N² egress).- Use TTL and smart cache invalidation to maximize CDN hit ratio.- Asynchronous and eventual consistency for cheaper replication; use synchronous only where necessary.- Leverage cloud provider native cross‑region replication pricing/discounts and use region placement to minimize egress (choose adjacent regions).

Decision guide (quick):- If reads are cacheable and strict freshness not required → Edge caching + single origin.- If writes are rare vs reads → Active‑passive or origin with CDN.- If global writes and high availability required → Active‑active, but budget for higher egress.- If data can be partitioned by geography → Geo-routing with regional primaries to minimize cross-region replication.

Example architecture snippets:- Read-heavy web API: regional read replicas + CDN in front of API responses; primary writes in single region with async CDC to replicas; chain replication to reduce egress.- Global low-latency app with writes: multi-master DB (DynamoDB Global Tables) with conditional writes and conflict resolution; use write throttling & batching to reduce egress.

Wrap-up: pick the least cross-region replication necessary to meet latency and consistency SLAs. Favor caching and regional primaries for read-heavy workloads; choose active‑active only when write locality or availability requirements justify the egress cost.

MediumTechnical

39 practiced

Describe how you would prioritize multiple cost-reduction opportunities across product teams when leadership asks for $X in monthly savings within 60 days. Detail the criteria you'd use to rank opportunities, communication plan with product owners, and quick governance to authorize changes.

Sample Answer

Situation: Leadership asked our org to deliver $X/month in savings within 60 days across multiple product teams.

Task: As the solutions architect leading the effort, I needed to rapidly prioritize opportunities, align product owners, and authorize safe, fast changes.

Action:- Rapid intake and scoring: I created a one-page template and asked teams to submit candidate actions (e.g., instance rightsizing, reserved instances, feature toggles to remove paid services, CI/CD pipeline pruning) with estimated monthly savings, implementation effort (days), customer impact, reversibility, and dependencies.- Prioritization criteria (used to rank each item): - Savings magnitude (absolute $/month) - Time-to-value (can be implemented within 60 days) - Implementation effort and cross-team coordination (low/medium/high) - Customer/user impact and SLA/risk (must be minimal or mitigatable) - Reversibility and observability (ability to rollback and monitor) - Compliance/security implications- Triage and short list: I scored each item and put them into A (high value/low risk), B (medium), C (low value/high effort). Focused first on A items until cumulative savings met $X.- Communication plan with product owners: - 24‑hour kickoff clarifying goal, constraints, and scoring rubric - Shared dashboard (spreadsheet + alerting) showing ranks, owners, timelines - Weekly 30‑minute syncs plus async updates in Slack; immediate escalation channel for blockers - Clear owner responsibilities: implement, test in staging, smoke test, deploy, verify savings - Customer communication plan templates if a user-facing change is needed- Quick governance to authorize changes: - Established a temporary 5‑person steering panel (CTO, Head of Product, Security Lead, Finance, me) empowered to approve A/B items within predefined thresholds (e.g., <$50k can be approved by panel lead; >$50k requires full panel) - Lightweight RFC: one‑page change description, risk checklist, rollback plan, monitoring metrics — approvals via Slack threads or a single Doc sign-off - Pre-approved safety gates: testing in staging, circuit-breaker metrics, automatic rollback conditions - Daily status report to leadership with cumulative savings and risks

Result: By focusing on A items (rightsizing, reserved instances, disabling unused add‑ons), we typically capture the bulk of short‑term savings quickly, while B/C items feed a 90‑day roadmap. This approach balances speed, risk control, and clear ownership so we meet the 60‑day target without compromising reliability or compliance.

Learning: Transparent scoring, clear owners, and a lightweight but empowered governance body enable fast, safe decisions under time pressure.

HardSystem Design

44 practiced

Design an automated, policy-driven enforcement system (policy-as-code) that blocks provisioning of oversized resources, prevents public egress-heavy configurations, and integrates with CI/CD pipelines and cloud consoles. Describe policy lifecycle, audit trails, exception handling, and how to safely roll out such enforcement across teams.

Sample Answer

Requirements & constraints:- Block provisioning of oversized resources (VM/DB/containers > organizational limits).- Prevent public egress-heavy configs (large outbound bandwidth to internet or public endpoints).- Integrate with CI/CD (pre-merge and runtime) and cloud consoles (real-time/preventative).- Policy-as-code, auditable, exception workflow, staged rollout, low false-positive impact.

High-level architecture:- Central Policy Engine (PCE) that evaluates policies written in a declarative language (e.g., Open Policy Agent/Rego or HashiCorp Sentinel).- Policy Registry + Git-backed repo for policy-as-code (PR-driven lifecycle).- Admission/Blocking Points: 1. CI/CD hooks (pre-merge lints, pipeline policy checks). 2. Infrastructure Provisioning (IaC plan-time—Terraform validate/plan checks). 3. Cloud Console & API (runtime enforcement via cloud-native policy frameworks: AWS Config + Service Control Policies, Azure Policy, GCP Org Policy or a control plane using cloud APIs to block). 4. Mutating/admission webhooks for Kubernetes.- Enforcement Agents/Connectors per platform that query PCE and enforce allow/deny or provide warnings.- Audit Log store (append-only, tamper-evident—e.g., S3+WORM or SIEM) and CI-integrations that attach policy scan reports to build artifacts.

Policy lifecycle:- Author: policies authored as code in feature branches with tests (unit and integration harness using representative IaC, manifests).- Review: PR workflow with automated policy linting and test runs in CI.- Stage: Canary environment and a "monitoring" enforcement mode that records violations but doesn't block.- Approve & Promote: Signed approval (owner + security) merges to main; CD deploys to PCE.- Enforce: Toggleable enforcement levels: advisory → soft-block (warn + require review) → hard-block.- Deprecate: Versioned policies; retire via staged deprecation notices and policy simulators.

Audit trails & observability:- Every evaluation logged: timestamp, actor, resource spec, policy id/version, decision, rationale, evaluation inputs, correlated CI/CD build id or console session.- Immutable storage + indexing (Elasticsearch/Opensearch or SIEM) for queries and alerts.- Dashboards for violations by team, policy, severity, and time.- Automated retention & export for compliance (e.g., SOC2/ISO).

Exception handling:- Short-lived exceptions via policy exceptions service: request with justification, risk owner, TTL, scope (resource, project, account). Requests create tickets (Jira) and are recorded in audit.- Exceptions evaluated by automated risk checks (e.g., compensating controls) and require approvals based on risk level.- Exceptions stored as allow-lists in policy repo with automated expiry and re-evaluation; expired exceptions trigger notifications and reapply enforcement.- Emergency bypass path (time-limited, multi-approver, fully logged) for critical outages.

Safe rollout strategy:1. Discovery phase: run policies in read-only mode across accounts to collect baseline metrics and false-positive rates.2. Pilot: enable soft-block in low-risk teams; gather feedback, tune thresholds and rule granularity.3. Phased enforcement: per-organization unit rollout with strict SLAs for remediations and dedicated support channel.4. Automation-first: integrate with CI scaffold (pre-commit hooks, PR checks) so developers get instant feedback before cloud interaction.5. Training & docs: publish policy catalog, runbooks for exceptions, and policy simulators for engineers.6. KPIs & gating: monitor false-positive rate < X%, MTTR for remediation, number of blocked deployments; only escalate to full enforcement when KPIs met.

Scalability & reliability:- PCE horizontally scalable, stateless, with cached policy bundles per-region.- Connectors use streaming (event-driven) for near-real-time evaluations; fallbacks degrade to advisory to avoid outages.- Use feature flags and gradual rollout (percentage-based) to limit blast radius.

Trade-offs:- Strong preventative enforcement reduces risk but increases developer friction — mitigate with good developer UX, early CI feedback, and automated remediation suggestions.- Centralized engine simplifies governance but introduces latency and single control plane; mitigate with caches, regional replicas, and local evaluation libraries.

Closing notes:- Emphasize policy test coverage, clear ownership, and measurable KPIs. This design balances safety (blocking risky provisioning) with developer productivity (CI feedback loops and controlled exceptions), and provides full auditability required for enterprise compliance.

EasyTechnical

67 practiced

Explain caching strategies a Solutions Architect can propose to reduce cost per request: CDN/edge caching, in-memory caches, write-through vs write-back, cache warming, cache invalidation TTLs, and use of stale-while-revalidate. For an API with 1M reads/day and 10% dynamic content, outline which approaches you'd combine and why.

Sample Answer

Start with the goals: reduce origin cost per request, lower latency, and keep acceptable freshness for dynamic items. For an API with 1M reads/day and 10% dynamic content (i.e., ~900k fully cacheable reads), I’d combine CDN/edge + in-memory caching with careful invalidation and a conservative write strategy.

Recommended stack and why:- CDN / edge caching (first layer): Cache public, read-heavy endpoints at the edge using TTLs and stale-while-revalidate (SWR). This offloads the majority of reads (≈900k/day) to cheap edge POPs, cutting origin egress and compute. Use SWR to return stale responses while asynchronously refreshing—good for user-perceived latency and cost when slight staleness is ok.- In-memory cache (Redis/Memcached) as origin cache (second layer): For cache-misses at edge, serve from a regional Redis to avoid hitting backend DB/API. This reduces origin compute and is much cheaper than repeated full backend calls.- Cache warming / prepopulate: For predictable hot keys (e.g., homepage, product lists), proactively populate CDN and Redis after deploys or during traffic spikes to avoid origin thundering and cold-start misses.- TTL strategy + invalidation: Use tiered TTLs—longer TTLs for truly static content, short TTLs for semi-dynamic. For the 10% dynamic content, use event-driven invalidation (pub/sub or cache-control with surrogate keys) so updates purge/refresh caches immediately. For other resources, rely on TTLs.- Write-through vs write-back: Use write-through for critical items where strong consistency is required (writes update cache synchronously, simpler correctness). Consider write-back only for non-critical, high-throughput workloads where occasional freshness lag is acceptable, but be cautious—complexity and data-loss risk increase.- Edge compute (optional): For personalized but cacheable transformations, perform them at edge workers to keep responses cacheable and reduce origin hits.

Example numbers & impact:- If 90% of 1M reads are served from CDN/in-memory, origin requests drop from 1M → 100k/day, dramatically reducing compute and egress costs.- SWR increases effective cache hit-rate and reduces p99 latency; event-driven invalidation keeps the 10% dynamic items correct without full cache TTL churn.

Trade-offs:- SWR and longer TTLs trade strict freshness for cost/latency; mitigate with targeted invalidation for critical flows.- Write-back can save write cost but complicates recovery and consistency—prefer write-through unless profiling shows strong need.

Summary: Use CDN + SWR + regional in-memory cache as the primary cost-saving layers, pre-warm hot keys, apply write-through for consistency-critical writes, and use event-driven invalidation for the 10% dynamic content to balance cost, latency, and correctness.

MediumTechnical

52 practiced

You must propose a Terraform-based approach (high-level, no code) to enforce S3 lifecycle policies and cost guardrails across dev, stage, and prod environments. Describe module design, policy as code integration, role-based permissions, and automated validation in CI to prevent expensive misconfigurations.

Sample Answer

Approach summary:Create a reusable Terraform module library that standardizes S3 lifecycle policies and cost guardrails, then integrate policy-as-code checks and CI validation to prevent risky configs across dev, stage, prod.

Module design:- s3-bucket-module (public API): accepts environment, retention tiers, transition rules, versioning, encryption, logging, and cost-guardrail flags.- lifecycle-policy-submodule: generates lifecycle rules (e.g., move to IA after X days, Glacier after Y, expiration after Z), parameterized per environment.- guardrail-policy-module: attaches CloudWatch billing alarms, S3 storage class quotas (via tags + AWS Budgets), and automated cost-optimization recommendations.- outputs: lifecycle JSON, applied bucket ARN, tags, and telemetry hooks (CloudWatch/Config).

Policy-as-code integration:- Store lifecycle and guardrail policies as JSON/YAML templates in a repo. Use Terraform to render and attach them. Maintain policies in a separate policy repo and version via Git tags.- Use tools like Open Policy Agent (Conftest/OPA) and Terrascan to encode rules (e.g., every prod bucket must have lifecycle with max TTL 365 days; dev can be more permissive).

Role-based permissions:- IAM roles scoped per environment: infra-deploy-role (CI) with least privilege to create buckets and attach policies; runtime roles with read/write scoped to specific buckets.- Use IAM condition keys for tag-based access control (e.g., allow actions only if tag env=${env}).- Enforce separation: only central cost-ops role can change guardrail thresholds in prod.

Automated CI validation:- Pre-merge: run static checks (terraform fmt, validate), run Terrascan/Checkov + Conftest policies against module usage, and run policy-unit tests (positive/negative fixtures).- Plan-time: require terraform plan output to be scanned; block merge if plan shows creation of buckets without required lifecycle or encryption.- Post-apply guard: automated smoke test that queries AWS Config and CloudWatch to verify lifecycle rules exist and budgets/alarms are configured; rollback or open incident if mismatch.- Gate deployments by environment: dev pipelines allow overrides with approvals; prod requires PR, automated checks, and an approver.

Why this works:- Centralized modules enforce consistency; policy-as-code provides auditable, versioned rules; CI + plan-time scanning prevents misconfigurations early; IAM scoping reduces blast radius. This balances flexibility for dev/stage and strict cost controls for prod.

Unlock Full Question Bank

Get access to hundreds of Cost Optimization at Scale interview questions and detailed answers.

Join thousands of developers preparing for their dream job.