Cost Aware Architecture and Design Questions

Focuses on how architectural decisions and design patterns affect operating cost and total cost of ownership. Interviewees should be able to reason about trade offs such as managed services versus self managed components, always on virtual machines versus event driven or serverless approaches, reserved versus on demand capacity, use of spot or preemptible instances, and multi region or edge placement. Candidates should demonstrate techniques for reducing cost through storage class selection and lifecycle policies, caching and batching, query and workload optimization, data transfer minimization, and workload isolation. The topic also covers modeling and communicating cost trade offs, estimating ongoing operating expense for alternative designs, and choosing architecture that balances budget constraints with reliability, performance, and engineering effort.

MediumSystem Design

30 practiced

Design an edge and CDN strategy for a budget-conscious video streaming app serving global users. Decide which assets to cache at edge, acceptable TTLs, origin placement, use of tiered caching (regional PoPs), and when to use edge compute for personalization vs doing it at origin. Explain how you will measure and optimize cache hit-rate vs cost.

Sample Answer

Requirements & constraints:- Global users, budget-conscious, video streaming (on-demand + short live), prioritize low playback startup time and low egress cost.- Trade-off: cache hit-rate vs CDN egress & origin compute cost.

High-level design:- Origin(s): Central object store (S3-compatible) in one or two primary regions (e.g., US-East + EU) for origin of truth; use multi-region replication for regulatory needs. Keep origins in cloud regions with low egress to chosen CDN.- CDN with tiered caching: global edge PoPs + regional mid-tier (regional PoPs / POP aggregation) to reduce origin fetchs and lower cost. Enable origin shield (single shielding POP) per region.

Which assets to cache and TTLs:- Video chunks (HLS/DASH segments, e.g., .ts/.m4s): cache at edge with long TTLs (24h–7d) depending on churn. For VOD stable content: 7d TTL + cache-control immutable when versioned. For new releases: shorter warm-up TTL (1h then promote).- Manifests (playlist/MPD): short TTL (5–30s) if live or dynamic; for VOD manifests with versioning use long TTL (1h–24h) and validation via ETag.- Static UI assets (JS/CSS/images): long TTL (7d–30d) with cache-busting fingerprints.- Ads / DRM license calls / auth endpoints: do not cache (or very short TTLs < 5s) or cache at regional mid-tier where allowed.- Thumbnails / poster images: medium TTL (1–24h).

Edge compute vs origin:- Use edge compute for lightweight personalization that avoids fetching from origin and reduces latency (e.g., inserting user-specific subtitle selection, device-specific transcoding hints, A/B test header variations, edge auth token validation, signed URL generation when keys are at edge). Keep sensitive logic and heavy processing (transcoding, DRM license issuance, billing) at origin/central services. Also perform personalization by serverless edge only when the operation is stateless and uses cached data or cookies/jwt; otherwise call origin.

Tiered caching behavior:- Configure edge TTLs but allow mid-tier to hold longer TTLs to reduce origin egress. Use origin shield for cache fills. Enable “stale-while-revalidate” to serve stale content during origin fetch.

Measuring & optimizing cache hit-rate vs cost:- Metrics to collect: edge hit ratio, regional mid-tier hit ratio, origin fetch rate, egress cost per region, cache fill rate, median startup time, rebuffer rate, and request latencies.- Dashboard & alerts: show hit-rate vs origin egress dollars by content class (hot, warm, cold).- Optimization steps: - Identify hot content with low cacheability → pin in edge long TTLs and ensure versioned URLs to avoid invalidation cost. - Use popularity-based prefetching / proactive caching for new releases into target regions for launch windows. - Adjust TTLs per content class: increase TTL for content with >X plays/day to reduce origin fetches. - Use analytics to detect cache pollution (many single-use objects) and route those to regional caching only or origin-direct to avoid occupying edge caches. - Apply cache-key normalization (strip irrelevant query params, canonicalize headers) to increase hit ratio. - Implement conditional requests (If-None-Match) to save bandwidth when validating. - Use metrics-driven simulated cost model: run A/B TTL changes in canary regions to measure hit-rate, latency, and cost impact before global rollout.

Edge cases & trade-offs:- Cache invalidation: prefer URL versioning over purging; restrict purges to emergencies (high cost).- Live streams: rely on short TTLs and chunk-based caching; mid-tier helps absorb spikes.- Security: keep DRM keys & sensitive user profile operations at origin or protected edge functions; log and monitor.

Result: this tiered, class-based caching strategy with selective edge compute yields high cache hit-rates for heavy video traffic, reduces origin egress and latency for users, and provides levers (TTL, prefetching, key normalization) to balance hit-rate vs cost using measured experiments.

MediumTechnical

31 practiced

A customer requires 1000 TPS with 5ms P99 read latency and GDPR residency. Compare deploying a managed relational database service (e.g., RDS/Cloud SQL) versus self-managing a clustered DB on VMs. Discuss the cost drivers, operational staffing, compliance considerations, performance tuning, backup/restore costs, and recommend the best approach with justification.

Sample Answer

Situation: Customer needs 1,000 TPS, P99 read ≤5 ms, and GDPR data residency.

Recommendation (summary): Use a managed relational DB service if the provider can guarantee regional (EU) residency and SLA-compliant instance types; choose self-managed only if the cloud provider’s managed service cannot meet the 5 ms P99 or residency/SLA/compliance requirements and you have strong ops capabilities.

Comparison:

- Cost drivers - Managed: instance sizes, IOPS/provisioned throughput, storage, backup retention, cross-AZ/region replicas, licensing. Predictable OPEX; pay-for-managed features. - Self-managed: VM sizes, block storage IOPS, network egress, HA cluster licensing, orchestration (e.g., Pacemaker/Kubernetes), higher CAPEX-like initial setup and variable OPEX for patching and automation.

- Operational staffing - Managed: smaller DBA/devops footprint — focus on schema, query tuning, capacity planning. - Self-managed: dedicated DBAs, SREs for HA, upgrades, failover testing, security patching, monitoring — significantly higher headcount and on-call.

- Compliance (GDPR residency) - Managed: verify provider’s EU-region data residency, contractual DPA, subprocessors, encryption, and audit logs. Some managed services offer region-lock and customer-managed keys (CMK). - Self-managed: full control over residency and keys; easier for stringent audit demands but requires rigorous process documentation and evidence.

- Performance tuning - Managed: choose high-IO instances, read replicas, caching (managed Redis), schema optimization, prepared statements, connection pooling, and query profiling. Limited OS/sysctl tuning. - Self-managed: deeper stack tuning (kernel params, storage driver, NUMA, CPU pinning), custom caching, and bespoke replication topologies — can squeeze lower latencies but higher risk and complexity.

- Backup/restore costs - Managed: built-in snapshots (storage and egress costs), fast restores often supported; cost scales with retention. - Self-managed: implement backup tooling, storage, transfer costs, runbook-tested restores; recovery time objectives may be longer and operationally costly.

Trade-offs and justification:- If managed service in the EU with right instance classes + provisioned IOPS and read replicas can meet 1,000 TPS and P99 5 ms with proper caching and schema design, pick managed for lower TCO, faster time-to-market, and simpler compliance posture (with DPA/CMK).- If benchmarking shows managed cannot meet 5 ms P99 even after tuning, or contractual compliance requires zero-subprocessor or on-prem control, choose self-managed and budget for higher staffing, automation, and rigorous testing.

Next steps: run load tests on candidate managed instance types with realistic workload and caching; validate residency/DPA details; produce TCO over 3 years including staff costs and disaster-recovery runbooks.

EasyTechnical

39 practiced

Explain the difference between instance reservations and compute savings plans (or equivalent) across cloud providers. Discuss flexibility differences (region/instance family constraints), common billing terms, and recommendations for which workload types to commit (steady baseline vs variable peak).

Sample Answer

Instance reservations (Reserved Instances, RIs) and Compute Savings Plans (or equivalent) are both commitment-based discounts that lower compute costs in exchange for a term commitment, but they differ in flexibility and scope.

Definitions:- Reserved Instances: You commit to a specific instance family, region (or AZ) and often tenancy/OS for 1–3 years. Discount applies to matching instances; some providers offer convertible RIs that allow limited changes.- Compute Savings Plans: You commit to a dollar-per-hour spend for compute across services for 1–3 years. The commitment applies across instance sizes, families, regions and sometimes between VMs and serverless compute, giving much more flexibility.

Flexibility differences:- RIs: More rigid — tied to instance family, region/AZ, and platform unless convertible; changing architecture may leave unused reservations.- Savings Plans: Broadly flexible — automatically apply to any eligible compute usage, so shifting instance sizes, families, or regions still benefits you.

Common billing terms:- Term length: 1 or 3 years- Payment options: All Upfront, Partial Upfront, No Upfront (affects effective discount)- Commitment scope: instance-specific vs $/hour spend- Convertible vs Standard (for RIs): convertible allows limited exchanges; standard gives higher discount

Recommendations by workload:- Steady baseline (always-on, predictable): Commit — either RIs (if architecture stable and you want max discount) or Savings Plans (if you expect some change). RIs can give slightly higher discounts for highly predictable, unchanging fleets.- Variable or evolving workloads (auto-scaling, frequent instance-type changes, multi-region migrations): Use Compute Savings Plans for their flexibility; avoid over-committing RIs.- Short-term/project or experimental: Prefer on-demand or short-term SAs; consider partial upfront or convertible options if you might extend.

Practical tip: analyze 6–12 months of usage, model savings under both options, and prefer a mixed strategy — cover the steady baseline with commitments and keep burst/ephemeral capacity on-demand.

MediumTechnical

33 practiced

A data pipeline ingests 5 TB/day and needs hourly aggregations. Compare always-on streaming (near real-time) vs micro-batch (hourly) processing in terms of cost, latency, and operational complexity. Recommend optimizations (file format, partitioning, compaction, compression) that reduce compute and query costs while keeping acceptable freshness.

Sample Answer

Situation: We must process 5 TB/day and produce hourly aggregates. Two patterns: always‑on streaming (near real‑time) vs hourly micro‑batches. Below I compare them and give pragmatic optimizations and a recommendation.

Comparison- Latency - Streaming: sub-second to seconds/minutes (depending on windowing and checkpointing). Best for SLA requiring <5–10 minute freshness. - Micro‑batch (hourly): deterministic ~1 hour freshness; simpler if hourly is acceptable.- Cost - Streaming: higher steady-state cost — always-on compute for ingestion, stateful windows, and more expensive orchestration/ops. - Micro‑batch: lower cost overall — compute runs only per hour (short-lived clusters, spot instances), less state to maintain.- Operational complexity - Streaming: more complex (state management, fault tolerance, exactly-once, backpressure tuning). Requires mature monitoring. - Micro‑batch: simpler to reason about, easier retries/backfills, fewer long‑running services.

Recommendation- If business SLA is hourly (acceptable), choose micro‑batch hourly processing for cost efficiency and lower ops burden.- If sub‑hour freshness required, adopt a hybrid: short-window streaming for recent minutes (materialized incremental view) + hourly micro‑batch compaction/rollup for analytics.

Optimizations to reduce compute/query costs while keeping freshness- File format: Parquet or ORC (columnar) for efficient projection and predicate pushdown.- Compression: Snappy or Zstd (balance CPU vs compression ratio); Zstd for long‑term storage if supported.- Target file size: 256 MB–1 GB per file to avoid small‑file overhead and maximize scan throughput.- Partitioning: date/hour partitioning (dt=YYYY-MM-DD/hr=HH) for hourly pruning. Avoid high‑cardinality columns as top-level partitions.- Bucketing / clustering: for high‑cardinality join keys use hash bucketing or clustering (Z-order in Iceberg) to reduce shuffle and accelerate point lookups.- Compaction: schedule compaction after each hourly job (or daily) to merge small files using Spark/Presto with Delta Lake / Apache Hudi / Iceberg; enables faster reads and lower query cost.- Table format: Use a transactional table format (Delta Lake, Iceberg, Hudi) to support upserts, time travel, and efficient file-level compaction.- Materialized hourly aggregates: store pre-computed hourly rollups (date/hr partitions) to avoid repeated full-table scans.- Incremental processing: use watermarking and incremental queries (streaming or CDC) to process only deltas.- Autoscaling & spot instances: for batch runs use ephemeral clusters and spot/preemptible instances; for streaming use autoscaling and reserved capacity where needed.- Predicate pushdown & vectorized reads: ensure query engine supports vectorized readers and stats-based pruning.

Example architecture (hybrid, recommended):- Ingest raw events into Kafka (or Kinesis).- Sink to object storage (S3) in small hourly files or write directly to Delta/Hudi/Iceberg via streaming sink.- Hourly micro‑batch Spark/Flink job: materialize hourly aggregate into partitioned Parquet/Delta table + run compaction job.- Optional short‑window streaming aggregation for sub‑hour SLA stored in a hot table (kept for N hours) for low‑latency queries.

Trade-offs: streaming reduces latency but increases cost/ops. Micro‑batch wins cost and simplicity for strict hourly requirements. Hybrid gives best of both with moderate complexity.

This balances cost, acceptable freshness, and query performance for a 5 TB/day pipeline.

MediumTechnical

39 practiced

A BI team runs expensive, recurring queries on a cloud warehouse (scan-heavy). Recommend concrete query and data design optimizations (partitioning, clustering, materialized views, denormalization, columnar formats, query rewrite, and caching) to reduce scanned bytes and cost while keeping freshness for dashboards.

Sample Answer

Start by clarifying SLAs: acceptable freshness (seconds/minutes/hours), peak query patterns, and which dashboards are mission-critical. Then apply layered optimizations:

1. Partitioning- Time-partition tables by ingestion/event date to prune scans. Use ingestion-time partitioning if events arrive late.- BigQuery example: CREATE TABLE … PARTITION BY DATE(event_time)

2. Clustering / Sort Keys- Cluster on high-cardinality filter/join columns (user_id, country) to reduce scanned blocks for selective predicates. Re-cluster regularly if data changes.

3. Columnar & File Format- Ensure columnar storage (native in BigQuery/Snowflake) and compact files: avoid many tiny files; use larger files (~100-500MB) for efficient column pruning.

4. Materialized Views & Incremental Refresh- Create materialized views for expensive aggregations used by dashboards; set refresh frequency to match freshness SLA. For BigQuery, use MV or scheduled incremental tables populated by INSERT…SELECT with partitioned writes.- Example MV: aggregate by date and product; refresh hourly.

5. Denormalization & Pre-agg Tables- For read-heavy dashboards, maintain denormalized or pre-aggregated tables keyed by dashboard filters to avoid large joins and scans. Use change-data-capture (CDC) / streaming jobs to keep near-real-time.

6. Query rewrite & Predicate Pushdown- Rewrite queries to push filters to the base tables, select only needed columns, avoid SELECT *. Use WHERE clauses on partition/clustering columns.

7. Caching- Use warehouse/result caching (Snowflake result cache, BigQuery cached results) for identical repeated queries; set dashboard refresh intervals to allow cache hits. Employ a cache hit monitoring policy.

8. Cost controls & Automation- Implement query cost alerts, enforce maximum bytes billed, and use query tags to attribute cost. Automate re-clustering/compaction jobs and MV refreshes via orchestration (Airflow).

Trade-offs: more MVs/denorm increases storage and ETL complexity but reduces compute. Clustering/partitioning improve scan efficiency but need maintenance. Recommend pilot with top 10-cost queries: add partitioning/clustering, create one MV and one denormed table, measure scanned bytes and latency, iterate.

Unlock Full Question Bank

Get access to hundreds of Cost Aware Architecture and Design interview questions and detailed answers.

Join thousands of developers preparing for their dream job.