Performance Engineering and Cost Optimization Questions

Engineering practices and trade offs for meeting performance objectives while controlling operational cost. Topics include setting latency and throughput targets and latency budgets; benchmarking profiling and tuning across application database and infrastructure layers; memory compute serialization and batching optimizations; asynchronous processing and workload shaping; capacity estimation and right sizing for compute and storage to reduce cost; understanding cost drivers in cloud environments including network egress and storage tiering; trade offs between real time and batch processing; and monitoring to detect and prevent performance regressions. Candidates should describe measurement driven approaches to optimization and be able to justify trade offs between cost complexity and user experience.

HardTechnical

59 practiced

You are asked to build a cost allocation model for a multi-region product that uses different storage tiers and has substantial egress. Describe a practical approach to attribute costs to product lines and features, what tagging and data you need from engineering, and how you would present results so product leads can make optimization decisions.

Sample Answer

Approach (framework)Start by defining objectives: accurate showback for product lines and features, actionable insights to reduce cost or improve pricing, and ability to run “what-if” optimizations. Use a hybrid attribution model: primary cost assignment by resource-level usage (granular) + allocation rules for shared infra.

Steps1. Inventory & mapping: catalog all cloud resources (buckets, VMs, CDN, inter-region links), storage tiers, egress flows, and which product/feature consumes them.2. Required engineering data and tagging: - Resource tags: product_id, feature_id, environment (prod/stage), team_owner, customer_segment, region, cost_center. - Application-level metadata: request_id -> feature_id mapping (or feature header), bucket/object lifecycle class, object size, access frequency class. - Network metadata: source_product, destination_region, bytes_transferred, protocol (to split inter-region egress). - Usage logs: storage object creation/deletion with size, read/write metrics, CDN logs for egress per object/path. - Billing export: raw line items, per-region pricing, reserved vs on-demand discounts.3. Data pipeline & model: - Ingest billing export + tagged inventory + usage logs into a warehouse. - Normalize prices by region and tier; apply discounts/prorations. - Attribute costs: a) Direct: storage tier costs, per-object egress, per-API request charged directly to product/feature via tags. b) Indirect/shared: compute/HA/db split by steady-state metrics (CPU/conn/time) or by proportional usage (e.g., number of requests). Document allocation rules. - Special handling for multi-region egress: attribute egress originating from product X in region A to product X; where cross-product traffic exists, use request-level headers or flow sampling to split.4. Validation: reconcile aggregated model with cloud bill monthly; sample traces end-to-end.

Presentation to product leads- Executive dashboard: cost by product/feature, trend (30/90/365 days), cost per MAU/transaction, top cost drivers (tier, egress, hot objects), and sensitivity sliders (e.g., move X% objects to IA tier => savings).- Drilldowns: heatmap by region × feature, top objects by cost (name, size, access), and traffic pathways causing egress.- Actionable recommendations: list of prioritized optimizations (e.g., enable CDN for static assets in EU to reduce cross-region egress — est. $Y/mo; lifecycle policy for logs to move 60% to cold tier — est. $Z/mo).- What-if simulator: toggles for lifecycle rules, compression, caching, regional routing, showing projected savings and impact on latency/UX.

Governance & next steps- Enforce mandatory tags via CI/CD/resource templates; automated tag compliance checks.- Monthly review with product leads, engineering, finance; track realized savings vs model.- Iterate model granularity: start with product-level showback, add feature-level attribution as tagging/trace fidelity improves.

Why this worksCombines accurate usage-level accounting where possible with principled allocations for shared costs, yields both trustworthy numbers and clear levers product leads can act on (tiering, caching, routing, lifecycle), and builds a repeatable governance loop to sustain optimizations.

MediumTechnical

50 practiced

You are evaluating a proposal to place a CDN in front of several API endpoints. List which types of endpoints are good CDN candidates, describe cache key and invalidation strategies, estimate cost implications and cache hit rate assumptions you would require, and propose deployment and rollback criteria.

Sample Answer

Which endpoints are good CDN candidates- Static or semi-static read-heavy APIs: public content (images, docs), product catalogs, pricing pages, feature flags that change infrequently.- Idempotent GET APIs with low personalization and stable query semantics (e.g., search results for common queries).- Edge-computable responses (geo-based content, A/B test variants served via edge logic).Not good: highly personalized endpoints (user dashboards), POST/PUT endpoints, endpoints behind strict per-request auth/consistency requirements.

Cache key strategy- Base key = HTTP method + hostname + path.- Include only safe query params (whitelist important ones; ignore session-like params).- Normalize order and lowercase where relevant.- Respect authentication: for public endpoints, cache anonymously; for user-specific, use token-hash only if responses are identical across users.- Consider device/geo variations via headers (Accept-Language, User-Agent bucketed).- Versioning: include API version or release tag in key to allow easy invalidation.

Invalidation strategies- TTL tiers: long TTLs (1h–24h) for stable data, short TTLs (30s–5m) for semi-fresh data.- Cache-control headers from origin (max-age, s-maxage) + CDN overrides.- Event-driven purge API for content updates (soft purge: mark stale and serve stale while revalidating).- Cache-busting via resource versioning (URL or header) for breaking changes.- Stale-while-revalidate to reduce origin spikes on misses.

Cost implications & hit-rate assumptions- Estimate: CDN egress $0.02–$0.12/GB (varies by provider/region); origin bandwidth + compute saved = primary benefit.- Required assumptions for rollout: target cache hit ratio ≥ 60% for read-heavy endpoints; for very static content expect 80–95%.- Example estimate: if endpoint traffic 10 TB/month, 70% hit = 7 TB from CDN (paid egress) vs 3 TB origin egress saved. Savings = origin egress cost avoided + reduced origin compute; subtract CDN egress cost.- Also account for extra requests (cache fill, purge API calls) and potential increased edge compute costs (if using edge functions).

Deployment & rollback criteria- Pre-deploy: automated integration tests, synthetic traffic simulating cache keys, and security review (auth, CORS).- Canary rollout: route 1–5% of traffic through CDN for each endpoint, monitor metrics for 24–72 hours.- Success metrics: cache hit ratio (target per endpoint), 95th/99th percentile latency reduction, origin request reduction, error rate (4xx/5xx) stable or reduced, no increase in time-to-first-byte for authenticated flows.- Rollback triggers: cache hit < target for 48h and no improvement; increase in user-visible errors or origin 5xx > baseline + X% (e.g., 50%); latency regressions > 20% p95; business KPIs degrade (conversion, revenue) beyond defined threshold.- Post-deploy: iterate TTLs and key rules, automate purge hooks from CMS/services, and schedule regular reviews of hit rates and cost vs. savings.

This approach balances performance gains, correctness, and cost control while providing clear operational criteria for safe rollout and rollback.

HardTechnical

59 practiced

A noisy neighbor in a multi-tenant platform is consuming disproportionate CPU causing other customers to see increased latency. As PM, propose short-term mitigations, medium-term changes to isolation or pricing, and long-term platform investments. Include how you would communicate changes to affected customers and internal teams.

Sample Answer

Situation: A tenant’s workload is consuming disproportionate CPU on our shared multi-tenant platform, causing tail latency for other customers. As PM I’d deliver a three-horizon plan (short, medium, long) plus a communications and internal alignment plan.

Short-term (hours–days) — mitigate impact quickly- Implement emergency CPU throttling and soft-capping for the noisy tenant (use existing orchestration features: cgroups/CFS quotas, Kubernetes CPU limits/requests, or hypervisor-level throttles). - Move noisy workload to a limited “burst” pool or isolated node if available.- Enable prioritized scheduling for latency-sensitive tenants (QoS class promotion).- Monitor and rollback plan: create dashboards (p95/p99 latency, CPU steal, tenant CPU %) and an incident runbook.Trade-offs: temporary performance hit for noisy tenant; quick to deploy.

Medium-term (weeks–months) — stronger isolation + pricing controls- Formalize resource guarantees: require CPU/memory requests and introduce soft reservations; enforce limit ranges and admission control.- Introduce tiered QoS: Basic (best-effort), Standard (baseline guarantees), Premium (SLA-backed reservations). Tie to pricing and throttling behavior.- Add automated anomaly detection that triggers autoscaling, throttling, or migration when a tenant exceeds baseline for X minutes.- Policy examples: burst credits, sustained-high usage surcharge, or mandatory dedicated nodes for sustained >Y% usage.Trade-offs: UX and billing complexity; need clear documentation and tooling.

Long-term (quarters) — platform investments- Build true strong isolation options: lightweight VMs, dedicated CPU pools, or hardware partitions (NUMA-aware placement).- Invest in resource-aware scheduler, predictive autoscaling, and per-tenant telemetry pipeline for forecasting noisy neighbors.- Offer “dedicated compute” SKUs and SLAs for customers needing predictability.- Continuous optimization: noise-resistant caching, request prioritization, and circuit-breakers at service boundaries.

Communication to customers- Immediate: targeted emails/console alerts to affected tenants explaining incident, temporary mitigations, expected impact, and actions they can take (opt into dedicated nodes, adjust workloads).- Follow-up: incident postmortem with timeline, root cause, and roadmap of fixes. Publish status page updates and FAQs.- For pricing/policy changes: 60–90 day notice, migration guides, credits for affected customers, and customer webinars + account manager outreach for high-value tenants.

Internal coordination- Form a cross-functional task force (SRE, infra eng, product, sales, legal, support) with daily standups during incident.- Define KPIs: p95/p99 latency, CPU isolation metric (tenant CPU share variance), MTTR, and revenue impact.- Prioritize roadmap items via RICE: isolation features and SLA products high priority; UI/billing automation mid.- Ensure legal/compliance review for pricing/policy changes and prepare sales enablement materials.

Why this approach- Short-term actions protect customers quickly with low engineering effort.- Medium-term aligns incentives (pricing + automation) to prevent recurrence.- Long-term investments provide reliable isolation for scale and new revenue through dedicated SKUs.- Clear customer communication + internal ownership preserves trust and reduces churn.

HardSystem Design

60 practiced

Design an SLO and error budget policy for a multi-tenant SaaS backend where premium customers pay for stricter guarantees. Explain how you would tier SLOs, how to measure and enforce them technically and contractually, and how error budgets affect release cadence and incident escalation for each tier.

Sample Answer

Requirements / constraints:- Multi-tenant SaaS with Premium (paid strict SLAs), Standard, and Free tiers.- Business: protect Premium revenue and reputation, while allowing velocity for lower tiers.- Non-functional: measure per-tenant SLOs, isolate noisy tenants, and automate enforcement.

High-level SLO tiers (examples):- Premium: 99.95% availability per calendar month, p99 latency < 200ms for API calls, 4 hr incident P1 response SLA.- Standard: 99.9% availability, p95 latency < 500ms, 8 hr P1 response.- Free: 99.0% availability, best-effort latency, 24 hr P1 response.

Measurable SLI definitions & aggregation:- Availability = successful 2xx responses / total valid requests (exclude client errors like 4xx where appropriate) measured per-tenant per region.- Latency = request duration percentiles (p50/p95/p99) per tenant.- Error rate = HTTP 5xx or application-level failures.- Use sliding window (monthly) and rolling burn-rate for short-term decisions.

Technical measurement & enforcement:- Instrumentation: distributed tracing, metrics (Prometheus), per-tenant tags on all telemetry, tenant-aware rate-limits.- Alerting: per-tenant SLO burn-rate alerts (e.g., burn rate > 2 over 1h triggers warnings; >4 triggers mitigation).- Automated protections: dynamic throttling/traffic shaping, feature flags to degrade non-critical features for lower tiers, circuit breakers, and tenant quiescing for noisy tenants.- Isolation: resource quotas (CPU/memory/DB IOPS) per tier and per-tenant to limit blast radius.

Contractual enforcement:- SLA document tied to SLOs: define measurement method, reporting cadence, exclusions (maintenance, force majeure), and credits (service credit %) for breaches.- Premium contract includes uptime credits, dedicated support channel, and penalty caps.- Standard: limited credits and moderate support; Free: no credits, best-effort support.- Transparency: monthly SLO reports portal per tenant and post-incident RCA for Premium customers.

Error budget policy & operational impact:- Error budget = 1 - SLO (e.g., Premium monthly error budget = 0.05% downtime).- Burn-rate windows: - Green: burn-rate < 1 — normal releases. - Yellow: burn-rate 1–3 — restrict risky changes (no database migrations, no large feature toggles); require extra QA and on-call review. - Red: burn-rate > 3 or remaining budget < threshold — pause non-critical releases for affected tier(s), run emergency triage, escalate to senior engineers and product leads.- Tiered cadence: - Premium: stricter gating — daily monitoring + automated rollback on burn spikes; release windows require pre-approved runbooks. - Standard: CI gating plus feature flags to quickly rollback for affected tenants. - Free: higher tolerance; can be included in canary experiments more aggressively.

Incident escalation matrix:- Define P0/P1/P2 with tier-specific response times and stakeholders (SRE, Eng Manager, Product PM, Account Exec for Premium).- Premium incidents auto-notify account owner and require incident chat + priority RCA within 48 hours.- Use postmortems and adjust SLOs/quotas if repeated hits occur.

Governance & product trade-offs:- Balance velocity vs. reliability: use feature flags and tenant targeting to ship fast while protecting Premium.- Periodically review SLOs with Sales and Legal to ensure pricing aligns with operational cost.- Monitor customer satisfaction and churn relative to SLO adherence; iterate on tiers or add an “Enterprise” tier with dedicated infra if needed.

Example enforcement flow (Premium):- Detect burn-rate 4x in 30m => automated alert -> auto-rollback of last deployment for Premium traffic, throttle nonessential endpoints, page SRE and PM, notify customer, start RCA. If SLO breached at month-end, issue credit per SLA.

This approach makes SLOs measurable, enforceable technically, and contractually clear, while using error budgets to objectively manage releases and escalation per customer tier.

HardTechnical

61 practiced

Third-party payment gateway performance affects checkout latency. Design an approach to set performance expectations and error budgets for suppliers, including contract terms, technical SLAs, monitoring, and how you would escalate or mitigate when a vendor misses targets.

Unlock Full Question Bank

Get access to hundreds of Performance Engineering and Cost Optimization interview questions and detailed answers.

Join thousands of developers preparing for their dream job.