Strategic Solution Design Questions
Covers the end to end process of diagnosing a business or customer problem and developing one or more viable solutions. Expect to generate multiple approaches with different risk and reward profiles, assess feasibility, implementation timeline, resource requirements, costs, trade offs, and alignment with business objectives. Include planning for execution details such as milestones, dependencies, and risk mitigation. Emphasize clear stakeholder communication and recommendation framing, customer focus, anticipation of implementation challenges, and creation of a coherent proposal that justifies the chosen approach and explains why other options were rejected.
MediumTechnical
54 practiced
Draft a risk register template for a 9-month implementation of a customer portal. Include risk categories (technical, operational, legal, market), likelihood-impact scoring, mitigation actions, owners, trigger conditions, and escalation path. Describe how you would keep this register current during delivery.
Sample Answer
Risk Register — Customer Portal Implementation (9 months)Columns / Template fields:- ID- Risk Title- Category (Technical / Operational / Legal / Market)- Description & Impact (concise business/technical impact)- Likelihood (1–5) × Impact (1–5) = Risk Score (1–25)- Priority (Low/Med/High based on score bands)- Mitigation / Preventive Actions (what we do to reduce likelihood)- Contingency / Recovery Actions (what we do if it occurs)- Owner (role — e.g., Solutions Architect, Dev Lead, PM, Legal)- Trigger Conditions (observable signs that risk is materializing)- Detection Method / Frequency (monitoring metric or checkpoint)- Status (Open/Monitoring/Mitigated/Closed)- Escalation Path (who to notify at score thresholds, e.g., PM → Engineering Director → CTO; time-to-escalate)- Target Review Date / Next ReviewScoring guidance:- Likelihood 1=Rare … 5=Almost certain- Impact 1=Negligible … 5=Catastrophic- Priorities: 1–6 Low, 7–14 Medium, 15–25 HighExample entry:- ID: T-01- Title: Integration failure with client SSO- Category: Technical- Description: SAML attributes mismatch causing login failures; blocks UAT.- Likelihood 3 × Impact 4 = 12 (Medium)- Mitigation: Early SSO PoC with sample metadata, dedicate integration sprint, schema validation tests- Contingency: Fallback to temporary OAuth client, hotfix plan- Owner: Solutions Architect (integration) / Dev Lead (implementation)- Trigger: Failed SSO smoke tests in sprint demo; >10% auth errors- Detection: Daily CI e2e auth tests; weekly integration demo- Status: Open- Escalation: PM if score persists >12 for 2 weeks → Head of Eng within 48 hoursKeeping the register current during delivery:- Make the risk register a living artifact in the project workspace (Confluence/Jira Risks board).- Assign each owner responsibility for weekly updates and a standing 15-minute risk review in the weekly status meeting.- Re-score risks at each major milestone (end of design, end of sprint 3, UAT start, pre-launch).- Automate detection where possible (CI test failures, monitoring alerts) to feed triggers.- Require owners to update status upon trigger, add root cause analysis and changes to mitigation, and follow escalation timelines.- Monthly executive summary of top 5 risks with trends and decisions for stakeholders.- After any incident, run a blameless postmortem and update register with new mitigations and lessons learned.
MediumSystem Design
34 practiced
Design a feature-flag system to support phased rollouts and canary releases across five regions. Describe flag hierarchy (global, region, tenant), ownership/governance, telemetry gating, kill-switch behavior, audit trails, and how to ensure compliance when flags modify critical flows.
Sample Answer
Requirements:- Phased rollouts and canaries across 5 regions- Multi-tenant support with per-tenant overrides- Safe kill-switch, telemetry gating, audit & compliance for critical flows- Low-latency evaluation in productionHigh-level design:- Central Flag Service (control plane) + distributed SDKs (data plane) in each region.- Flags stored in hierarchical datastore: Global → Region → Tenant. Resolution: most-specific wins (tenant > region > global). Support percentage rollouts via consistent hashing + bucketing key (userID, session, or request fingerprint).Flag Hierarchy & semantics:- Global flag: default behavior for all regions/tenants.- Region flag: overrides global for specific region (region granularity: us-east, eu-west, etc.).- Tenant flag: customer-specific overrides, can be sticky (session or user-scoped).- Flag types: boolean, percentage, rule-based (attribute match), and variant payloads.Ownership & Governance:- Owners: Product owner owns flag intent; Engineering owner owns implementation; Security/Compliance owns gating for sensitive flags.- RBAC in Control Plane: roles (Admin, Product, Engineer, Auditor). Changes require approvals for flags tagged critical.- Approval workflow: create → draft → test/canary → approval (1+ approvers) → rollout.Telemetry gating & canaries:- Telemetry pipeline collects metrics (latency, error rate, business KPIs) per flag, region, tenant, and cohort.- Gate rules: automated rollback/stop conditions expressed as SLO breach or delta thresholds (e.g., +2% error or 5% latency increase).- Canary flow: create canary cohort (small percentage or specific hosts), monitor for evaluation period (e.g., 30m), gate evaluates metrics and either promotes or rolls back automatically.Kill-switch behavior:- Fast path: SDKs maintain local cached flag state with short refresh + push updates via regional streaming (e.g., SSE or gRPC). For kill-switch trigger, Control Plane emits immediate push; SDKs enforce immediately.- Network partition safe: SDK defaults to conservative behavior for critical flags (fail-closed or fail-open per policy). Critical flags default require explicit safe default and circuit-breaker.- Emergency global kill: Admin role can trigger global override; Control Plane pushes high-priority invalidate message to all regions.Audit trails & observability:- Immutable audit log for every change: who, when, before/after, justification, approval records, and rollout plan. Store logs in append-only store (e.g., write-once S3 + checksum).- Event tracing: flag-evaluation events include trace IDs; correlate with distributed traces and logs.- Reporting: per-tenant/regional rollout history, access exports for compliance.Ensuring compliance when flags touch critical flows:- Tagging & policy: flags that modify critical flows must be labeled “critical” and require stricter workflows (mandatory code review, staged environments, automated tests, formal approval).- Testing: generate synthetic traffic exercise critical paths in staging; require canary in non-production and production with defined success criteria.- Access control: only specific roles can edit critical flags; changes require two-person approval and are time-limited (expiration).- Monitoring & SLA integration: tie flag gate outputs to incident management (PagerDuty) and require post-mortem for any gated rollback.- Evidence generation: automated snapshots (config, audit, telemetry) attached to change record to satisfy compliance audits.Trade-offs & scaling:- Use local caching + push for low latency; eventual consistency acceptable for non-critical flags.- Central control plane horizontally scalable (statestore + coordination via leader election); regional proxies reduce cross-region latency.- Complexity: stricter governance increases cycle time—use risk-based policies to balance agility vs safety.This design provides hierarchical control, fast reaction (kill-switch), telemetry-driven gating for automated canaries, strict governance for critical flags, and immutable audits to meet compliance needs.
MediumSystem Design
65 practiced
A client requires 99.99% availability with RTO < 15 minutes and RPO < 1 minute for an application handling 5k TPS. Propose architecture patterns, data replication strategies, automated failover orchestration, monitoring/health checks, and a DR testing cadence. Explain cost vs availability trade-offs.
Sample Answer
Requirements & constraints:- SLA: 99.99% (~52.6 min downtime/year)- RTO < 15 min, RPO < 1 min- 5k TPS steady load- Likely multi-region for resilienceHigh-level architecture patterns:- Active-active multi-AZ, multi-region deployment behind global LB (DNS + Anycast), with local regional load balancers for low latency and failover.- Stateless application tier (containers/K8s with Horizontal Pod Autoscaling) + stateful services separated (databases, caches, queues).- Circuit breakers, retry/backoff, bulkheads.Data replication strategies:- Primary-secondary DB per region with near-synchronous replication (synchronous within AZ, async cross-AZ) where possible; for RPO <1m use semi-synchronous replication between primary region and warm-standby secondary region with WAL shipping and replication lag monitoring.- Use distributed consensus DB (e.g., CockroachDB, Yugabyte) or leaderless replication (Cassandra) if multi-master required; ensure conflict resolution.- For caches, use cross-region replication with eventual consistency and tombstones for invalidation.- Use durable queuing (Kafka with replication factor 3 across racks and MirrorMaker/Clusters for cross-region replication) to buffer spikes and preserve messages.Automated failover orchestration:- Orchestrate with runbooks in automation platform (Terraform + cloud provider APIs + orchestration engine like Spinnaker or custom Lambda/Cloud Functions).- Health-driven promotion: automated promotion of standby DB when primary unhealthy and replication lag < 60s; use automated DNS failover with low TTL and health checks at both LB and app levels.- Use orchestration lock (etcd/lock service) to prevent split-brain; include manual approval gateway if replication lag > threshold.- Automated traffic shifting with phased percentage ramp and canary checks.Monitoring & health checks:- Multi-layer health checks: infra (host/VM), k8s pod liveness/readiness, app-level (business transactions), DB replication lag, queue lag, error rates, latency p99/p95.- SLOs and alerting: alert when replication lag > 30s, p99 latency > target, error budget burn.- Observability: distributed tracing (OpenTelemetry), logs centralized, metrics in Prometheus + Grafana, synthetic transactions every 10–30s.- Post-failover verification suite to run automated smoke tests.DR testing cadence:- Automated DR drills quarterly for regional failover (simulate AZ/region loss), monthly smoke tests, yearly full-scale failover with RTO verification.- Tabletop exercises monthly with stakeholders; rolling failover rehearsals in lower environments weekly.- Post-test blameless retrospectives, update runbooks.Cost vs availability trade-offs:- Higher availability (multi-region active-active, synchronous cross-region replication, higher replication factor) increases cost exponentially: compute, cross-region bandwidth, standby DB licensing, and operational complexity.- Warm-standby reduces cost vs active-active but increases recovery complexity and slightly higher RTO risk.- Recommend tiered approach: critical path components (DB, queue) get active-active / high-cost protection; less critical components use warm-standby or faster rebuild automation.- Provide TCO options: target 99.99 with warm-standby + strict automation (balanced), or 4/9s with active-active (premium).This design meets RTO <15m and RPO <1m when configured with semi-sync replication, strict lag monitoring, automated promotion, and regular DR drills; cost and complexity should be aligned to business risk tolerance.
EasyTechnical
41 practiced
You are engaged as a Solutions Architect with an enterprise experiencing a 20% customer churn following a major release. Walk me through your approach to diagnose root causes: what stakeholders you would interview, what quantitative and qualitative data you would collect, what quick experiments or telemetry you'd add, and what early solution options you'd propose to the product and executive teams.
Sample Answer
Start by clarifying scope and constraints (which customers, regions, timeframe, KPI definitions). Then run a parallel discovery: stakeholder interviews, data collection, quick telemetry, and early mitigations.Stakeholders to interview- Product manager (roadmap, feature goals, rollout plan)- Engineering lead / release manager (deploy strategy, known defects)- Customer success / support leads (ticket volume, sentiment)- Sales and account managers (churned accounts reasons, contract impact)- Ops/SRE (performance/availability metrics)- A sample of affected customers (qualitative feedback)Quantitative data to collect- Churn cohort analysis (by release date, plan, geography, channel)- Support ticket volume, severity, and time-to-first-response before/after release- Usage telemetry: DAU/MAU, feature adoption, drop-off funnels- Performance and error rates (APM traces, latency, error spikes)- Business metrics: MRR lost, churn by ARR segmentQualitative data to collect- Support transcripts, NPS/CSAT changes, recorded customer interviews, account churn notes, beta feedback.Quick telemetry & experiments (low-risk, fast)- Add feature-flagged rollout and rollback capability for suspect features- Add fine-grained event logging around critical flows and error contexts- Instrument session replays / funnel analytics for top 3 abandoned flows- A/B test small fixes or UX rollbacks to measure immediate impact- Short-run targeted customer surveys + rapid follow-up interviewsEarly solution options to propose- Immediate: roll back or disable the highest-risk feature(s) behind flags; increase support capacity and proactive outreach to at-risk accounts- Short-term (2–4 weeks): patch critical bugs, UX fixes identified via session data, SLA/perf fixes; offer retention incentives for high-value customers- Medium-term (1–3 months): rework release process (canary deployments, dark launches, automated pre-release regression suites), improve observability and customer feedback loops- Strategic: revise product acceptance criteria, embed CS metrics in release gating, and introduce cross-functional postmortems with actionable remediation ownersCommunicate a data-backed remediation plan with timelines, KPIs (reduced churn %, decreased errors, improved CSAT), and decision points (rollback vs iterate). Prioritize safety (rollback) when evidence points to regressions; prioritize targeted fixes when issues are localized.
HardSystem Design
45 practiced
Design three tenancy models for a multi-tenant SaaS product: shared-schema, separate-schema-per-tenant, and isolated-VPC per tenant. For each option, analyze security/isolation, upgrade and migration complexity, operational cost at 1k/10k/100k tenants, and recommend a migration path and backup strategy for the chosen model.
Sample Answer
Clarify requirements: tenant SLAs, compliance (PCI/HIPAA), per-tenant customization, expected tenant churn, read/write workload, RTO/RPO targets, and budget constraints. Based on that, compare three tenancy models.1) Shared-schema (single DB, tenant_id column)- Security/isolation: Logical isolation only; lowest isolation risk but acceptable with strong row-level security, parameterized queries, and strict RBAC. Harder for strict compliance.- Upgrade/migration complexity: Easiest schema rollout (single migration), feature flags for gradual rollouts. Hard to migrate a tenant out individually.- Operational cost @1k/10k/100k: Lowest per-tenant cost; marginal DB cost grows sublinearly (single cluster). Network/compute cheap.- Pros/cons: Cost-efficient, simple to operate; riskier for noisy neighbors and compliance.2) Separate-schema-per-tenant (one DB, many schemas or per-tenant DB on same instance)- Security/isolation: Better logical isolation; per-schema permissions reduce blast radius. Easier to grant/ revoke access for audits.- Upgrade/migration complexity: More complex migrations (run migrations per schema), but can roll tenants progressively. Easier to move single tenant between instances.- Operational cost @1k/10k/100k: Moderate; management overhead (connection pools, schema lifecycle). At 100k, operational tooling required (schema provisioning automation, sharding).- Pros/cons: Balance of isolation and cost; greater operational complexity.3) Isolated VPC per tenant (dedicated VPC, DB, compute)- Security/isolation: Strongest; network+compute+storage isolation meets highest compliance and performance SLAs.- Upgrade/migration complexity: High operational effort per-tenant but upgrades can be scheduled per-tenant with minimal cross-impact.- Operational cost @1k/10k/100k: Very high; linear cost per tenant. At scale (>10k) becomes cost-prohibitive unless only used for premium tenants.- Pros/cons: Best security/perf; expensive and complex.Recommendation & migration path:- Default: Use separate-schema-per-tenant as primary model for most tenants; reserve isolated-VPC for enterprise/premium tenants needing strict isolation. Use shared-schema only for ephemeral/free/very small tenants to maximize cost-efficiency.- Migration path: Start with a shared-schema MVP, instrument tenant metadata and feature flags. As customers grow, offer an automated upgrade to separate-schema (export tenant data -> create schema/DB -> run migration -> switch routing). For enterprise, provide an automated or manual migration to isolated-VPC: replicate data (logical replication or backup/restore), provision infra via IaC (Terraform), validate, cutover with DNS/VPN.- Automation: Build provisioning service, migration pipelines, canary migrations, tenant feature toggles, and a tenancy registry.Backup & DR strategy (for chosen model: separate-schema-per-tenant):- Backups: Regular full DB cluster snapshots + incremental WAL/transaction-log backups. For per-schema restore, keep logical backups (pg_dump of individual schemas) for fast tenant-level restores.- RTO/RPO: Use continuous replication to a warm standby region for low RTO for critical tenants; tiered SLAs (enterprise get warm-standby, standard get daily snapshots).- Retention & compliance: Retention policies per-tier; encrypt backups at rest and in transit, maintain KMS audit logs.- Testing: Automated periodic restore drills for both cluster-level and per-tenant restores; validate data integrity and application-level behavior post-restore.- Security: Isolate backup access, use least-privilege service accounts, and log all backup/restore operations.Trade-offs summary: separate-schema-per-tenant gives best compromise between isolation, cost, and operational complexity at scale; isolate-VPC for few high-need tenants; shared-schema only for cost-sensitive low-risk tenants.
Unlock Full Question Bank
Get access to hundreds of Strategic Solution Design interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.