Domain Specific Problem Solving Questions
Addresses problem solving in a specific business or technical domain. Candidates demonstrate familiarity with domain constraints, typical failure modes, stakeholder expectations, and domain specific testing or rollout considerations. Examples include marketing technology challenges such as campaign integrity, data sync issues, third party integrations, privacy and compliance impacts, and cross functional coordination. Candidates should show how they adapt general methodology to domain specifics and evaluate trade offs relevant to the domain.
MediumSystem Design
98 practiced
Design an end-to-end data sync approach between an internal campaign store and a third-party DSP that must remain eventually consistent while minimizing duplicate deliveries. Describe message formats, versioning, reconciliation strategy, conflict resolution rules, and how you would monitor sync health for 100k updates per day.
Sample Answer
Requirements:- Eventually consistent updates from internal Campaign Store → third-party DSP.- Minimize duplicate deliveries.- Handle 100k updates/day (~1.16 updates/sec average, peaks higher).- Detect/reconcile missed or conflicting state.High-level design:- Outbound change-capture pipeline: write-ahead log (WAL) in Campaign Store emits events (CREATE/UPDATE/DELETE) to a Kafka topic. A Sync Worker reads topic, transforms to DSP message, and sends via DSP API (HTTP or MQ). Responses recorded to an Ack store.- Periodic reconciliation job compares authoritative Campaign Store state to DSP-reported state (via DSP API) and replays missing/incorrect events.Message format & versioning:- JSON envelope: { "campaign_id":"string", "op":"create|update|delete", "payload":{...}, // full desired state (idempotent) "version": integer, // monotonically increasing per-campaign "timestamp": ISO8601, "source":"campaign-store-v1" }- Versioning: campaign-level sequence number incremented on each change. Use semantic message version header for schema evolution (e.g., schema_v=1). Keep payloads full-state to allow idempotency and reconciliation.Delivery & duplicate minimization:- Use idempotent PUT-like semantics at DSP: include campaign_id + version. DSP should ignore messages with version <= already-applied version.- Sync Worker keeps an in-flight dedupe cache (Redis) keyed by (campaign_id, version) for short TTL.- Retry with exponential backoff for transient failures; use at-least-once delivery but idempotent payloads + versioning to avoid duplicate effect.Reconciliation & conflict resolution:- Reconciler runs daily/hourly and also on-demand for alerts. It: - Fetches campaigns modified since last run from Campaign Store. - Calls DSP API to read corresponding state and compare version & payload hash. - If DSP version < store version: enqueue message resend. - If DSP version > store version: mark as conflict and follow policy: - Prefer authoritative Campaign Store (push store's state) if business rule. - Or, if DSP holds buyer-side overrides, surface to product/ops for manual resolution and tag campaign for intervention.- When partial mismatches exist (fields differ), reconciler chooses field-level merge rules: store wins for targeting/pricing; DSP wins for runtime metrics.Monitoring & observability:- Metrics: counts of outbound events, success/failure rates, retry counts, reconciliation actions, conflict count, lag (WAL offset vs processed).- Alerts: error rate > threshold, reconciliation corrections > x/hour, persistent failures for a campaign, queue lag.- Traces: distributed tracing for each message (correlation_id).- Dashboards: throughput, latency to DSP, per-campaign failure hotspots, age of last successful sync.- Health checks: consumer group lag, Redis cache size, Ack store consistency.- Audit logs: immutable event log and action history for each campaign.Operational considerations:- Backfills use snapshot export & version stamping.- Secure: sign/encrypt messages, rotate credentials.- Scale: Kafka partitions by campaign_id hash, horizontally scale workers.- SLA: target eventual convergence within N minutes/hours depending on priority; provide per-campaign retry/backoff policies.
HardSystem Design
76 practiced
Design a contract-testing framework for integrating with multiple ad platforms that expose different API versions and sometimes return non-deterministic responses. Include consumer-driven contract ideas, a mock server strategy for CI, version matrices, and how to escalate when a breaking change is detected in production.
Sample Answer
Requirements & constraints:- Multiple ad platforms, varying API versions, occasional non-deterministic responses (rate limits, ad auction randomness).- CI must validate integrations before deploys; production fast detection + rollback/escalation for breaking changes.- Consumer-driven contracts (CDC), reproducible mocks, matrix testing across versions.High-level architecture:1. Contract Registry (central store) — stores consumer-driven contracts per consumer-provider-version (use Pact or OpenAPI contracts with semantic version tags).2. Contract Test Runner — runs CDCs in CI, validates provider against consumer expectations.3. Mock Server Factory — generates deterministic mocks from contracts (WireMock/Pact Mock Service / custom mock generator) used in unit/integration tests and CI.4. Version Matrix Orchestrator — schedules matrix runs (consumer x provider-version) in CI.5. Production Verification & Observability — runtime monitors, canary/shadowing, automated contract checks, alerting + runbook.Design details:- Consumer-driven contracts: each consumer team publishes expectations (requests, response schemas, allowed variability) into the registry. Use schema + interaction examples and “matching rules” to allow controlled non-determinism (e.g., regex for timestamps, ranges for numeric fields, probabilistic enumerations represented as allowed set).- Contract validation: Two-way: - Consumer tests: consumer runs tests against provider mocks derived from contract (CI pre-merge). - Provider verification: provider runs a provider-test suite that validates it satisfies all consumer contracts for supported versions (CI on provider side).- Mock server strategy for CI: - Generate mocks per contract and spin ephemeral mock servers in CI (containerized). Use seeded randomness and deterministic stubs for non-deterministic fields (configurable seeds) so tests are reproducible. - Provide two modes: strict (fail on unexpected request/response) for contract enforcement, and lenient for exploratory/integration tests. - Cache common mock images for speed; run parallel matrix jobs.- Version matrix: - Maintain matrix metadata: consumers, provider versions supported, backward/forward compatibility guarantees. - CI pipeline generates a matrix job graph: for each consumer, test against every provider version the consumer claims to support; for provider changes, run provider verification across all consumer contracts for versions provider advertises. - Prioritize critical paths (highest traffic consumers) and run full matrix nightly.- Handling non-determinism: - Contract includes matching rules instead of exact values; mock server provides deterministic samples that conform. - For stochastic behaviors (auction outcomes), include deterministic determinism mode and stochastic mode with recorded seeds; validate statistical properties in separate end-to-end tests rather than unit contracts.- Production breaking-change detection & escalation: - Runtime contract watchdog: lightweight agent validates live provider responses against published contract schemas and matching rules; emits metrics and SLO-style error budgets. - Canary & traffic shadowing: deploy provider changes to canary population and shadow production requests; compare responses to baseline contract expectations with automatic diffing. - Alerting & automation: if contracts fail beyond threshold, trigger: - Automated rollback (if safe) for recent deploys - Pager + dedicated triage channel listing failing contracts, sample requests/responses, and affected consumers - Runbook actions: revert, hotfix, or coordinated consumer update; include suggested mitigation (feature flag toggles, degrade to fallback ad provider). - Post-incident: record failing contract version, update registry, add compatibility tests, adjust matching rules if change is intentional.Trade-offs and alternatives:- Using Pact/WireMock speeds delivery; building custom registry gives tighter control but costs maintenance.- Strict contract enforcement prevents regressions but can block necessary provider evolution; mitigate with explicit semantic versioning and deprecation windows.- Full matrix every PR is costly—use risk-based sampling and incremental verification (fast path for critical consumers, nightly full matrix).Operational notes:- Enforce semantic versioning for provider APIs; require provider to publish migration guide + compatibility claims in registry.- Make contract publishing part of CI/CD for consumers and providers; treat contracts as first-class artifacts.- Maintain dashboards: contract coverage, failing contracts, matrix test durations, production contract violations.This framework provides deterministic CI testing via mocks, pragmatic handling of non-determinism, systematic version matrix validation, and a clear escalation path when production breaking changes occur.
HardTechnical
92 practiced
You must roll out a breaking change to attribution logic used for billing. Stakeholders include finance, product, sales, and customers. Propose a rollout plan that uses dual-processing or shadow runs to compare old and new outputs, defines acceptance criteria, outlines communication to customers, and details a manual reconciliation path if differences are discovered after rollout.
Sample Answer
Situation: We needed to change core attribution logic used to calculate customer billing. This was a breaking change that would affect invoices and revenue recognition, so finance, product, sales and customers were all stakeholders and risk tolerance was low.Task: As the engineer owner, I had to design a safe rollout that validated the new logic end-to-end, kept stakeholders informed, and provided a clear reconciliation and remediation path if discrepancies appeared.Action:- Preparation: I partnered with product and finance to document the exact scope and business rules of the change and produced an impact matrix (which customers, SKUs, billing flows).- Dual-processing / Shadow Runs: Implemented the new attribution code behind a feature-flag and set up a parallel processing pipeline that ran both old and new logic on the same inputs. Results were written to a comparison table with diffs, root-cause tags, and request IDs for traceability.- Acceptance Criteria: Agreed with finance/product on thresholds: - 0% critical revenue-class differences (>1% for any single invoice) allowed in first 2 weeks; - Aggregate monthly revenue delta < 0.2%; - No unexplained attribution for top 10 customers; all diffs must be triaged and justified.- Monitoring & Validation: Built dashboards showing per-customer deltas, histograms of difference magnitude, and alerts if thresholds breached. Automated daily reports to stakeholders and weekly deep dives with finance.- Phased Rollout: 1) Internal sandbox customers (0% external impact) for 1 week; 2) 5 pilot customers (low-risk) for 2 weeks; 3) 20% of traffic for 2 billing cycles; 4) full rollout if criteria met.- Customer Communication: Drafted email and portal notices explaining the change, expected effects, when to expect invoices, and support channels. For pilots we obtained explicit opt-in and provided reconciliation spreadsheets.- Reconciliation & Remediation: Defined manual path: if post-rollout differences discovered, we would (a) reconcile using the shadow-run diff table, (b) generate adjustment invoices or credit memos with finance approval, and (c) ship an audit report to affected customers. For complex disputes we offered a temporary rollback per-customer by toggling feature flags and reprocessing affected billing periods.- Governance: Weekly sign-off by a small steering committee (engineering lead, finance, product, legal) before progressing phases. All code and scripts reviewed and automated tests expanded to include golden-file comparisons.Result: This approach reduced surprise billing differences, gave finance and sales confidence through transparent metrics, and provided rapid remediation paths. The phased, data-driven rollout minimized customer impact and allowed us to catch and fix edge cases before full deployment.
HardTechnical
83 practiced
A scheduled data-sync job inserted duplicate campaign records affecting billing. Walk through your incident response: containment steps, root cause analysis approach, immediate remediation both technical and customer-facing, and the long-term processes and automation you would implement to prevent recurrence and regain stakeholder trust.
Sample Answer
Situation: Our scheduled data-sync job ran and inserted duplicate campaign records into the billing database, causing overbilling for a subset of customers and triggering support tickets.Containment (first 0–2 hours)- Immediately stop the offending job and any dependent downstream pipelines (disable scheduler, pause CI/CD job triggers).- Put a read-only maintenance flag on billing processes to prevent further invoices or automated charge runs.- Notify Incident Response lead, SRE, product, billing, and support channels with incident severity, owner, and timeline.Root-cause analysis approach (2–24 hours)- Snapshot current DB state and create immutable exports for forensic analysis.- Reproduce the issue in a sandbox using the same job input, code version, and schedule metadata.- Gather logs, job config, recent deploys, migration history, and schema changes; run a diff to identify what changed prior to incident.- Hypothesis-driven RCA: test likely causes (idempotency failure, missing unique constraint, race condition in upsert, clock skew/duplicate triggered runs).- Use distributed tracing and timestamps to map how duplicates entered billing pipeline.Immediate remediation (technical)- Roll back any recent deployments that introduced the regression if identified.- Run a safe deduplication script against billing table using authoritative keys (campaign_id, source_run_id, created_at) in a transactional manner; write operations to an audit table.- Recompute affected customers’ invoices in a controlled environment; generate corrected invoices and flags marking previous charges as superseded.Immediate remediation (customer-facing)- Send proactive, transparent communication to affected customers: incident summary, impact, actions taken, ETA for correction and refunds/credits.- Open a dedicated support line and prioritize ticket routing; offer credits/refunds where appropriate.- Regular status updates (every 2–4 hours initially) until resolved.Long-term prevention and automation- Add database constraints and unique indexes to enforce idempotency at the data layer (e.g., unique on campaign_id + run_id).- Change sync job to idempotent upsert semantics with explicit dedupe keys and optimistic concurrency control.- Introduce transactional audit logging and a “source of truth” ledger for billing entries.- Add pre-commit integration tests and end-to-end tests simulating duplicate run scenarios.- Implement canary/deployment gates and feature flags for data-affecting jobs, plus automated smoke tests post-deploy.- Add monitoring and alerts: anomaly detectors for sudden increases in insert rate, invoice count, or billing totals per customer; automated rollback triggers on threshold breaches.- Build an automated reconciliation job that compares source vs billing daily and opens alerts if discrepancies exceed tolerance.Regaining stakeholder trust- Deliver a blameless postmortem within 72 hours: timeline, root cause, corrective actions, owner, and deadlines.- Publish a customer-facing postmortem explaining cause, fixes, and compensation policy.- Track remediation items to completion with weekly updates and invite product and customer success to review progress.- Propose a SLA improvement plan and demonstrate improved metrics (reduction in discrepancies, faster detection mean-time-to-detect/resolve).What I’d personally own- Lead the immediate technical containment and the dedupe/remediation script, own the RCA artifacts, and coordinate with SRE to implement monitoring and DB constraints. I’d also co-author the postmortem and customer communications with PMs and support to ensure technical accuracy and clarity.
MediumTechnical
75 practiced
How do you prioritize alerts and reduce on-call fatigue while ensuring critical campaign delivery failures are caught quickly? Describe severity classification, deduplication strategies, alert aggregation, owner routing, and a policy for escalating flapping alerts into higher-severity incidents.
Sample Answer
Situation: On a campaign-delivery platform we had noisy alerts that masked real failures and burned out on-call engineers, risking missed high-value campaign delivery.Approach (severity classification):- Define 3 severity levels: P1 (critical: campaign delivery failure impacting >x% customers or SLAs), P2 (partial degradation: retries, delays for subset), P3 (informational: background errors, low-impact anomalies).- Map concrete signals to levels (e.g., delivery queue backlog > 2x SLA and sustained → P1).Deduplication strategies:- De-duplicate by logical fingerprint: campaign_id + error_type + hostname cluster within a time window. Use first-seen alert as canonical and suppress subsequent identical alerts for N minutes while updating counts and timestamps.Alert aggregation:- Aggregate related low-level events into single incident tickets: group by campaign_id or service domain and provide summary (error counts, affected partitions, recent logs, sample failed message). Use rate thresholds to promote aggregated counts into higher severity.Owner routing:- Maintain service ownership metadata in alerting rules (team, primary on-call, escalation chain). Route P1 directly to paging (SMS/call) of primary + secondary; P2 to on-call via push/IM with longer acknowledgement window; P3 to dashboards/email only.- Include runbook links, quick remediation steps, and playbook in the alert payload.Flapping/escalation policy:- Track incident state and alert frequency. If an alert toggles between resolved and firing > 3 times in 15 minutes (or total error-rate variance > 50%), automatically escalate severity by one level and page on-call, mark as “flapping incident,” and create a postmortem task. Optionally put automated throttling or temporary suppression after escalation to force human triage.Result / Rationale:- This reduces noise, ensures real problems surface quickly, ties alerts to owners and runbooks, and treats flapping as an indicator of instability requiring human attention — reducing fatigue and improving SLA compliance.
Unlock Full Question Bank
Get access to hundreds of Domain Specific Problem Solving interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.