Ad Server Simulation and Auction Mechanics Architecture Questions
Explores the architecture of ad-serving platforms, including modeling and simulating ad server workloads, the real-time bidding (RTB) auction flow, ad exchange integrations, and the end-to-end pipeline from impression to bid decision. Covers low-latency design patterns, throughput and latency budgets, distributed components (ad server, DSP/SSP, bid stream processors), caching, data consistency, fault tolerance, sharding/partitioning, deployment strategies, telemetry and monitoring, testing approaches for high-frequency decisioning, and considerations for privacy and measurement accuracy within large-scale ad ecosystems.
MediumTechnical
51 practiced
How would you design feature pipelines to support sub-20ms model scoring for an RTB system? Discuss precomputation strategies, hot-path caching, compressed feature encodings, approximate data structures (e.g., Bloom filters, sketches), and the trade-offs between feature freshness and scoring latency.
Sample Answer
Requirements & constraints:- End-to-end scoring <20ms per RTB request, high QPS, features from streaming and batch sources, acceptable staleness window (define with product).High-level approach:- Push as much work offline / upstream as possible; keep hot-path minimal and in-memory.Precomputation strategies:- Precompute heavy aggregates and user/item embeddings in batch (hourly/daily) and materialize into a fast key-value store (Redis, Aerospike).- Maintain incremental online updates for counters or TTL-based derived features using stream processors (Flink, Kafka Streams) to reduce full recompute on hot keys.Hot-path caching:- Use a two-tier store: in-process LRU cache (nanoseconds) for top-N hot keys + low-latency remote KV for cold keys. Populate in-process cache from remote asynchronously.- Warm caches proactively for expected auctions (top bidders, premium users).Compressed feature encodings:- Pack features into compact binary blobs (protobufs, FlatBuffers) or fixed-width arrays to minimize serialization/deserialization.- Quantize floats (e.g., 8-bit) and use delta encoding for timestamps to reduce payload and parsing time.Approximate data structures:- Use Bloom filters to test membership (e.g., cookie inclusion) cheaply.- Use count-min sketches or HyperLogLog for approximate high-cardinality counts or frequency features where exactness is not critical.- Use locality-sensitive hashing (LSH) or product quantization for approximate nearest-neighbor lookups for embeddings.Freshness vs latency trade-offs:- Categorize features: - Critical real-time: must be freshest (e.g., session-level indicator) — compute in hot-path with minimal ops. - Nearline: tolerable staleness minutes/hours (user lifetime value) — precompute offline. - Approximate: can be stale/approximate (frequency sketches).- Policy: favor precomputation + approximation for expensive features; reserve real-time computation for low-cost, high-signal features.- Monitor model degradation vs staleness and implement fallback to conservative defaults when fresh features missing.Observability and operations:- Track cache hit rates, tail latency, feature staleness, and model performance. Implement A/B tests to quantify impact of staleness vs accuracy.- Graceful degradation: if remote store times out, use cached/approximate features or model fallback.Trade-offs summary:- More precomputation and approximation => lower latency, reduced accuracy/freshness risk.- More real-time computation => fresher features but higher tail latency and cost.Choose mix based on signal importance, cost, and SLA.
HardSystem Design
69 practiced
Design a multi-region bidding architecture that provides low per-region latency, respects regional privacy laws, and avoids cross-region tail latency. Discuss replication strategies for feature stores (synchronous vs asynchronous), routing logic to DSPs closest to requests, and consistency models for global counters like budgets and frequency caps.
Sample Answer
Requirements & constraints:- Low per-region latency (<10–20ms decision loop)- Regional data residency: PII and regionally-protected features must not leave region- Avoid cross-region tail latency during bid-time- Correctness for global counters: budgets, frequency capsHigh-level approach (hybrid, region-first):- Deploy full bidding stack in each region: feature store (regional), model scorer, auction router, and local cache of DSP endpoints.- Keep raw PII and any protected features only in-region; replicate only anonymized/non-sensitive feature vectors where allowed.Feature-store replication strategy:- Synchronous replication: only for tiny, critical metadata that must be consistent (e.g., DSP status flags, global kill-switch). Use consensus (Raft/ETCD) across regions only when updates are rare and read-latency tolerant.- Asynchronous replication (recommended for most features): update primary-region store and fan-out deltas to other regions. Use versioned features (vector clock / timestamps) and schema for eventual convergence. For sensitive features, do not replicate—compute on-demand from in-region sources or return aggregated safe signals.- Materialized views: keep local feature caches refreshed frequently (sub-second to seconds) to serve scoring without cross-region calls.Routing logic to DSPs:- Edge routing: geo-DNS / Anycast + regional edge LB selects nearest regional bidding instance.- DSP affinity: maintain per-region DSP endpoint list with latency measurements; route to the nearest DSP endpoint to the user region (not the bidder). Use health/latency scoring and prefer local POPs. If a DSP requires cross-region delivery, use asynchronous pre-warming or relay proxies in target region to avoid bid-time cross-region hops.- Consistent hashing or key-affinity for user IDs ensures warm caches and stable affinity, reducing cold-starts and tail latency.Consistency models for global counters (budgets, freq caps):- Budgets: use a hybrid token-bucket reservation model: - Globally allocate budget tokens to regions periodically (e.g., every minute/hour) via a coordinator. Each region consumes from its local token bucket with strong local guarantees (no cross-region call at bid-time). Coordinator reconciles remaining tokens and rebalances. For high-precision budgets (billing), perform async reconciliation and corrective accounting (allow eventual compensation).- Frequency caps: keep local per-user counters in-region for immediate enforcement. To avoid double-serve across regions, use one of: - Sticky routing (prefer same region / user‑affinity) + local counters — simple and low-latency but may overcount under multi-region access. - CRDT (G-Counter with windowing) asynchronously merged and materialized; enforce cap conservatively by subtracting a safety margin when local counter uncertain. - For strict global uniqueness, use a fast consensus-based reservation for critical users (rare path) — e.g., lease/key ownership for short windows.- In all cases maintain an audit/log stream for reconciliation and backfill corrections.Trade-offs & reasoning:- Synchronous cross-region writes at bid-time kill latency and tail behavior; so minimize synchronous global operations.- Asynchronous replication + local caches optimize latency and scale; accept eventual consistency with compensating reconciliation for billing/fraud.- Critical small metadata can be strongly-consistent (consensus), but cost is served rarely.Operational considerations:- Metrics & monitoring for drift between regions, reconciliation lag, and token exhaustion.- Feature versioning and shadow testing when changing replication cadence.- Privacy: automate data residency rules in replication pipelines; redaction/anonymization at boundary.Example numbers:- Keep bid path <10ms: local cache hit <2ms, model score 3–5ms, routing/DSP call setup 2–3ms.- Token rebalancing interval: tune (30s–5min) based on budget granularity and traffic variability.This hybrid design balances low latency and legal constraints while using tokenized local guarantees plus async global reconciliation for counters and feature replication.
HardSystem Design
66 practiced
Propose a multi-tier scoring architecture: a cheap model filters impressions and a more expensive model refines top candidates. Quantify expected throughput gains, sketch threshold selection methodology, and discuss selection bias introduced by the cascade and how to mitigate it during training and evaluation.
Sample Answer
Requirements & constraints:- Functional: filter N impressions/sec with end-to-end latency budget L (e.g., 50 ms).- Non-functional: maximize throughput, minimize cost, keep recall of positive events ≥ R_target (e.g., 99%), limit false negatives that reduce revenue.- Models: cheap model M1 (lightweight features, e.g., logistic regression/trees/1-layer NN) and expensive model M2 (heavy features, e.g., deep ranking model, large ensemble).- Metrics: throughput (impressions/sec processed by M2), overall latency, precision/recall, revenue impact.High-level architecture:- Inference pipeline: Ingest → Feature extraction tiered (fast features for M1, on-demand heavy features for M2) → M1 scores all impressions → Threshold T sends top fraction f to M2 → M2 refines and final decision → Logging for training/eval.Throughput gains (quantified example):- Let baseline cost per impression: C_baseline = C2 (cost/time to run M2). With cascade: every impression pays C1 (cheap) and fraction f pays C2.- Throughput gain ≈ 1 / ( (C1 + f*C2) / C2 ) = 1 / (f + C1/C2).- Example: C1 = 0.05*C2 (M1 20x cheaper). If we send f=0.1 (10%) to M2 → relative cost = 0.05 + 0.1 = 0.15 → throughput ~ 6.7x vs. all-M2. Latency: average latency ≈ latency1 + f * latency2 (plus queuing effects).Threshold selection methodology:1. Offline ROC/PR analysis for M1 using holdout labeled data; compute score -> probability of positive.2. Choose T by optimizing objective constrained by business metric: - Maximize throughput subject to Recall >= R_target. - Or maximize expected revenue: E[rev | score ≥ T]*P(score ≥ T) subject to cost constraint.3. Use calibration (isotonic/Platt) so M1 probabilities meaningful.4. Use cost-sensitive threshold: define cost_fn = cost_of_missing_positive * FN_rate(T) + cost_of_processing * P(score ≥ T). Pick T minimizing expected cost.5. Safety margin: set T to meet recall on validation and then adjust via A/B test with small traffic fraction, monitoring revenue and latency.Selection bias and mitigation:- Problem: cascade only sends top f impressions to M2, so training data for M2 / offline eval is biased (missing lower-scoring M1 samples), causing overoptimistic estimates.- Mitigations: - Randomized logging: sample a small random fraction r of impressions regardless of M1 score and send to M2 (or at least log true labels). Use these for unbiased estimator of performance and for training M2 calibration. - Importance weighting: when training/evaluating M2 on biased sample, weight examples by 1 / P(selection | features) (propensity score estimated from logged selection policy) to correct distribution. - Two-headed training: train M2 partly on top-scored examples and partly on randomly sampled examples to cover tails. - Data augmentation / simulation: synthesize negative examples where labels are known, but ensure validation uses randomized holdout. - Counterfactual policy evaluation: use IPS (inverse propensity scoring) to estimate expected metrics under different thresholds.Training & evaluation setup:- Maintain three datasets: (1) full historical labeled set where possible (randomized sample), (2) M1-scored dataset, (3) M2 labeled set enriched with positives (but corrected via importance weights).- For threshold tuning, use unbiased metrics from randomized holdout to estimate recall/precision at candidate T.- Periodically re-calibrate M1 and recompute T to account for drift; use continuous monitoring and automatic rollback if revenue drops.Trade-offs and operational considerations:- Sending randomized sample increases cost but small r (0.1–1%) suffices for unbiased estimates.- Lowering f increases throughput but risks business loss; prefer conservative T with gradual ramp.- Feature engineering: compute heavy features lazily only for f fraction to save I/O.- Robustness: fail-open policy (if M1 fails, forward to M2) vs fail-closed depending on SLAs.- Monitoring: track selection distribution, drift in M1 score distribution, IPS-corrected metrics, and live revenue signals.Summary:Implement cheap M1 to filter most impressions, tune threshold T via cost/recall optimization using calibrated scores and randomized holdouts, quantify throughput gains by C1/C2 and f, and mitigate selection bias with randomized logging and propensity-weighted training/evaluation to ensure unbiased performance estimates and stable business outcomes.
EasyTechnical
61 practiced
Given a simplified OpenRTB-like bid request with fields such as 'id', 'imp' (impression), 'site', 'device', 'user', and 'bcat', explain which fields you would use for CTR and bid-price prediction, and which fields need special handling due to privacy, sparsity, or high-cardinality.
Sample Answer
For CTR prediction I'd prioritize fields that directly reflect user intent/context and ad exposure risk; for bid-price prediction I'd use those plus supply-side value signals. Example fields and treatment:Useful straightforward features- imp (banner/video size, position, bidfloor, instl): encode size, position, bidfloor as numeric/categorical — strong predictors of CTR and bid price.- site (domain, page, category): site category and domain-level revenue signals are important — use category as categorical, domain with frequency-based encoding.- device (make, model, os, ua, language): use OS, device type, connection type; normalize UA to extract browser and device class.- user (geo: country/region, age, gender, user_id hashed): geo is important; demographic signals help CTR; hashed user_id for counts/recency features.Special handling (privacy, sparsity, high-cardinality)- user.id / cookie / mobile_id: treat as PII — hash/anonymize, use only aggregated features (user frequency, recency, lifetime value). Avoid raw IDs in models.- site.domain, app.bundle, device.model: high-cardinality — use target/frequency encoding, hashing trick, or embeddings; cap rare categories to “other”.- bcat (blocked categories): use as binary flags for sensitive categories; may suppress bidding or penalize predicted CTR/bid.- ua / user agent free text and page URL: sparse/noisy — extract structured tokens (browser, version, path segments) and/or use NLP/embeddings with dimensionality reduction.- Sparse user attributes (custom segments): group low-frequency segments, use l2 regularization, and calibrate to avoid overfitting.Other considerations- Time features: timestamp → hour, daypart, weekday for temporal behavior.- Privacy/regulatory: ensure GDPR/CCPA compliance, use consent flags, minimize raw PII, and prefer aggregated/hashed signals.- Feature validation: monitor drift and sparsity; retrain encoders regularly and maintain fallback defaults for unseen values.
MediumTechnical
63 practiced
In RTB, conversion labels are delayed and sparse. Describe how you would design attribution windows, label construction, and unbiased estimators for CVR that balance recency and label completeness. Include discussion of click vs view-through windows and consequences for training model targets.
Sample Answer
High-level approach- Tradeoff: longer windows increase label completeness (catch late conversions) but introduce recency bias and label-delay (training data lags); shorter windows improve freshness but undercount conversions. Combine multiple windows + weighting to balance.Attribution windows & rules- Click-through (CT) window: typically shorter and higher-signal (e.g., 1–7 days). Use 7-day CT as primary signal for direct intent.- View-through (VT) window: longer but weaker (e.g., 1–30 days). Use 1–14 days depending on product purchase cycle.- Priority: attribute CT over VT if both occur; record time-since-touch and interaction type.Label construction- For each impression/impression+click event, record: - conversion observed within CT window (binary), time-to-convert t_c - conversion observed within VT window if no qualifying click (binary), t_v- Create multi-horizon labels (y_1d, y_7d, y_30d) so models can learn temporal dynamics and let downstream choose horizon.- Include censoring flag for examples where observation window is truncated (recent events).Unbiased estimators & handling label delay- Use survival / time-to-event modeling (Cox or parametric) or discrete-time hazard models to incorporate censored examples: estimate instantaneous CVR and cumulative incidence up to horizon without discarding recent but uncensored data.- Alternatively use inverse probability-of-censoring weighting (IPCW): weight examples by 1 / P(not-censored) to correct bias from excluding recent events.- For logging bias (non-random exposure), apply inverse propensity weighting (IPW) using a model of the exposure/click probability; stabilized weights reduce variance.Practical weighting scheme- Combine horizon labels with exponential decay weighting for recency in training loss: weight = alpha * IPCW * exposure_stabilizer * exp(-lambda * age). Tune lambda via validation metric aligned to business window.Consequences for model targets & training- Using only long-horizon binary labels yields stale targets and underestimates short-term CVR; using only short-horizon misses late converters. Multi-horizon targets or survival outputs allow flexibility.- IPCW/IPW increases variance; mitigate via regularization, clipping weights, or using ensemble of horizon-specific models.- Monitor calibration separately per horizon and per channel (click vs view). Evaluate with uplift-style metrics and cumulative incidence curves.Example: train discrete-time hazard model with time bins (day 0..30), censoring handled in log-likelihood; infer CVR@7d by summing hazards. This gives unbiased, timely estimates and supports both CT/VT attribution rules.
Unlock Full Question Bank
Get access to hundreds of Ad Server Simulation and Auction Mechanics Architecture interview questions and detailed answers.
Sign in to ContinueJoin thousands of developers preparing for their dream job.