Attribution & Conversion Measurement Questions

Measuring what drives a conversion: event tracking and instrumentation, attribution models (first-touch, last-touch, multi-touch), and connecting user actions to outcomes. Covers the analytics plumbing for reliable conversion measurement and the limits of each attribution approach. The scope is the measurement layer for conversions, not the creative or channel strategy that generates them.

MediumTechnical

31 practiced

Compare first-touch, last-touch, and multi-touch attribution for an e-commerce conversion funnel. For each, discuss pros/cons, typical business use-cases, and how attribution choice affects reported acquisition channel ROAS.

Sample Answer

First-touch, last-touch, and multi-touch attribution allocate credit for an e-commerce conversion differently. Below is a concise comparison with pros/cons, typical use-cases, and how each choice shifts reported acquisition-channel ROAS.

First-touch- What: 100% credit to the first touchpoint (e.g., initial ad, organic search) that introduced the user.- Pros: Simple; highlights channels that drive awareness and top-of-funnel growth; easy to implement in SQL dashboards.- Cons: Ignores later influences (retargeting, email); overvalues channels that generate clicks but don’t close.- Use-cases: Evaluating brand awareness campaigns, new-market entry, or acquisition channel discovery.- ROAS impact: Inflates ROAS for channels that generate many first visits; underreports performance of conversion-focused channels (retargeting, email).

Last-touch- What: 100% credit to the final touch before conversion.- Pros: Simple; aligns with short-funnel conversion optimization; useful when attribution window is short.- Cons: Ignores upper-funnel and assist interactions; can bias toward paid search and direct channels.- Use-cases: Optimizing checkout experience, crediting last-step promos, short sales cycles.- ROAS impact: Boosts ROAS for channels that appear late in funnel (paid search, direct, cart-recovery email); downweights channels that primarily assist earlier.

Multi-touch- What: Distributes credit across multiple touchpoints (linear, time-decay, position-based, algorithmic).- Pros: More realistic view of customer journey; credits assists; supports nuanced budget allocation.- Cons: More complex to implement and explain; requires clean user-level attribution across devices/sessions; model choice affects outcomes.- Use-cases: Holistic media mix optimization, cross-channel planning, lifetime value modeling.- ROAS impact: Smooths extremes—reduces first/last-touch inflation by assigning partial credit to assist channels. Choice of weighting (time-decay favors recent touches; position-based favors first/last) materially changes reported ROAS by channel.

Practical recommendations for a Data Analyst- Report multiple views (first-touch, last-touch, and a chosen multi-touch) side-by-side to show sensitivity.- Use multi-touch for strategic budgeting, but maintain first/last-touch for simpler tactical KPIs.- Always document attribution window, deduplication rules, and user identity assumptions—these materially change ROAS calculations.

HardTechnical

36 practiced

A paid-marketing campaign ran last month overlapping with organic growth. How would you attribute observed conversion lift to the campaign vs organic effects? Discuss at least three approaches (last-touch, multi-touch regression, incrementality test), their assumptions, and preferred diagnostics to validate attribution.

Sample Answer

High-level goal: separate conversions driven by the paid campaign from those that would have happened via organic growth. Below are three approaches, their core assumptions, failure modes, and diagnostics I’d run as a Data Analyst.

1) Last-touch attribution- What: Attribute conversion to the last channel a user interacted with before converting.- Assumptions: Last interaction is causal; no significant cross-channel carryover; user journeys are short and non-overlapping.- Failure modes: Over-credits paid if it often occurs last (e.g., retargeting); ignores assisted conversions.- Diagnostics: Compare channel touch-path distributions, conversion rate by time-since-last-touch, average touches per user. Check bias by cohort (users exposed vs unexposed) and examine change in organic baseline before/during campaign.

2) Multi-touch / regression-based attribution- What: Use a model (e.g., logistic regression, Poisson, or time-decayed attribution) to estimate contribution of each touch while controlling for covariates.- Assumptions: Model correctly specifies relationships; observed confounders capture user heterogeneity; no unobserved time-varying confounders.- Failure modes: Omitted-variable bias, multicollinearity between channels, overfitting.- Diagnostics: Check VIFs for multicollinearity, holdout validation and calibration, residual/time-series diagnostics, coefficient stability across subsamples, include temporal dummies and user-fixed effects if possible. Use propensity scores to balance exposures and test sensitivity to included covariates.

3) Incrementality / randomized or quasi-experimental test (preferred for causal claim)- What: Randomize ad exposure (or use geo/time A/B, holdout groups) and measure difference-in-differences in conversion.- Assumptions: Randomization (or parallel trends for DID) holds; no spillovers between treatment and control (SUTVA).- Failure modes: Poor randomization, contamination (users see ads via other channels), insufficient power.- Diagnostics: Pre-period balance checks, pre-trend tests (for DID), power calculation and minimum detectable effect, check for contamination via impressions logs, placebo windows (pretend test period) and permutation tests, compute confidence intervals and lift % with p-values.

Practical recommendation: If you need causal claims, run an incrementality test (geo or user-level holdout). If that’s not feasible retroactively, use regression with rich controls, user-fixed effects, and robustness checks (placebo periods, sensitivity to omitted confounders). Always report uncertainty, assumptions, and diagnostics so stakeholders understand confidence in the attributed lift.

MediumSystem Design

38 practiced

Design a measurement and attribution plan for a multi-channel marketing campaign (email, paid search, social) with goals of acquisition and revenue. Specify primary events to track, attribution window and model, necessary data joins, how to surface incremental lift vs baseline, and how to handle cross-device users.

Sample Answer

Requirements & constraints:- Goals: new user acquisition and revenue (LTV). Channels: email, paid search, social. Need channel-level and campaign-level ROI, incremental lift, cross-device coverage, privacy-compliant.

High-level plan:1. Primary events to track- Exposure events: ad_impression (platform, campaign, creative, timestamp, click_id)- Interaction events: ad_click, email_send, email_open, email_click- Engagement & conversion: signup (user_id, anonymous_id, ts), purchase (order_id, user_id, revenue, ts), downstream events (subscription_renewal, churn)- Micro-conversions: product_view, add_to_cart- User identity events: login, account_create (email/hash), device_id, cookie_id

2. Attribution window & model- Window: clicks — 7-day for acquisition, 30-day for revenue; view-through — 1 day for paid; configurable per analysis.- Primary model: multi-touch probabilistic MTA (Markov chains or Shapley) for channel contribution across touchpaths, supplemented by last-click for simple operational reporting.- Use time-decay weighting for analyses focusing nearer interactions.

3. Necessary data joins & keys- Source tables: Ad platforms (gclid/fbclid/etc), Email platform, Web analytics (page events), Backend orders, CRM.- Join keys: deterministic: user_id (logged-in), email_hash, order_id, click_id when available. Fallback: cookie_id/device_id with timestamp overlap.- Pipeline: ingest raw events -> normalize schema (event_type, ts, attrs) -> identity stitching -> sessionize -> touchpath construction.- SQL joins: prefer left joins from conversions to exposures using click/impression windows and dedupe by earliest relevant touch.

4. Incrementality vs baseline- Best practice: randomized holdout (test/control) at user or geo level for campaigns; measure difference in acquisition and revenue over the attribution window. Compute lift = (metric_test - metric_control) / metric_control with bootstrap CIs.- If holdouts unavailable: use observational causal methods — difference-in-differences, propensity score matching, or uplift models; still report uncertainty and potential biases.- Surface baseline: control group's metric baseline; incremental dashboards show absolute lift, relative lift, p-value, and confidence intervals.

5. Cross-device handling- Deterministic stitching first: unify events by authenticated user_id/email_hash across devices.- Probabilistic linking: device graph using IP/time/UA patterns only when deterministic absent, with conservative thresholds and privacy review.- Report coverage: percent of conversions deterministically linked vs probabilistic; flag analyses limited to deterministic subset to avoid overclaiming.

6. Reporting & dashboards- KPI views: acquisition (new users), revenue (7/30/90 day), CAC, ROAS, LTV.- Attribution reports: MTA channel contributions, touchpath funnel, last-click table.- Incrementality reports: test vs control, lift by channel/campaign, statistical significance, recommended budget reallocations.- Data quality metrics: missing IDs, duplicate orders, join failure rates.

Trade-offs & notes:- MTA gives nuanced credit but needs good touchpath data; holdouts give causal lift but cost exposure of withheld audiences.- Be explicit about assumptions, windows, and identity coverage in all stakeholder reports.

HardTechnical

67 practiced

Design an analysis strategy for experiments that span multiple devices where identity stitching is imperfect. Explain how you would estimate treatment effect bias due to misattribution, propose sensitivity analyses, and recommend conservative decision rules when cross-device matching is noisy.

Sample Answer

Situation/Goal: We must estimate treatment effects for experiments where users appear on multiple devices but identity stitching is imperfect. Misattribution (false merges/splits) biases estimated effects. The strategy below treats the problem as measurement error in the assignment and outcome and produces bias bounds, sensitivity checks, and conservative decision rules.

1) Model the bias mechanism- Define true unit = user, observed unit = device or stitched-ID. Let p_match = probability that two records from same user are stitched; let α = false positive rate (distinct users merged); β = false negative rate (same user split).- If treatment is assigned at user level but observed at device-level, attenuation bias occurs. Under classical nondifferential misclassification, observed ATE_obs ≈ (1−γ)·ATE_true where γ depends on α,β and the distribution of device counts.

2) Estimate or bound matching error parameters- Use deterministic subsets: logged-in users, hashed emails, or deterministic cookies as ground truth to estimate α, β, p_match.- Use deterministic-tagged holdout users to measure cross-device exposure rates.- If no ground truth, bound α,β using plausible ranges from engineering estimates (e.g., match algorithm precision/recall).

3) Bias correction and bounding- Analytic correction: if you can estimate sensitivity (Se = 1−β) and specificity (Sp = 1−α) of treatment assignment mapping, invert misclassification to adjust ATE: ATE_true ≈ (ATE_obs − (1−Sp)·Δ_control)/(Se + Sp −1) where Δ_control is difference induced by false merges on controls (approx 0 if random).- If analytic inversion unstable, compute worst-case bounds (Manski-style) by scanning α,β over plausible ranges and report min/max ATE consistent with data.

4) Sensitivity analyses (recommended battery)- Parameter sweep: vary α,β across credible ranges and plot resulting ATE and p-values.- Simulation-based: simulate user-device graphs consistent with observed device-count distribution, inject matching errors, and re-run A/B analysis to get distribution of bias.- Subgroup validation: run analysis on high-confidence matched cohort (logged-in users) and compare effect size to full cohort.- Time-based: if treatment rollout staggered by device usage patterns, check temporal stability to detect differential misattrition.- Placebo tests: run faux treatments or outcomes unlikely to be affected; significant effects indicate residual bias.

5) Conservative decision rules- Prefer decisions based on high-confidence cohorts (logged-in) when results differ.- Require stronger evidence when cross-device matching noisy: raise required minimum detectable effect (MDE) or tighten alpha (e.g., use 99% CI) before productizing.- Use directional rules: if bounds include zero but sign consistent and economically meaningful at upper bound, consider limited rollout with monitoring; if sign flips under plausible α,β, delay decision.- Implement sequential rollouts: start with small Canary on high-confidence matched users, collect behavioral metrics, then expand.- Report decisions with provenance: publish ATE_obs, corrected ATE with point estimate (if stable), and bound interval + assumed α,β; include automated dashboard that toggles parameters.

6) Practical implementation notes- Instrumentation: log deterministic identifiers where possible; capture device graphs and join quality metrics.- Reporting: provide plots of ATE vs α/β, cohort overlap matrices, and subgroup comparisons.- SQL-style check: count unique devices per user, fraction of multi-device users among treated/controls, and compute ATE on logged-in subset.

Outcome: This approach quantifies how much misattribution could change conclusions, provides transparent sensitivity analyses, and enforces conservative rules that reduce wrong product decisions when cross-device matching is noisy.

HardTechnical

44 practiced

Describe methods to attribute revenue impact across overlapping marketing channels (multi-touch attribution). Explain rule-based, heuristic, and data-driven approaches (last-touch, linear, U-shaped, regression-based, Markov models), their assumptions, trade-offs, and how you would validate any chosen approach under business constraints.

Sample Answer

Multi-touch attribution assigns credit for a conversion across multiple marketing touchpoints. Approaches fall into rule-based/heuristic and data-driven — each with different assumptions, complexity, and validation needs.

Rule-based / heuristic- Last-touch: credit all to final touch. Simple, stable, easy to implement (SQL/dashboard), but overweights lower-funnel channels and ignores prior influence.- Linear: equal credit across touches. Neutral, easy to explain, but assumes equal contribution which is rarely true.- U-shaped (position-based): more weight to first and last touches (e.g., 40/20/40). Reflects discovery and conversion, intuitive for marketers, but weights are arbitrary and may misallocate mid-funnel impact.

Data-driven- Regression-based (e.g., logistic/linear regression, uplift models): model conversion probability as function of exposures; coefficients estimate marginal contributions. Assumes independence, correct feature specification, and sufficient data. Good for interpretability and controlling covariates (seasonality, spend), but vulnerable to multicollinearity and selection bias.- Markov chain / removal effect: model touchpoint transitions and compute removal impact on conversion probability. Makes fewer parametric assumptions, captures sequence effects, and quantifies incremental impact. Requires session-level sequence data and is compute/heavy; may under-represent long-term brand effects.

Trade-offs- Simplicity vs. accuracy: heuristics are fast and explainable; data-driven approaches are more accurate but require data quality, engineering, and statistical expertise.- Interpretability: regression is interpretable; Markov yields intuitive marginal effects but is less directly linked to spend-efficiency.- Data requirements: more advanced methods need session-level logs, user identifiers, and sufficient sample size.

Validation under business constraints1. Sanity checks: compare aggregated channel credit to known spend/CPAs; check seasonality and unusual spikes.2. Holdout / incrementality tests: run A/B or geo experiments for one or two channels (if possible) to compare predicted vs. observed lift — gold standard for validation.3. Predictive validation: use model to predict conversions on holdout periods; measure calibration, AUC, or mean absolute error vs. real conversions.4. Sensitivity analysis: vary model assumptions (e.g., weights in heuristics, feature sets in regression) and measure stability of channel rankings.5. Business-criteria checks: align results with qualitative knowledge (campaign timing, creative changes) and stakeholder feedback.

Recommended practical path for a Data Analyst- Start with heuristics (last-touch, linear, U-shaped) in dashboards for quick insights and stakeholder alignment.- Parallel-build a regression or Markov implementation (sessionized data) to demonstrate improved allocation and run predictive/holdout validation.- Where feasible, run targeted incrementality tests to confirm high-impact re-allocations before changing budgets.

Example quick SQL for last-touch aggregation:

sql

SELECT channel, COUNT(*) AS conversions
FROM (
  SELECT user_id, MAX(event_time) AS last_time,
         MAX(CASE WHEN event_time = MAX(event_time) OVER (PARTITION BY user_id) THEN channel END) AS last_channel
  FROM events WHERE is_conversion OR is_touch
  GROUP BY user_id
) t
GROUP BY channel;

This balanced approach provides actionable insights while managing complexity and business risk.

Unlock Full Question Bank

Get access to hundreds of Attribution & Conversion Measurement interview questions and detailed answers.

Join thousands of developers preparing for their dream job.