Attribution Modeling and Multi Touch Attribution Questions

Covers the theory and practice of assigning credit for conversions across marketing touchpoints. Candidates should know single touch models such as first touch and last touch, deterministic multi touch models like linear and time decay, and algorithmic or data driven models that use statistical or machine learning techniques. Discuss the pros and cons of each approach including bias introduced by simple models, the data and engineering requirements for algorithmic models, and trade offs between interpretability and accuracy. Topics include model selection aligned to business questions, dealing with long purchase cycles, cross device and cross channel journeys, limitations of deterministic attribution, approaches to model validation, and how attribution differs from causal incrementality testing.

EasyTechnical

38 practiced

Explain how a time-decay attribution model assigns credit across touchpoints. Describe commonly used decay functions (for example exponential half-life and linear decay) and how you would select a decay rate for a six-month purchase cycle.

MediumTechnical

35 practiced

Write a PySpark code snippet (pseudo-code is acceptable) that takes raw event logs with columns user_id, event_time, event_type, campaign_id and outputs ordered sessions per user with a session_id assigned using a 30-minute inactivity threshold. Include key transforms and partitioning choices.

Sample Answer

Approach: Use window functions to sort events per user, compute time difference to previous event, mark session boundaries where gap > 30 minutes, then cumulative-sum to assign session_id. Partition by user for parallelism and locality.

python

# pyspark (pseudo-code)
from pyspark.sql import functions as F
from pyspark.sql.window import Window

# raw: DataFrame(user_id, event_time (timestamp), event_type, campaign_id)
df = raw.select("user_id", F.col("event_time").cast("timestamp").alias("ts"), "event_type", "campaign_id")

# partition/sort for deterministic results and locality
w = Window.partitionBy("user_id").orderBy("ts")

# lag previous timestamp, compute gap in minutes
df2 = df.withColumn("prev_ts", F.lag("ts").over(w)) \
        .withColumn("gap_min", F.when(F.col("prev_ts").isNull(), 0)
                                 .otherwise(F.unix_timestamp("ts") - F.unix_timestamp("prev_ts"))/60)

# mark new session when gap > 30 minutes or first event
df3 = df2.withColumn("is_new_session", F.when((F.col("prev_ts").isNull()) | (F.col("gap_min") > 30), 1).otherwise(0))

# cumulative sum to assign session number per user
df4 = df3.withColumn("session_num", F.sum("is_new_session").over(w.rowsBetween(Window.unboundedPreceding, 0)))

# build session_id (e.g., user + session_num) and order events
result = df4.withColumn("session_id", F.concat_ws("_", F.col("user_id").cast("string"), F.col("session_num").cast("string"))) \
            .select("user_id", "session_id", "ts", "event_type", "campaign_id") \
            .orderBy("user_id", "ts")

Key points:- PartitionBy user_id in windows for parallelism and correctness.- Use timestamp cast + unix_timestamp for gap calc.- session_id generation via cumulative sum ensures contiguous numbering.- Consider handling late-arriving events: run watermarking or re-sessionize in streaming, or bucket users (hash) if single user hotspot. Edge cases: identical timestamps, timezone normalization, events out-of-order — ensure proper sorting and possibly deduplication.

MediumTechnical

41 practiced

When building an ML attribution model that predicts conversion probability, how do you detect and prevent target leakage? Provide concrete examples of features that cause leakage in attribution and describe mitigation techniques such as time-based feature engineering and causal holdouts.

Sample Answer

Situation: Building an ML attribution model to predict conversion probability, target leakage is a major risk because features that include future or post-conversion information will make the model unrealistically optimistic and non-actionable.

How I detect leakage:- Audit feature provenance: ask “could this value be influenced by the target after the prediction time?”- Time-split validation: train/val/test split strictly by time to catch models that exploit future signals.- Leak tests: shuffle timestamps or use permutation importance; extremely high importance for features measured after prediction time is a red flag.- Model calibration drift: near-perfect performance on training but poor temporal generalization signals leakage.

Concrete leaking feature examples:- “time_since_last_conversion” or “is_converted_recently” — includes target info.- “post-click_revenue” or revenue aggregated over window including conversion timestamp.- “session_duration_total” that counts post-conversion activity.- Any flag derived from downstream systems after conversion (order_id, confirmation_flag).

Mitigations:1) Time-based feature engineering: - Define a clear prediction cutoff T_pred per user/session. Only use features created from events with timestamp < T_pred. - Use rolling windows that end at T_pred (e.g., clicks_last_7d_before_T_pred). - When aggregating counts, ensure window excludes time after conversion could have occurred.

2) Label delay and causal holdouts: - Introduce a conversion observation window after T_pred to define labels (e.g., conversion within 7 days). But build features only from data before T_pred. - Causal holdout: reserve a random subset of users where you suppress certain features or apply an intervention to estimate causal effect and detect leakage (e.g., hold out post-impression personalization signals). - Use A/B or quasi-experimental holdouts to validate model predictions in production without feeding back downstream signals.

3) Engineering & validation controls: - Enforce feature pipelines that attach event timestamps and automatically drop features using future events. - Feature lineage tracking and unit tests that assert max(feature_event_time) < T_pred. - Monitor online vs offline performance; large gaps imply leakage or data-distribution shift.

Result: These practices prevent models from learning future/post-conversion signals, ensure realistic performance estimates, and produce actionable attribution insights that generalize in production.

MediumTechnical

30 practiced

How would you attribute credit when users have multiple conversions along a journey (for example add-to-cart, trial-signup, purchase)? Propose a principled method to attribute credit across micro and macro conversions and describe how this would influence channel valuation.

Sample Answer

Approach: treat the user journey as a sequence of events with different business value. Use a principled, transparent credit-allocation framework combining business-weighted conversion values and data-driven decay/causal signals.

Proposal:1. Define conversion hierarchy and business weights. Assign monetary or relative weights to events (e.g., add-to-cart = $1, trial-signup = $10, purchase = $100) reflecting expected LT value or margin. These weights can come from CLV models or historical funnel conversion rates.2. Use a hybrid attribution model: - Primary layer: fractional value-splitting by event weight. For a user who had n events, allocate credit proportionally to event weights normalized to the total weight for that journey. - Secondary layer: time-decay and causal adjustment. Multiply each event’s share by a decay factor based on recency (e.g., exponential decay) and by uplift estimates where available (from experiments or causal inference like difference-in-differences or propensity-score matched uplift). Normalize so total credit equals final-event value.3. Implementation steps: - Build CLV or expected-value per event from historical cohorts. - Instrument events and channel touch metadata. - Estimate causal uplift per channel/event using holdout groups or geo/temporal experiments; fallback to observational uplift models when experiments aren’t possible. - Compute per-journey allocations in batch/stream, aggregating by channel.

How this influences channel valuation:- Channels that drive early micro-conversions get partial credit proportional to the event’s expected downstream value; this prevents overvaluing last-touch channels while still acknowledging purchase drivers.- Causal uplift adjustments penalize channels that appear correlated but don’t causally lift conversions.- Result: a monetized channel ROI that sums expected future value rather than raw counts. This enables better budget allocation toward channels that produce high-value micro-conversions or strong causal lift.

Edge cases and governance:- Re-estimate weights periodically as CLV changes.- Use conservative priors where causal estimates are noisy.- Provide transparency (weights, decay params, uplift sources) so stakeholders can interrogate and simulate alternate scenarios.

Example: a user with add-to-cart, trial, purchase. If weights [1,10,100], decay neutral, and no uplift adjustments, channel A that drove add-to-cart gets 1/(1+10+100)=0.9% of purchase value; channel B that drove trial gets 8.9%; final-touch channel gets 90.9%. If experiments show trial touch has high uplift, increase its multiplier accordingly, shifting valuation toward channels that truly drive conversion.

MediumTechnical

57 practiced

For computationally expensive algorithmic attribution such as Shapley, propose sampling strategies to approximate contribution estimates on massive datasets while controlling bias and variance. Discuss stratified sampling, importance sampling, and bootstrap approaches and trade-offs for each.

Sample Answer

Goal: approximate Shapley value-like contributions on massive datasets with controllable bias/variance and bounded compute.

Framework: treat exact Shapley as expectation over random permutations or subsets; sampling approximates that expectation. Key levers: how to draw subsets, how to weight samples, and how to quantify uncertainty.

1) Stratified sampling- Approach: partition dataset (or feature-coalition space) into strata with similar marginal contribution behavior (e.g., by feature presence, value quantiles, or model score buckets). Sample subsets/permutations within each stratum proportionally or optimally (Neyman allocation) to reduce within-stratum variance.- Pros: large variance reduction when strata are well chosen; unbiased if sample weights reflect stratum probabilities.- Cons: requires domain knowledge to define strata; mis-specified strata can leave bias/variance; implementation complexity increases.- Practical: use pilot runs to estimate per-stratum variance and apply Neyman allocation; combine with control variates for further variance reduction.

2) Importance sampling- Approach: sample subsets/permutations from a proposal distribution that over-samples high-impact coalitions (e.g., subsets with likely large marginal contributions) and reweight samples by importance weights to keep estimator unbiased: w = p(target)/q(proposal).- Pros: can drastically reduce variance if proposal aligns with contribution magnitude; flexible to prioritize rare but important interactions.- Cons: can produce high variance if weights are heavy-tailed; requires good proposal design—often via pilot estimates or parametric approximations (e.g., logistic model predicting marginal gain).- Practical: clip or normalize extreme weights, use adaptive schemes (sequential Monte Carlo) to refine q, and monitor effective sample size (ESS).

3) Bootstrap (resampling for uncertainty)- Approach: use bootstrap over sampled subsets or over data points to estimate estimator variance and confidence intervals. For Shapley, bootstrap can be applied to permutation samples or to dataset-level resampling, yielding empirical CI and bias estimates.- Pros: simple, nonparametric, gives uncertainty quantification and bias diagnostics.- Cons: bootstrap does not reduce bias inherently; expensive if naive; naive bootstrap on dependent structure (e.g., correlated features) can mislead.- Practical: combine with stratified or importance sampling (bootstrap the weighted samples or strata) and use bias-corrected and accelerated (BCa) intervals when appropriate.

Trade-offs and recommendations- Start with stratified sampling to get robust variance reduction with low risk of exploding weights.- If you can identify high-impact coalitions (via model insights), move to importance sampling with careful weight management and adaptive proposals.- Always bootstrap weighted samples to quantify uncertainty and detect residual bias.- Use pilot budget to estimate per-stratum variances and to build proposal q. Monitor ESS, weight kurtosis, and coverage of CI to ensure estimator quality.- Compute: aim for O(M · cost_eval) where M is sampled subsets; choose M by target CI width using variance estimates from pilot/bootstraps.

Unlock Full Question Bank

Get access to hundreds of Attribution Modeling and Multi Touch Attribution interview questions and detailed answers.

Join thousands of developers preparing for their dream job.