Marketing and Growth Analytics Questions

Shapley-value and Markov-chain removal (also called removal-effect or conditional Markov attribution) are two principled approaches to multi-touch channel attribution; both aim to allocate conversion credit across channels but use different assumptions and mechanics.

Definitions

• Shapley value: From cooperative game theory, computes each channel’s average marginal contribution across all possible permutations of channel orderings. It’s model-agnostic and accounts for interactions symmetrically.
• Markov-chain removal effect: Models user journeys as a Markov chain (states = channels, transitions = moves), computes conversion probability, then attributes credit by measuring drop in conversion when a channel (state) is removed (absorption probabilities).

Strengths

• Shapley:
  • Fairness: axiomatic (efficiency, symmetry, additivity).
  • Captures interactions and synergy among channels regardless of temporal structure.
  • Intuitive decomposition into marginal contributions.
• Markov removal:
  • Respects sequential behavior and transition dynamics.
  • Naturally handles order-dependent effects, path-length, and looping behaviors.
  • Gives interpretable metrics like transition probabilities and conversion likelihoods.

Weaknesses

• Shapley:
  • Ignores temporal ordering unless features include sequence info; treats channels as set elements.
  • Computationally expensive: O(2^n) subsets; needs approximations (sampling, Aumann-Shapley, or stratified permutations).
  • Sensitive to definition of “value” (conversion function used per coalition).
• Markov removal:
  • Assumes Markov property (memoryless beyond current state) which may be violated.
  • Removal can distort transition structure; need careful re-normalization.
  • Less suited to capture complex higher-order interactions or long-range dependencies unless using higher-order Markov models (costly).

Computational considerations

• Shapley: exact for small n; for many channels use Monte Carlo sampling or model-based Shapley (e.g., trained response model to compute coalition value). Sampling error decreases with O(1/sqrt(samples)).
• Markov: build transition matrix from paths; computing absorption probabilities is O(k^3) for k states (matrix inversion); removal requires recomputing absorption or using Sherman–Morrison updates for efficiency.

When to prefer which

• Use Shapley when:
  • You need axiomatic fairness and to capture synergy across channels.
  • Channel set is moderate, or you can approximate Shapley reliably.
  • Ordering is less important (e.g., channels represent customer cohorts or exposures aggregated).
• Use Markov removal when:
  • Sequence/order and transition dynamics matter (web journeys, multiple clicks).
  • You have dense path data and want insights into paths and intermediate drop-offs.
  • You prefer a mechanistic model that yields transition-level diagnostics.

Hybrid / practical note

• Combine: use Markov for sequence-level modeling and compute Shapley over meaningful grouped states (first-touch, last-touch, multistep segments), or use Shapley with features derived from sequence models.
• Validate: compare both on holdout conversion predictions and business KPIs; check robustness to data sparsity and channel aggregation.

Final tip: pick the method that matches your data assumptions (order vs set), interpretability needs, and computational budget; when in doubt, run both and surface differences to stakeholders.

I’d evaluate the choice across five pragmatic criteria, weigh them, and document a decision rubric for stakeholders.

1. Business impact & required accuracy

• Quantify downstream cost of prediction errors (revenue, fraud loss, user experience).
• If higher accuracy materially reduces risk/cost, favor the ensemble; if gains are marginal, prefer simpler model.

2. Stakeholder needs & trust

• Who consumes model outputs? Product managers and ops may accept a black-box if performance uplift is clear and measurable; legal, customer support, or executives may require clear explanations.
• If decisions require user-facing explanations or manual overrides, prefer interpretable models.

3. Regulatory & compliance constraints

• If regulations mandate explainability, variable-level accountability, or audit trails (e.g., finance, healthcare, GDPR-related profiling), use the interpretable model or add explainability layers that meet legal standards.

4. Maintenance, monitoring, and operational cost

• Ensembles often need more compute, retraining, and complex monitoring. Estimate infra and engineering cost, latency constraints, and model lifecycle effort.
• If team bandwidth or infra budget is limited, prefer simpler models that degrade gracefully and are easier to maintain.

5. Mitigation & hybrid approaches

• Consider middle-ground: use interpretable model in production with ensemble as a backstop, or stack models where ensemble scores are explained via SHAP/LIME and stable, simpler surrogates.
• Run A/B tests, shadow deployments, and produce counterfactual explanations; require model cards, documented trade-offs, and rollback plans.

Decision process:

• Score each criterion (impact, trust, compliance, cost, feasibility) weighted by stakeholder priorities; run a pilot (2–4 weeks) comparing metrics and explainability artifacts; present results and recommended option with runtime and governance plan.

This ensures the choice balances performance with trust, compliance, and operational reality.

Cohort analysis groups users by a shared start event (here: signup week) and tracks behavior (here: events) over time. 30-day retention per signup_week cohort measures the fraction of users who return (have any event) within 30 days after signup.

Approach:

• Bucket signups into signup_week (e.g., ISO week or week starting Monday).
• For each user, check if they had at least one event within 30 days after their signup_date.
• Aggregate by cohort to compute retention rate = users_with_event_within_30_days / cohort_size.

SQL snippet (standard SQL):

WITH users_cohort AS (
  SELECT
    user_id,
    signup_date,
    DATE_TRUNC('week', signup_date) AS signup_week -- adjust function per dialect
  FROM users
),
events_in_30 AS (
  SELECT DISTINCT u.user_id, u.signup_week
  FROM users_cohort u
  JOIN events e
    ON e.user_id = u.user_id
   AND e.event_date BETWEEN u.signup_date AND u.signup_date + INTERVAL '30 days'
)
SELECT
  u.signup_week,
  COUNT(DISTINCT u.user_id) AS cohort_size,
  COUNT(DISTINCT e.user_id) AS retained_30d,
  ROUND(100.0 * COUNT(DISTINCT e.user_id) / NULLIF(COUNT(DISTINCT u.user_id),0), 2) AS retention_30d_pct
FROM users_cohort u
LEFT JOIN events_in_30 e
  ON u.user_id = e.user_id
GROUP BY u.signup_week
ORDER BY u.signup_week;

Notes & edge cases:

• Use DATE_TRUNC/DATE_ADD syntax per SQL dialect.
• Decide whether to include day 0 (signup day) as retention.
• Handle timezones and users with multiple events (use DISTINCT).
• Exclude bots/test accounts.

Visualization for product managers:

• Primary: Line chart of retention_30d_pct over signup_week to show trend.
• Secondary: Bar chart with cohort_size and overlayed retention percent.
• Add confidence intervals if cohort sizes small; annotate major releases/marketing campaigns for context.
• Provide table with cohort_size, retained_30d, retention_30d_pct and filters (country, platform) for deep-dive.

Approach: use a window function to rank touches per conversion_id by event_time (descending) and pick the top row per conversion. Add a deterministic tie-breaker (e.g., session_id) if multiple touches share the same timestamp.

-- ANSI SQL: last-touch per conversion_id using ROW_NUMBER()
SELECT
  conversion_id,
  touch_channel AS last_touch_channel,
  event_time AS last_touch_time,
  user_id,
  session_id
FROM (
  SELECT
    t.*,
    ROW_NUMBER() OVER (
      PARTITION BY conversion_id
      ORDER BY event_time DESC, session_id DESC  -- tie-breaker
    ) AS rn
  FROM touches t
  WHERE conversion_id IS NOT NULL  -- only consider touches tied to a conversion
) x
WHERE rn = 1;

Key points:

• ROW_NUMBER() picks the single latest touch per conversion_id.
• ORDER BY event_time DESC finds the last-touch; session_id DESC is an optional deterministic tie-breaker.
• If you want to allow multiple channels when timestamps tie, replace ROW_NUMBER() with RANK() and filter rank = 1.

Edge cases:

• Null conversion_id: filtered out above; include if needed.
• Identical event_time: decide policy (tie-breaker or return multiple).

Definition & formula:
Customer retention rate (monthly) = (Number of users active in month M and still active in month M+1) / (Number of users active in month M) × 100%.
This measures month-to-month stickiness for an existing active population.

Assumptions (explicit):

• A user is "active in month X" if they have any active subscription-days in that calendar month (partial-month activity counts as active).
• cancel_date is the last active day (inclusive). start_date is first active day.
• Single subscription per user; reactivations are treated as new active periods but the same user can be counted as retained if active across months.

Sample users (2023):

• A: start 2023-01-10, cancel 2023-03-05 → active in Jan, Feb, Mar (partial)
• B: start 2023-02-01, active → active Feb, Mar...
• C: start 2023-01-15, cancel 2023-01-31 → active only Jan

Compute:

• Jan → Feb:
  • Active in Jan = {A, C} = 2
  • Of those, active in Feb = {A} = 1
  • Retention = 1/2 = 50%
• Feb → Mar:
  • Active in Feb = {A, B} = 2
  • Of those, active in Mar = {A, B} = 2
  • Retention = 2/2 = 100%

Edge cases & notes:

• Partial-months: alternate approach is prorated-retention (count by days active / days in month) — use when revenue-weighted retention is needed.
• Reactivations: decide whether to treat reactivated users as retained (continuous) or new (depends on business definition).
• Trials/comped periods: usually exclude or label separately.
• Timezones, missing dates, multiple subscriptions, and cancels effective at period boundary (midnight vs inclusive) need consistent handling.

Reporting to stakeholders:

• Provide cohort retention table (rows = cohort month, columns = month offset) + line charts for N-day/30-day rolling retention.
• Show counts alongside percentages, confidence intervals, and segment by acquisition channel, plan, and region.
• Annotate product/marketing events that could explain changes.
• Recommend KPIs: 30/60/90-day retention, LTV implications, and actionable segments (e.g., low-retention cohorts for targeted onboarding).