Business Problem Structuring and Case Frameworks Questions

1. Question: Does offering bundled discounts (e.g., "Buy 2, get 10% off") increase AOV without reducing margin?

• Short-term A/B test: Randomize visitors 50/50 for 4 weeks: Control sees regular pricing, Test sees bundling with displayed savings. Primary metric: AOV; secondary: margin per order, conversion rate. Statistical plan: minimum detectable lift 3%, 80% power.
• Long-term analysis: 6–12 month cohort analysis comparing revenue, gross margin, and repeat purchase rate for users exposed to bundles. Check for cannibalization (shift from higher-margin single-item sales) and seasonality.

2. Question: Does free shipping threshold (e.g., free shipping over $50) raise AOV and lifetime value?

• Short-term A/B test: Split traffic; Control current threshold, Test raises/lowers threshold for 6 weeks. Track AOV, conversion, average items per order, and incremental shipping cost.
• Long-term analysis: Track cohorts for 12 months to measure changes in CLTV, return rate, and whether customers permanently shift basket size or only during promotion periods.

3. Question: Do personalized cross-sell recommendations increase AOV among returning customers?

• Short-term A/B test: For logged-in users, Test shows personalized "Frequently bought together" based on past purchases; Control shows generic suggestions. Run 6 weeks; measure AOV, add-to-cart rate, and click-through on recommendations.
• Long-term analysis: Segment users by personalization exposure and compare retention, AOV, and product return rates over 9–12 months. Use uplift modeling to isolate incremental impact and monitor for recommendation fatigue or negative UX effects.

For all tests: ensure randomization, monitor secondary KPIs (conversion, returns, customer satisfaction), and run sensitivity analyses by segment (device, geography, new vs. returning).

1. Clarify the objective — Ask targeted questions to confirm the decision to be supported, success metrics, stakeholders, timeline, and constraints. Deliverable: One-line decision statement (objective & KPI).

2. Scope the problem — Define in/out boundaries (geography, product lines, time window, data availability) and assumptions to fast-track analysis. Deliverable: Scope & assumptions sheet.

3. Form hypotheses — Propose 2–4 plausible, testable hypotheses about drivers/solutions (e.g., pricing, churn, channel mix). Deliverable: Hypothesis list (ranked).

4. Build an issue tree — Decompose the main question into mutually exclusive, collectively exhaustive branches mapping to data-enabled analyses (revenue = price × volume; volume = acquisition × retention). Deliverable: Issue tree diagram.

5. Prioritize analyses — Estimate expected impact, effort, and data risk; choose which analyses to run first (80/20). Deliverable: Analysis prioritization matrix.

6. Quantify impacts — Run SQL/statistical analyses, forecasts, sensitivity tests; produce charts and key metrics to measure each hypothesis. Deliverable: Analysis results & model (tables + visuals).

7. Synthesize recommendation — Combine quantified findings, trade-offs, confidence levels, and next steps into a concise recommendation with implementation risks and monitoring plan. Deliverable: One-page recommendation memo (with dashboard links).

Each step aligns with a data-analyst workflow: ask, prepare, test, measure, and communicate.

High-level goal: determine why one cohort’s 30‑day retention dropped 10% vs prior cohorts, identify actionable root causes, and validate with experiments.

1. Exploratory data steps

• Verify metric computation and time alignment (cohort definition, first‑touch timestamp, retention window).
• Run sanity checks for data gaps, ETL failures, or schema changes around cohort start date.
• Trend diagnostics: plot daily/weekly cohort size, 1/7/14/30‑day retention, DAU/MAU, funnel steps leading to day‑30 (A→B→C).
• Statistical test: compare cohort retention with previous cohorts using proportion z‑test and compute confidence intervals.

Example quick cohort query:

SELECT cohort_start, COUNT(DISTINCT user_id) AS users,
  SUM(CASE WHEN retained_day30=1 THEN 1 ELSE 0 END) AS retained30
FROM user_cohorts
WHERE cohort_start BETWEEN '2025-09-01' AND '2025-10-15'
GROUP BY cohort_start;

2. Segmentation to narrow causes

• Product segments: onboarding vs returning users, platform (iOS/Android/Web), acquisition channel, geography, device OS/version.
• Behavior segments: power users vs low-engagement (events in first 7 days), feature adopters vs non-adopters.
• Customer value: payers vs free users.
• Time segments: users who signed up during known incidents, marketing campaigns, or billing cycle boundaries.

3. Instrumentation & feature-flag checks

• Audit event volumes for missing events (e.g., retention heartbeat) and SDK versions deployed to affected cohort.
• Check recent deployments and feature-flag toggles active at cohort start; run “what changed” diff for telemetry and client builds.
• Validate server logs for increased error rates, throttling, or third‑party failures.

4. Qualitative signals

• Rapid surveys to the cohort (short NPS + reason dropdown + free text), and in‑app intercept for lapsed users asking why they left.
• Targeted 15–30 minute interviews with 10–20 users sampled across segments (especially new users and those who used key features).
• Customer support ticket/tag analysis and sentiment trends for the cohort period.

5. Prioritized experiments (rank by ease, impact, confidence)

• Quick checks (low effort): rollback suspect feature flag to previous state for a subset (A/B test) and measure retention.
• Instrumentation fix (if missing events): backfill vs re-instrument and rerun retention computation.
• Product experiments:
  1. Restore previous onboarding flow for 10% random sample vs current (high impact if onboarding suspected).
  2. Re-enable or tweak feature that coincided with drop (feature-flag A/B) for 20% sample.
  3. Email/push re-engagement campaign to the cohort’s at‑risk users vs control.
• Each experiment should have pre-defined metrics, sample size/power calculation, and a measurement plan (primary: 30‑day retention; secondary: 7/14-day retention, conversion to paid).

6. Decision framework & next steps

• Triage: if instrumentation/deployment bug → hotfix + recompute; if product regression → run targeted rollback A/B immediately.
• Use a 2-week investigation window: daily updates to stakeholders, final causal hypothesis with supporting quantitative and qualitative evidence, and recommended fixes/experiments prioritized by ROI and cost.
  This plan ensures we rule out data issues first, narrow likely causes via segmentation, confirm with qualitative signals, and validate via controlled experiments.

Approach summary: present a structured, data-driven assessment that tests clear hypotheses, quantifies market opportunity and unit economics, models risk/sensitivity, and recommends a go/no-go with contingencies.

Key hypotheses (to validate):

• H1: Country X’s addressable market (SAM) in year 1 is ≥ threshold needed to meet revenue targets.
• H2: Unit economics (CAC, LTV, contribution margin) reach break-even within target timeline.
• H3: Competitive landscape and regulatory constraints allow viable entry without prohibitive barriers.
• H4: Operational costs (supply chain, staffing, tax) permit target margins after localization.
• H5: Pricing and product-market fit are acceptable given local willingness-to-pay and channel dynamics.

Required data sources:

• Market size: national statistics, industry reports (Euromonitor, Statista), trade associations, government data.
• Demand signals: search trends, app/store data, social listening, pilot/test sales.
• Competitors: market share, pricing, distribution channels, feature set (public filings, web scraping, mystery-shop).
• Pricing: competitor price lists, consumer surveys, conjoint analysis results.
• Costs: labor rates, rent, logistics, tariffs, tax regime, local supplier quotes.
• Customer acquisition: channel CPM/CPA benchmarks, agency inputs.
• Regulatory: legal counsel, import/export rules, licensing timelines.
• Internal: product costs, margin targets, historical CAC/LTV in comparable markets.

Analytic methods:

• Market sizing: present top-down (macro / penetration rates) and bottom-up (target segments × expected adoption × price) TAM/SAM/SOM, with assumptions documented.
• Price elasticity and willingness-to-pay: run survey/conjoint analysis; estimate demand curve.
• Unit-economics model: cohort-based LTV, CAC by channel, payback period, contribution margin.
• Competitive mapping: feature/price matrix, Porter’s Five Forces assessment.
• Sensitivity & scenario analysis: Monte Carlo or deterministic one-way/two-way sensitivity on key drivers (penetration, CAC, price, churn). Produce best/likely/worst cases.
• Break-even and runway: calculate months to break-even, capex/opex burn.
• Risk quantification: probability-weighted impact of regulatory, currency, supply risks.

Decision criteria (quantitative + qualitative):

• Quantitative thresholds: NPV (3-year) > 0 at base case, IRR above hurdle, LTV/CAC ≥ target (e.g., ≥3), payback ≤ defined months.
• Operational: regulatory clearance feasible within 12 months, supplier/partner identified, acceptable logistics lead times.
• Strategic fit: alignment with long-term strategy, brand considerations.
• If 3+ criteria fail in base case or worst-case exposes >X% downside, recommend delay or pilot-only.

Deliverables:

• Interactive dashboard (Tableau/Power BI) with scenario toggles, assumptions sheet.
• Financial model (Excel) with bottom-up build, sensitivity tabs.
• One-page executive recommendation + appendix with data sources and method notes.

Estimated timeline & resources (12-week plan for decision readiness):
Weeks 1–2: scoping, stakeholder interviews, finalize hypotheses, data sourcing (1 lead analyst, 0.5 PM).
Weeks 3–5: data collection & cleaning, market-sizing (1 analyst).
Weeks 6–7: pricing studies, competitor analysis, regulator check (1 analyst + external consultant for legal/market intelligence).
Weeks 8–9: build unit-economics & scenario models, sensitivity analysis (1 senior analyst).
Weeks 10–11: dashboards, validation, executive draft (1 analyst, 1 PM).
Week 12: presentation, decision workshop, handoffs (1 analyst, leadership).

Team & budget (estimate): 1 Senior Data Analyst (0.8 FTE), 1 Data Analyst (0.6 FTE), 0.2 PM, external market/legal consultant (ad-hoc, 2–4 weeks), survey/conjoint vendor; estimated cost: $60–120k depending on consultant/survey scope.

Final recommendation format: present base/optimistic/pessimistic outcomes, recommended path (enter, pilot, or defer), required gating criteria to trigger full launch.

Below are grouped clarifying questions to scope the analysis, with notes on which answers would change immediate next steps.

Product / Offering

1. Which product(s) or SKU(s) experienced the drop? (If single SKU, analyze product-specific issues immediately)
2. Were there recent price changes, promotions, returns, or inventory outages? (If pricing/promo change, prioritize revenue impact by price/promotions)

Customer / Segment
3. Which customer segments (region, account tier, industry) showed the decline?
4. Any major customers churned, paused, or delayed orders last week? (If a top customer impacted, escalate to account team)

Channel / Sales Motion
5. Which sales channels dropped (web, direct sales, partners, marketplaces)?
6. Any changes in marketing spend, campaign pauses, or channel outages? (If channel outage or paused campaign, coordinate with ops/marketing immediately)

Timing / External Factors
7. Was there any seasonality, holiday, or one-off timing shift (billing cutoffs, settlement delays)?
8. Any known external events (competitor launches, regulatory, payment processor issues)? (If billing/payment processor issue, cease further analysis and work with finance/ops)

Data / Measurement
9. Which revenue metric are we using (booked, billed, recognized)? Any changes in definitions or ETL last week? (If metric/ETL change, stop and validate data pipelines first)
10. Are there reporting delays or missing data sources for last week? (If missing/incomplete data, delay conclusions until corrected)

Operational / Execution
11. Any recent product outages, site downtime, or fulfillment/logistics issues?
12. Were there changes in discounting, refunds, or invoicing practices?

Priority / Impact
13. What is the magnitude threshold for “drop” (absolute $ or %)? Which stakeholders need the finding and timeline?
14. Do you want a quick hypothesis-driven snapshot (2–4 hours) or a comprehensive root-cause (2–3 days)?

Immediate next-step triggers: answers indicating a metric/ETL change, missing data, payment/billing processor issues, top-customer churn, channel outage, or inventory stockouts require urgent operational coordination before deeper analysis.