Data Science & Analytics Topics
Statistical analysis, data analytics, big data technologies, and data visualization. Covers statistical methods, exploratory analysis, and data storytelling.
Analysis to Recommendation and Decision Framing
Ability to move from analysis to a concise, justified recommendation and a pragmatic plan for decision and implementation. Candidates should lead with a clear recommendation or conditional decision, support it with evidence and trade offs, quantify expected business impact, estimate effort and time horizon, and state assumptions and limitations. The skill set includes proposing prioritized action plans and alternative options, anticipating objections, defining monitoring and rollback strategies, translating technical remediation or risk into business terms and measurable success metrics, and tailoring recommendations to stakeholder needs and constraints.
Measurement Design and Analysis
Practical measurement design and analytic techniques for producing reliable metric signals and proving impact. Includes instrumentation and tracking plans, experiment selection and validation, attribution modeling and its limitations, sample size and statistical considerations, identifying confounding variables, and reasoning about correlation versus causation. Also covers tradeoffs in data collection and data quality checks, cohort and segmentation design, baselining and threshold setting, designing dashboards and monitoring cadence, and connecting engineering and telemetry data to business outcomes. Candidates should be able to write clear measurement plans and success criteria, describe experiment and validation approaches, and explain how to operationalize results through reporting and iteration.
Technical Analysis and Methodology
Focuses on the technical depth and concrete analytical methods you use to produce reliable quantitative results. Interviewers look for how you validate assumptions, stress test key inputs, choose modeling techniques, and apply appropriate tools and processes. This includes building and auditing models, performing sensitivity and scenario analysis, data cleaning and transformation, statistical or econometric methods where relevant, and using software such as advanced spreadsheet techniques, scripting languages, or database queries to manipulate data. Candidates should be able to articulate their preferred tools and methods at a level appropriate to the interview and explain trade offs between model complexity and interpretability.
Data Interpretation & Dashboard Literacy
Practice interpreting data visualizations, trend lines, and metric dashboards. Develop ability to identify what's noteworthy (seasonality, anomalies, correlations) vs. normal variation. Think about causation vs. correlation. Practice explaining what a metric trend means in business terms and what actions it might suggest.
Design and Product Analytics
Using quantitative metrics to inform product and design decisions. Covers key user engagement metrics such as conversion rates, task completion, retention, and feature adoption, and how to instrument and interpret these signals using analytics platforms and product dashboards. Explains how quantitative data complements qualitative research, how to identify design problems from metrics, design experiments and metrics for validation, and how to translate findings into design priorities and success criteria.
Real World Experimental Challenges and Solutions
Discuss practical complications in running experiments at scale: user heterogeneity, segment-specific effects, long-term vs. short-term metrics, novelty effects, network effects, and infrastructure constraints. Know techniques for variance reduction (CUPED), segmentation strategies, and how to detect and correct for data quality issues during experiments.
Correlation vs. Causation and Confounding Variables
Recognize that correlation (statistical relationship between variables) doesn't imply causation (direct cause-and-effect relationship). Identify confounding variables that might explain an observed correlation. For example, summer ice cream sales and crime rates both increase but neither causes the other—warm weather is the confounder. Practice identifying lurking variables in business scenarios.
Metrics Analysis and Data Driven Problem Solving
Skills for using quantitative metrics to diagnose and solve business, product, or operational problems across functions. Candidates should be able to identify the key performance indicators relevant to their domain (for example: conversion rate, retention, revenue per user, pipeline velocity, response time, or customer satisfaction), detect anomalies and trends in metrics, formulate and prioritize hypotheses about root causes, design experiments and controlled tests (such as A/B tests) to validate hypotheses, perform cohort and time series analysis, evaluate statistical significance versus practical business impact, and implement and monitor data backed solutions. This also includes instrumentation and data collection best practices, dashboarding and visualization to surface insights, trade off analysis when balancing multiple competing metrics, and communicating findings and recommended changes to cross functional stakeholders.
Interest in Data and Analytics
Evaluates a candidate's genuine curiosity about working with data and their practical comfort with quantitative information, spreadsheets, dashboards, reporting, and analytics tools. Strong responses describe specific hands on experience with data analysis, measurement, reporting, or analytics projects, including concrete examples of metrics tracked, analyses performed, dashboards or reports built, and outcomes or decisions influenced by those insights. Candidates should be able to articulate learning activities and motivations such as courses, personal or open source projects, reading, or tool exploration, and to candidly identify development areas such as structured query language, statistical methods, experiment design, or visualization techniques. The topic also assesses the candidate's ability to explain why data matters for the role and how they use evidence to inform product, process, or business decisions.
Large Dataset Management and Technical Analysis
Develop skills in working efficiently with large datasets: data cleaning and validation, efficient aggregation and manipulation, handling missing data, identifying and managing outliers. Master advanced Excel features or learn SQL for database queries. Practice data quality assessment. Learn efficient workflows that scale with dataset size. Understand data security and privacy considerations.
Advanced Data Analysis and Statistics
Focuses on higher level analytical and statistical techniques for interpreting data and testing hypotheses. Topics include time series analysis, cohort and segmentation analysis, correlation and causation distinctions, descriptive versus inferential statistics, experimental design and hypothesis testing, consideration of sample size and power, detection of confounding variables including Simpson s paradox, and practical interpretation of results and limitations. Emphasizes choosing appropriate methods for given questions and communicating statistical findings clearly.
A/B Test Results Analysis
Learn to analyze results from A/B tests and experiments. Understand key statistics: sample size, statistical significance, confidence intervals, and p-values at a practical level (not deep theory). Practice interpreting test results: 'Is this difference real or just noise?' Learn common mistakes: stopping tests early, p-hacking, and running too many tests simultaneously.
Program Evaluation and Measurement
Assessing whether a program, initiative, or intervention achieves its intended objectives and delivers measurable value, across domains such as training and development, product or feature rollouts, operational process changes, and organizational or culture initiatives. This includes defining success criteria and baseline metrics before implementation, selecting quantitative and qualitative measures during and after delivery, and evaluating impact across multiple levels: immediate reaction, learning or adoption, behavior or usage change, and downstream business results (the logic behind frameworks like the Kirkpatrick model, applied broadly to any program with a change-in-behavior goal, not only training). Candidates should be able to design evaluation plans that include completion and engagement metrics, knowledge or skill assessments, behavior or application measures, retention or usage indicators, and business outcomes. The topic covers leading and lagging indicators, approaches to isolating program impact from confounding factors, simple experimental or quasi-experimental designs when feasible, pragmatic trade offs between ideal and practical measurement, data collection methods and tools, calculating and communicating return on investment (both financial and non-financial), and tailoring reporting to different stakeholders. Examples might include measuring onboarding's effect on time to productivity, a new internal tool's effect on team throughput, a communications campaign's effect on feature adoption, or a process change's effect on error rates. For junior level roles, demonstrate familiarity with measurement choices and their limitations; for senior level roles, include designing robust evaluation frameworks and translating findings into business recommendations.
Critical Thinking About Data Limitations
Understanding bias in data collection, limitations of sample size, context that affects interpretation, and alternative explanations for findings. Being able to discuss what you can and cannot confidently conclude from data. Recognizing when more research is needed.
Metrics, Guardrails, and Evaluation Criteria
Design appropriate success metrics for experiments. Understand primary metrics, secondary metrics, and guardrail metrics. Know how to choose metrics that align with business goals while avoiding unintended consequences.
End To End Case Study: Measurement Frameworks
Practice designing measurement frameworks for new products, features, or business models. Include defining the success criteria, identifying key user segments, setting up tracking, and planning for ongoing analysis.
Data Analysis and Synthesis Frameworks
Expertise in both qualitative analysis (coding, thematic analysis, narrative analysis, grounded theory building) and quantitative analysis (descriptive statistics, inferential statistics, regression, segmentation). Understand data visualization, synthesis across multiple data sources, and generating actionable insights from complex data. Familiarity with analysis software and tools (NVivo, Atlas.ti, SPSS, R, Python for analysis).
Experiment Design and Practical Considerations
Defining metrics to measure (primary and secondary). Estimating sample size and duration needed. Choosing between between-subjects and within-subjects designs. Considering confounding variables and how to control for them. Planning for randomization strategy. Discussing trade-offs between statistical rigor and practical constraints.
Quantitative Analysis & Insights Synthesis
Demonstrate ability to work with quantitative data: analyzing analytics data, conducting cohort analysis, interpreting experiment results, drawing valid conclusions from data. Show how you synthesize findings from both qualitative and quantitative research into insights that inform design. Discuss common analysis pitfalls (correlation vs. causation, confirmation bias, sample bias) and how you avoid them.
Data and Business Outcomes
This topic focuses on converting data analysis and insights into actionable business decisions and measurable outcomes. Candidates should demonstrate the ability to translate trends into business implications, choose appropriate key performance indicators, design and interpret experiments, perform cohort or funnel analysis, reason about causality and data quality, and build dashboards or reports that inform stakeholders. Emphasis should be on storytelling with data, framing recommendations in terms of business levers such as revenue, retention, acquisition cost, and operational efficiency, and explaining instrumentation and measurement approaches that make impact measurable.
Probability and Statistical Inference
Covers fundamental probability theory and statistical inference from first principles to practical applications. Core probability concepts include sample spaces and events, independence, conditional probability, Bayes theorem, expected value, variance, and standard deviation. Reviews common probability distributions such as normal, binomial, Poisson, uniform, and exponential, their parameters, typical use cases, computation of probabilities, and approximation methods. Explains sampling distributions and the Central Limit Theorem and their implications for estimation and confidence intervals. Presents descriptive statistics and data summary measures including mean, median, variance, and standard deviation. Details the hypothesis testing workflow including null and alternative hypotheses, p values, statistical significance, type one and type two errors, power, effect size, and interpretation of results. Reviews commonly used tests and methods and guidance for selection and assumptions checking, including z tests, t tests, chi square tests, analysis of variance, and basic nonparametric alternatives. Emphasizes practical issues such as correlation versus causation, impact of sample size and data quality, assumptions validation, reasoning about rare events and tail risks, and communicating uncertainty. At more advanced levels expect experimental design and interpretation at scale including A B tests, sample size and power calculations, multiple testing and false discovery rate adjustment, and design choices for robust inference in real world systems.
Data Driven Problem Solving and Recommendations
Use data to define business problems, form and test hypotheses, identify root causes, and produce clear, prioritized, evidence based recommendations. Candidates should be able to translate a business question into measurable metrics, choose appropriate analyses, segment and compare cohorts, validate assumptions, quantify expected impact and implementation effort, and surface limitations or data quality issues. Good answers explain the analysis steps, any assumptions made, how results would be validated, and the communication approach to ensure stakeholders can act on the recommendation.
Statistical Analysis and Interpretation
Demonstrates sound statistical methods for analyzing quantitative data and translating findings into actionable business insight. Core skills include descriptive statistics and distributional analysis (percentiles, median, mean, standard deviation, measures of dispersion), outlier detection and handling, correlation assessment, and basic to multivariable regression to control for confounding variables. Also covers hypothesis testing with interpretation of p-values and confidence intervals, evaluation of sample size and statistical power, and awareness of pitfalls such as confounding, multiple comparisons, and violations of model assumptions. Candidates should be able to choose appropriate statistical tests for a given analysis, validate models, and present statistical results clearly to nontechnical stakeholders.
Statistical Thinking and Quantitative Analysis
Covers core quantitative reasoning and applied statistics for user research and product analytics. Topics include descriptive statistics, sampling and representativeness, bias and measurement error, data cleaning and preparation, selection of appropriate statistical tests and models for different types of data, assumptions and limitations of common methods, interpretation of statistical significance, effect sizes and confidence intervals, sample size and power considerations, basics of regression and correlation, experimental design and controlled experiments, analysis of survey data and product metrics, and the ability to translate quantitative findings into actionable product and design recommendations while communicating uncertainty and practical significance.
Quantitative Analysis and Statistics
Designing and interpreting quantitative studies and statistical analyses to generate valid, actionable conclusions. Topics include hypothesis testing, statistical significance versus practical significance, effect sizes, confidence intervals, regression and basic causal inference, power and sample size considerations, handling missing data and outliers, segmentation and weighting, controlled experiment design and analysis, data cleaning, and effective visualization and communication of quantitative results with appropriate attention to uncertainty and limitations.
Baseline and Comparative Analysis
Designing and executing rigorous comparative evaluations that demonstrate whether and why a new method provides meaningful benefit. Topics include selecting simple and strong baselines, ensuring hyperparameter parity, designing ablation studies, choosing appropriate evaluation metrics for the problem, conducting error analysis, running statistical tests and power analysis, correcting for multiple comparisons, avoiding improper data dredging, and building reproducible evaluation pipelines that produce results interpretable for product and research stakeholders.
Analytical Problem Solving and Hypothesis Testing
Assesses the candidate's ability to convert an ambiguous business problem or signal into structured hypotheses, design analyses or experiments, and draw defensible conclusions. Expect discussion of problem framing, prioritizing hypotheses, selecting data sources, defining cohorts and metrics, designing queries or tests, validating assumptions, controlling for confounders, and communicating actionable recommendations. Core skills include critical thinking, data exploration, statistical reasoning, and translating insights into measurable action plans.
Data Driven Recommendations and Impact
Covers the end to end practice of using quantitative and qualitative evidence to identify opportunities, form actionable recommendations, and measure business impact. Topics include problem framing, identifying and instrumenting relevant metrics and key performance indicators, measurement design and diagnostics, experiment design such as A B tests and pilots, and basic causal inference considerations including distinguishing correlation from causation and handling limited or noisy data. Candidates should be able to translate analysis into clear recommendations by quantifying expected impacts and costs, stating key assumptions, presenting trade offs between alternatives, defining success criteria and timelines, and proposing decision rules and go no go criteria. This also covers risk identification and mitigation plans, prioritization frameworks that weigh impact effort and strategic alignment, building dashboards and visualizations to surface signals across HR sales operations and product, communicating concise executive level recommendations with data backed rationale, and designing follow up monitoring to measure adoption and downstream outcomes and iterate on the solution.
Business Impact Measurement and Metrics
Selecting, measuring, and interpreting the metrics that show whether an initiative, product, or program actually delivered value, and using that evidence to guide decisions. Covers headline outcome metrics (revenue decomposition, customer lifetime value, churn and retention, average revenue per user, unit economics and cost per transaction) alongside operational indicators (throughput, quality, reliability) and how to connect the two. Candidates should be able to distinguish leading from lagging indicators, map operational metrics to business outcomes, form and test hypotheses about what is driving a metric, choose an evaluation window, and recommend changes to what gets measured. Also covers the fundamentals of establishing a valid baseline and comparison group (before/after checks, A/B tests, and other quasi-experimental comparisons when a controlled test is not possible), reasoning about whether an observed change is large enough and reliable enough to act on, and ruling out obvious confounding explanations. Includes quick back-of-the-envelope estimation for order-of-magnitude impact, translating technical or operational metrics into business consequences, building a simple health dashboard for a program or initiative, and communicating results (including uncertainty) as a clear, decision-ready narrative for stakeholders. Depth and specific techniques (for example difference-in-differences, regression discontinuity, or survival analysis) should scale to the role: some interviews probe rigorous experimental design, others probe sound judgment using simpler before/after comparisons.
Analytical Background
The candidate's approach to analytical, evidence-based problem solving: how they take an ambiguous question, break it into testable pieces, gather and examine relevant information or data, choose appropriate methods to reach a conclusion, and turn that conclusion into a concrete recommendation or decision. This can show up as quantitative work (statistics, data analysis, experimentation, dashboards) or as qualitative and domain-specific analysis (reviewing logs or incidents, case or contract research, market or process analysis, root-cause investigation). Draw on academic projects, internships, or professional work. Focus on the end-to-end path: how the question or hypothesis was framed, what evidence was examined and with what tools or methods, what trade-offs were considered, and how the resulting insight changed a real decision or outcome.
Data Analysis and Insight Generation
Ability to convert raw data into clear, evidence based business insights and prioritized recommendations. Candidates should demonstrate end to end analytical thinking including data cleaning and validation, exploratory analysis, summary statistics, distributions, aggregations, pivot tables, time series and trend analysis, segmentation and cohort analysis, anomaly detection, and interpretation of relationships between metrics. This topic covers hypothesis generation and validation, basic statistical testing, controlled experiments and split testing, sensitivity and robustness checks, and sense checking results against domain knowledge. It emphasizes connecting metrics to business outcomes, defining success criteria and measurement plans, synthesizing quantitative and qualitative evidence, and prioritizing recommendations based on impact feasibility risk and dependencies. Practical communication skills are assessed including charting dashboards crafting concise narratives and tailoring findings to non technical and technical stakeholders, along with documenting next steps experiments and how outcomes will be measured.
Hypothesis Testing and Inference
Fundamental framework and application of hypothesis testing and statistical inference. Topics include formulating null and alternative hypotheses, understanding Type I and Type II errors, interpreting p values and confidence intervals, selecting and applying common tests such as t tests, chi square tests, analysis of variance, and non parametric alternatives, checking test assumptions, and discussing statistical versus practical significance. Candidates should explain power, significance levels, effect sizes, and common pitfalls such as misinterpreting p values or violating independence assumptions. At more advanced levels, discuss limitations of null hypothesis significance testing, alternatives such as Bayesian inference, and guidance for when different approaches are appropriate.
Statistical Foundations for Experimentation
Core statistical concepts and inference needed to design analyze and interpret experiments. Topics include hypothesis testing p values confidence intervals Type One and Type Two errors the relationship between sample size variability and interval width statistical power minimum detectable effect and effect size versus practical significance. Candidates should be able to choose and explain common statistical tests such as t tests and chi square tests contrast Bayesian and frequentist approaches at a conceptual level and describe variance estimation and variance reduction techniques. The topic covers corrections for multiple comparisons sequential testing and the risks of peeking and p hacking common misconceptions about p values and limitations of inference such as confounding and selection bias. Candidates should also be able to translate statistical findings into clear language for non technical stakeholders and explain uncertainty and limitations.
Metrics and KPI Fundamentals
Core principles and practical fluency for defining, measuring, and interpreting metrics and key performance indicators, applicable across any professional domain. Candidates should be able to select meaningful metrics aligned to business objectives rather than vanity metrics, explain the difference between a metric and a target, and distinguish leading indicators from lagging indicators. Coverage includes decomposing complex outcomes into actionable component metrics, writing precise metric definitions (for example what counts as an active user, a completed case, a qualified lead, or a resolved ticket, depending on the domain), calculating common rate-based metrics such as engagement rate, churn rate, conversion rate, cycle time, or utilization rate, establishing baselines and sensible targets, and interpreting signal versus noise including awareness of statistical variability. Also includes using segmentation and cohort analysis to diagnose metric movements, and recommending two to three meaningful metrics for a hypothetical problem in the candidate's own domain with justification and action plans.
Experiment Design, Analysis, and Causal Methods
Design and analysis of experiments and causal inference methods for when randomization is not possible. Candidates should know strategies to ensure randomization and evaluate experiment quality compute sample size and minimum detectable effect select and interpret primary and guardrail metrics and design appropriate test duration. Analysis skills include hypothesis testing p values confidence intervals effect size estimation variance estimation and variance reduction segmentation and interaction analysis and robust reporting of uncertainty. This topic covers observational and quasi experimental approaches such as propensity score matching difference in differences and regression discontinuity how to reason about confounding and selection bias and when to prefer a quasi experimental approach over a randomized test. Candidates should be able to translate causal conclusions into actionable guidance recommend follow up analyses and triangulate evidence across methods.
Insight Translation and Recommendations
The ability to move beyond reporting numbers to produce clear, actionable business recommendations and narratives. This includes summarizing the problem statement, approach, key findings, model or analysis performance, limitations, and recommended next steps framed as business actions. Candidates should demonstrate how insights map to business metrics and priorities, quantify potential impact and tradeoffs, propose experiments or interventions, and prioritize recommended actions. Effective communication techniques include concise storytelling, appropriate visualizations, translating technical metrics into business terms, anticipating stakeholder questions, and explicitly answering the questions so what and now what. Senior analysts connect root cause analysis to concrete proposals such as feature changes, pricing experiments, targeted support, or investment decisions, and explain risks, data assumptions, and implementation considerations.
Data Analysis and Requirements Translation
Focuses on translating ambiguous business questions into concrete, actionable data analysis plans. Candidates should identify what data is needed to answer the question, define the metrics or KPIs that would settle it, state and validate the assumptions behind those definitions, and lay out the concrete analysis steps or queries that would produce an answer. Strong answers connect analysis choices back to the business decision at stake: what would change stakeholder behavior or strategy, what data quality or data availability issues could undermine the conclusion, and what additional data collection, reporting, or systems changes would be needed to answer the question reliably going forward.
Problem Definition and Hypothesis Formation
Break down ambiguous business questions into specific, answerable analytics problems and define what success looks like. Ask clarifying questions about business context, constraints, stakeholder expectations, and acceptance criteria. Use structured diagnosis and root cause analysis to isolate where a problem occurs by segmenting users, products, time periods, or geographies. Generate multiple testable hypotheses that explain observed outcomes, distinguish correlation from causation, and prioritize hypotheses by likelihood, potential impact, and ease of validation. Frame measurable metrics for each hypothesis and propose high level validation approaches or experiments to confirm or reject the hypotheses.
Experimentation Metrics and Strategy
Designing experiments and selecting appropriate primary, secondary, and guardrail metrics to evaluate hypotheses while protecting long term user value. This includes choosing metrics that reflect both short term signal and long term outcomes, reasoning about metric interactions and potential unintended consequences, and applying statistical considerations such as minimum detectable effect, sample size and power analysis, test duration, and external validity across segments and platforms. Candidates should also discuss experiment risk mitigation, stopping rules, and how to operationalize experiment results into product decisions.
Metrics Selection and Dashboard Storytelling
Focuses on selecting metrics and designing dashboards and reports that directly support stakeholder decision making. Candidates should be able to identify distinct audiences and the specific decisions each audience must make, choose actionable metrics rather than vanity metrics, and balance leading indicators with lagging indicators as well as strategic metrics with operational metrics. This topic covers defining key performance indicators and targets and justifying each metric by the decision it enables, setting data freshness requirements and update cadence, and ensuring instrumentation and data quality to make metrics reliable. It includes dashboard architecture and visual narrative design such as layering from high level summaries to detailed drill down, tailoring views for executives, managers, and operational teams, selecting appropriate visualizations and annotations to guide interpretation, and enabling root cause analysis. Reporting practices are covered, including formatting, distribution channels, and alerting. Governance and metric definition topics include creating a single source of truth, assigning ownership, documenting definitions, and change control. Candidates must also recognize metric interactions and common pitfalls that can make metrics misleading such as aggregation bias, sampling issues, correlation versus causation, and perverse incentives, and propose mitigations. Interview questions typically ask candidates to design metric sets and dashboards for hypothetical scenarios, explain why metrics were chosen based on decisions they support, and describe cadence, distribution, drilling, and governance approaches.
Data Driven Decision Making
Using metrics and analytics to inform operational and strategic decisions. Topics include defining and interpreting operational measures such as throughput cycle time error rates resource utilization cost per unit quality measures and on time delivery, as well as growth and lifecycle metrics across acquisition activation retention and revenue. Emphasis is on building audience segmented dashboards and reports presenting insights to influence stakeholders diagnosing problems through variance analysis and performance analytics identifying bottlenecks measuring campaign effectiveness and guiding resource allocation and investment decisions. Also covers how metric expectations change with seniority and how to shape organizational metric strategy and scorecards to drive accountability.
End To End Data Preprocessing & Exploration
Follow a systematic, tool-agnostic data pipeline before deeper analysis: load the data, check shape and dtypes, identify missing values and duplicates, explore distributions, check for outliers, understand class or category balance where relevant, and summarize key statistics. Document findings and build visualizations that surface relationships in the data. This exploration is the foundation for whatever comes next: feature engineering and model selection for predictive/ML work, or clean aggregations and trustworthy KPIs for dashboards and reporting.
Audience Segmentation and Cohorts
Covers methods for dividing users or consumers into meaningful segments and analyzing their behavior over time using cohort analysis. Candidates should be able to choose segmentation dimensions such as demographics, acquisition channel, product usage, geography, device, or behavioral attributes, and justify those choices for a given business question. They should know how to design cohort analyses to measure retention, churn, lifetime value, and conversion funnels, and how to avoid common pitfalls such as Simpson's Paradox and survivorship bias. This topic also includes deriving behavioral insights to inform personalization, content and product strategy, marketing targeting, and persona development, as well as identifying underserved or high value segments. Expect discussion of relevant metrics, data requirements and quality considerations, approaches to visualization and interpretation, and typical tools and techniques used in analytics and experimentation to validate segment driven hypotheses.
Cohort Analysis and Retention Metrics
Cohort analysis and retention metrics cover methods for grouping users into cohorts by acquisition date, behavior, channel, geography, or other attributes and tracking their behavior over time. Candidates should be comfortable defining cohorts, computing retention curves and retention tables, and calculating key metrics such as day one retention, day seven retention, rolling retention, repeat engagement, churn rates, and cohort lifetime value. Understand how to interpret retention curve shapes and cohort trends to diagnose product market fit, onboarding problems, or channel quality, and how retention drives unit economics and revenue. Practical skills include writing queries in structured query language to segment users and produce cohort tables, plotting retention curves, comparing cohorts across acquisition channels, running cohort based experiments and A B tests, and using cohort insights to prioritize product changes and growth experiments.
Statistical Fundamentals and Exploratory Analysis
Core descriptive and exploratory statistical techniques used to summarize data, detect patterns, and generate testable hypotheses. Covers measures of central tendency and dispersion such as mean median and standard deviation, distributional assumptions, frequency and cross tabulation, visualization for exploration, cohorting and segmentation, identifying biases and data quality issues, and designing exploratory analyses to suggest causal hypotheses. Understand when to apply EDA to prepare data for formal tests and how to translate exploratory findings into confirmatory analyses. Candidates should demonstrate ability to summarize quantitative data, detect anomalies, and propose appropriate follow up hypothesis tests or experiments.
Quantitative Analysis and Metrics Interpretation
Core skills for working with numeric business data: calculating and interpreting key metrics, comparing options numerically, identifying trends and anomalies, performing variance checks, and testing assumptions. Includes reading dashboards and query results, extracting meaningful insights from revenue and operational metrics, segmenting data, identifying outliers, and understanding what metrics indicate about business performance. Candidates should be comfortable stating and justifying assumptions, performing simple break even and cost benefit reasoning, and translating numbers into prioritized actions or follow up analyses. This topic covers cross functional metric types from sales and operations to product and marketing, and emphasizes structured thinking, correct metric definitions, basic descriptive statistics, and how to use data to support recommendations.
Data and Trend Analysis with Pattern Recognition
Analyzing quantitative and qualitative data to identify patterns, trends, correlations, and meaningful insights. Skills assessed include descriptive statistics, time series and trend analysis, visualization and dashboarding, hypothesis generation and testing, identifying seasonality and structural changes, distinguishing signal from noise, and synthesizing findings into clear recommendations. For qualitative inputs candidates should demonstrate coding, theme extraction, categorization, and synthesis of transcripts or survey responses. Emphasis is on choosing appropriate methods, validating patterns, avoiding common pitfalls such as confounding and spurious correlation, and communicating insights effectively to stakeholders.
Data Exploration and Quality Assessment
Investigate a dataset thoroughly before analysis or reporting by profiling its structure, contents, and reliability. Typical steps include examining row counts and data volume, inspecting column data types and sample values, validating date formats and ranges, and identifying missing values, duplicates, outliers, and impossible values. Understand schema and relationships between tables or files, check data freshness and latency, and characterize data completeness and coverage with simple metrics and queries. Document discovered issues, their likely causes and impacts on conclusions, and pragmatic workarounds or transformation strategies to mitigate risk. Use exploratory queries and summary statistics to quantify data quality, note limitations and assumptions, and allocate an appropriate portion of case study time to data assessment before proceeding to modeling or visualization.
Experimental Design and Analysis Pitfalls
Covers the principles of designing credible experiments and the common errors that invalidate results. Topics include defining clear hypotheses and control and treatment groups, randomization strategies, blocking and stratification, sample size and power calculations, valid run length and avoiding early stopping, and handling unequal or missing samples. It also covers analysis level pitfalls such as multiple comparisons and appropriate corrections, selection bias and nonrandom assignment, data quality issues, seasonal and temporal confounds, network effects and interference, and paradoxes such as Simpson paradox. Candidates should be able to critique flawed experiment designs, identify specific failure modes, quantify their impact, and propose concrete mitigations such as pre registration, A and B testing best practices, adjustment methods, intention to treat analysis, A over A checks, cluster randomization, and robustness checks.
Quantitative Research and Analysis
Covers end to end quantitative research methods used to measure and validate product and user behavior hypotheses. Topics include experimental and quasi experimental design, split testing and controlled experiments, metric definition and success criteria, sample size calculation and statistical power, selection of appropriate statistical tests and interpretation of statistical significance and effect sizes, confidence intervals, correlation versus causation, common statistical pitfalls and biases, analytics instrumentation and metric tracking, survey design and quantitative measurement, and data analysis workflows and tools used to analyze large scale user data. Candidates should be able to design experiments, justify metric choices, calculate sample size and duration, analyze results rigorously, and make data driven recommendations.
Data Quality and Bias
Covers both the conceptual and technical aspects of data quality assessment, bias identification, and remediation. Candidates should be able to recognize common sources of bias including selection bias, confirmation bias, measurement bias, and sample limitations, and describe how these biases and methodological limitations affect conclusions. They should be able to document and communicate caveats and limitations clearly and responsibly. On the technical side, candidates should demonstrate techniques for detecting and handling missing values, duplicates, outliers, and inconsistent data types; explain trade offs between filtering, imputing, and transforming data; and discuss how data cleaning choices influence downstream analysis. Additional expected skills include validating cleaned data against expectations, performing sensitivity analyses to show how results change under different data handling decisions, tracking data provenance, and describing reproducible processes for data quality management.
Problem Framing and Data Driven Recommendations
Covers the end to end process of turning ambiguous business questions into clear, actionable solutions using structured thinking and empirical evidence. Includes decomposing complex problems into root causes and manageable components, defining success criteria and key metrics, and selecting appropriate analytical approaches and frameworks. Encompasses extracting, cleaning, and synthesizing raw data into insights, using quantitative and qualitative evidence to generate and evaluate multiple solution options, and applying trade off and prioritization frameworks such as impact and effort. Requires producing evidence backed, prioritized recommendations with implementation considerations, sequencing and monitoring plans, and communicating findings clearly to stakeholders with varying levels of technical knowledge.
Communicating Statistical Results to Business Stakeholders
Translating statistical findings into actionable business language. Explaining confidence, risk, and decision frameworks to non-technical audiences. Presenting trade-offs and uncertainties honestly. Building trust through clear communication.
User Behavior and Analytics
Covers the ability to collect, analyze, and interpret user interaction data using analytics platforms and quantitative techniques. Candidates should demonstrate working knowledge of common analytics tools such as Google Analytics, Mixpanel, and Amplitude to navigate dashboards, create custom reports, define and instrument events, build funnels, and extract cohort and retention data. Core skills include segmentation by activity, demographics, or lifecycle stage, cohort analysis, funnel breakdowns by segment, retention curve interpretation, session and task level metrics, conversion rate analysis, and identifying behavioral patterns. Candidates should be able to pose testable hypotheses about user behavior, validate them using analytics data, surface data quality and instrumentation issues, support split testing, and translate findings into actionable product or design recommendations and prioritized next steps.
Data Storytelling and Insight Communication
Skills for converting quantitative and qualitative analysis into a clear, persuasive narrative that guides stakeholders from findings to action. This includes leading with the headline insight, defining the business question, selecting the most relevant metrics and visual evidence, and structuring a concise story that explains what happened, why it happened, and what the recommended next steps are. Candidates should demonstrate tailoring of language and technical depth for diverse audiences from engineers to product managers to executives, summarizing trade offs and uncertainty in plain language, distinguishing correlation from causation, proposing follow up experiments or investigations, and producing concise executive summaries and status reports with an appropriate cadence. Interviewers evaluate the ability to persuade and align cross functional partners, answer questions about data validity and methodology, synthesize qualitative signals with quantitative results, and adapt presentation format and level of detail to the decision maker.