Amazon Entry-Level Data Analyst Interview Preparation Guide

As a candidate primary success metric for onboarding, time to first interaction captures speed but nothing about whether the interaction was meaningful, so it needs to be paired with metrics that catch the cases where speed alone would mislead.

What it captures well

It's simple to measure, hard to game accidentally, and correlates reasonably with a smooth technical onboarding experience (a slow time-to-first-interaction often does flag real friction, like a confusing sign-up form or slow app load).

What it can miss

It says nothing about whether the interaction was the RIGHT one; a user who fast-taps through onboarding without understanding anything, or accidentally triggers an interaction, would look identical in this metric to a user who deliberately engaged with the core feature. A product could 'improve' this metric by making an accidental first tap easier to trigger, with zero real improvement in onboarding quality.

Secondary metrics to pair with it

1. Completion rate of the full onboarding flow (did they finish, not just start interacting).
2. Time to a MEANINGFUL first interaction, defined as a specific core action (not any tap), to distinguish genuine engagement from an accidental touch.
3. Day-1 retention for users segmented by their time-to-first-interaction, to check whether a faster time actually correlates with users coming back.

Two misleading scenarios

1. A UI redesign that makes SOME element more prominent (even an unimportant one) could lower time-to-first-interaction purely because users now tap something sooner, without any real improvement in onboarding comprehension or eventual retention.
2. On a slow network or older device, a legitimately engaged user might take longer to reach their first interaction for purely technical reasons unrelated to product quality, making the metric look worse even though nothing about the ONBOARDING EXPERIENCE itself changed.

Trade-offs and pitfalls

Using time-to-first-interaction as the SOLE primary metric risks a team optimizing for speed of any tap rather than speed of genuine engagement; it's best used as a supporting, diagnostic signal for technical friction, paired with a metric that specifically requires the CORE action, not just any interaction.

A TIMESTAMP column includes a time-of-day component, so a date-only BETWEEN upper bound like '2024-03-31' is treated as midnight at the start of that day, silently excluding every row later that same day; a half-open range using the next day's start avoids the gap.

Structured elaboration

-- Misses late-March-31 rows
SELECT order_id FROM orders WHERE created_at BETWEEN '2024-03-01' AND '2024-03-31';
-- Correct half-open range
SELECT order_id FROM orders WHERE created_at >= '2024-03-01' AND created_at < '2024-04-01';

BETWEEN '2024-03-01' AND '2024-03-31' is shorthand for >= '2024-03-01' AND <= '2024-03-31', and that upper bound, when compared against a TIMESTAMP, is interpreted as 2024-03-31 00:00:00. Any row timestamped later that same day (say, 2024-03-31 23:00:00) is strictly greater than midnight and gets excluded, even though it clearly belongs in "March 2024" by any reasonable interpretation.

Worked example

Given orders at 2024-03-31 23:00:00 and 2024-03-15 10:00:00: the BETWEEN version returns only the March 15 order, silently dropping the March 31 order entirely. The half-open version (< '2024-04-01') correctly returns both orders.

Trade-offs and pitfalls

The half-open-range fix generalizes to any period boundary (a single day, a month, a quarter, a year): always express the upper bound as "the start of the next period, exclusive" rather than "the last moment of this period, inclusive", since the latter requires knowing the column's precision exactly (is it seconds? milliseconds? does '2024-03-31 23:59:59' actually cover a row at '2024-03-31 23:59:59.500'?) while the former never does. This same half-open form is also what makes the query index-friendly, which the question asks for by name: created_at >= '2024-03-01' AND created_at < '2024-04-01' compares the raw created_at column directly against two literal bounds, exactly the shape a B-tree index (the default index structure in most relational engines, storing column values in sorted order) on created_at can use for a plain range scan. A common alternative fix, wrapping the column instead of the bound, e.g. WHERE DATE(created_at) BETWEEN '2024-03-01' AND '2024-03-31' or CAST(created_at AS DATE) = ..., looks similarly correct but applies a function to every row's created_at value before comparing it, which prevents a plain index on created_at from being used at all, since the engine can no longer look up sorted values directly and instead has to compute the expression for every row. Keeping the column bare on one side of a plain comparison is what index-friendly means in practice.

Approach: Triage systematically from orchestration → metadata → data contents → access. Run reproducible checks so you can prove whether it’s scheduling, permissions, or corruption.

1. Check job schedules & runs

• Inspect scheduler (Airflow/cron/DBT): confirm DAG/task run history and timings for the ETL that populates that partition.
• Look for gaps/retries/errors within the time window the dashboard shows missing rows.
• Command/example (Airflow CLI):
  • airflow tasks list <dag_id> --tree
  • airflow tasks failed <dag_id> <task_id> <execution_date>
• Query scheduler tables for last successful run timestamps.

2. Verify partition metadata

• List partitions and their last modified timestamps (example for Hive/BigQuery/Postgres partitioned table).
• Hive:

SHOW PARTITIONS my_table PARTITION(ds);
-- Or
DESCRIBE FORMATTED my_table PARTITION (ds='2025-12-01');

• BigQuery:

SELECT
  partition_id, TIMESTAMP_MILLIS(creation_time) as created
FROM `project.dataset.__PARTITIONS_SUMMARY__`
WHERE partition_id = '20251201';

3. Inspect partition contents with sample SQL

• Row counts and basic integrity:

SELECT COUNT(*) AS cnt, COUNT(DISTINCT id) AS unique_ids
FROM dataset.my_table
WHERE partition_date = '2025-12-01';

• Compare expected vs actual last N rows:

SELECT *
FROM dataset.my_table
WHERE partition_date = '2025-12-01'
ORDER BY event_time DESC
LIMIT 100;

• Check checksum/hash to detect corruption:

SELECT MD5(GROUP_CONCAT(id,',' ORDER BY id)) as checksum
FROM dataset.my_table
WHERE partition_date = '2025-12-01';

(Or use HASH_AGG in platform-specific SQL.)

4. Cross-check source vs target

• Query source system for same partition and compare counts/checksums:

-- source
SELECT COUNT(*) FROM source.events WHERE date = '2025-12-01';
-- target
SELECT COUNT(*) FROM dataset.my_table WHERE partition_date = '2025-12-01';

5. Permissions & access checks

• Confirm service account / user running dashboard has SELECT on that partition/table.
• Example (Postgres):

SELECT has_table_privilege('dashboard_user','my_table','SELECT');

• Attempt a simple SELECT as the dashboard's service account or impersonate.

6. Detect data corruption

• Look for NULLs in key columns, out-of-range values, deserialization failures in ETL logs.
• Query for malformed rows:

SELECT *
FROM dataset.my_table
WHERE important_col IS NULL OR NOT REGEXP_CONTAINS(json_col, r'^\{');

• Check file-level integrity (S3/GCS): missing files, zero-byte files, inconsistent file timestamps.

7. Determine root cause

• Scheduling issue: ETL didn't run or completed before source data available (correlate run times + source availability).
• Permissions issue: ETL created data but dashboard user cannot read partition (dashboard errors show permission denied).
• Data corruption: counts/checksums mismatch, malformed rows, ETL errors during processing.

8. Remediation & monitoring

• Re-run ETL for affected partition; validate counts/checksum.
• Add alerts: missing partition count, zero-row partitions, ETL failures.
• Add a dashboard health query that compares source vs target daily and surface anomalies.

What to report: timeline of events, diffs (counts/checksums), scheduler logs, SQL results, and the remediation performed (or recommended). This gives stakeholders a reproducible explanation and fixes the immediate missing-rows issue.

Direct answer. NOT IN returns NULL (not TRUE or FALSE) for every comparison once the subquery's result set contains even a single NULL, and NULL propagates through NOT IN's implicit ANDed comparisons in a way that makes the whole condition unsatisfiable, so the query returns zero rows even though "obviously" some should qualify.

Structured elaboration. x NOT IN (a, b, NULL) is shorthand for x <> a AND x <> b AND x <> NULL. The last comparison, x <> NULL, evaluates to NULL (unknown) for every value of x, not FALSE, and ANDing anything with NULL can never produce TRUE. So the entire NOT IN expression becomes NULL for every row, and a WHERE clause only keeps rows where the condition is TRUE, which is now none of them. NOT EXISTS and LEFT JOIN ... IS NULL don't have this trap because they check for the presence of a matching ROW, not equality against a list that might contain NULL.

Worked example. customers(1),(2),(3). orders(order_id=100, customer_id=1), (order_id=101, customer_id=NULL) (an order that failed to record its customer).

-- broken: returns ZERO rows, because orders.customer_id contains a NULL
SELECT customer_id FROM customers
WHERE customer_id NOT IN (SELECT customer_id FROM orders);

-- correct: NOT EXISTS ignores the NULL row entirely and returns the right answer
SELECT c.customer_id FROM customers c
WHERE NOT EXISTS (
  SELECT 1 FROM orders o WHERE o.customer_id = c.customer_id
)
ORDER BY c.customer_id;

The first query returns no rows at all. The second correctly returns (2,) and (3,), the customers who never placed an order.

Trade-offs and pitfalls. The safest default is NOT EXISTS: it's immune to NULLs in the subquery by construction and its correlated form generally lets the optimizer use an index on the inner table's join column efficiently, including on large, partitioned tables. LEFT JOIN ... WHERE right.key IS NULL is logically equivalent and often just as fast, and some engineers prefer its readability, but it's easy to get wrong if additional filters accidentally end up in the WHERE clause instead of the ON clause (which reintroduces the WHERE-vs-ON join-demotion bug). NOT IN is only safe when you can prove, structurally, that the subquery's column is NEVER NULL, which is a fragile guarantee to rely on in a query that will be copy-pasted and modified later, so most style guides ban NOT IN against a subquery outright.

Direct answer

The pyramid principle says to structure a data narrative top-down: state your main conclusion first, then the two or three arguments that support it, then the evidence beneath each argument, rather than building up to the conclusion the way you actually did the analysis. The closely related SCQA shape (Situation, Complication, Question, Answer) is a way to construct that top line: state the shared context, name what changed or went wrong, pose the question that creates, then answer it, with the Answer being the same headline the pyramid puts first.

Structured elaboration

1. Why top-down beats bottom-up for a busy decision-maker.
Analysis is naturally built bottom-up: you gather data, run tests, notice patterns, and arrive at a conclusion at the end of that process. But a decision-maker reading or hearing the result does not have time to retrace that path and does not need to; they need the conclusion first so they can decide how much of the supporting detail they actually want. Presenting bottom-up (data first, conclusion last) forces every reader to sit through the full derivation before learning the point, and it means anyone who stops reading after the first paragraph, which is common in a busy inbox or meeting, misses the actual finding.

2. The pyramid's three layers.
At the top: a single governing conclusion or recommendation, stated as a complete sentence, not a topic label ('Churn is a problem' is a topic; 'Churn among enterprise accounts rose 4 points last quarter and threatens renewal revenue, we recommend X' is a conclusion). In the middle: two to four supporting arguments, each one a reason the top conclusion is true, ideally grouped so they are mutually exclusive and collectively exhaustive of the case you're making, not an arbitrary list. At the base: the specific evidence, numbers, and analysis behind each supporting argument, which is where the detail-oriented reader or a skeptical stakeholder can drill in.

3. The SCQA framing for arriving at that top line.
Situation: state the shared, uncontested context ("Enterprise renewal rates have been stable around 92% for six quarters"). Complication: name what changed or what tension that creates ("This quarter renewal dropped to 88%, concentrated in accounts onboarded in the last year"). Question: the natural question the complication raises ("What's driving the drop, and can we intervene before renewal season peaks?"). Answer: your actual conclusion and recommendation, which becomes the pyramid's top line. SCQA is really a technique for constructing a compelling, honest top line; the pyramid is what you do with that top line once you have it.

4. Restructuring a bottom-up finding into this shape.
Take the order you actually worked in (data pull, exploratory checks, a few dead ends, the eventual pattern, the conclusion) and literally invert it for the write-up: conclusion first, then the two or three strongest reasons, then evidence for each reason. The dead ends and exploratory detours from your real process almost never belong in the final artifact at all; they belong in an appendix or nowhere, because the pyramid is a communication structure, not a lab notebook.

Worked example

An analyst investigates a support-ticket increase by pulling ticket volume by category, checking for a recent product release, cross-referencing with a signup cohort analysis, and eventually finding the pattern. Built bottom-up, the write-up would read: "We pulled ticket data for the last 90 days... we checked release notes... we then looked at signups by cohort... and found that tickets from users onboarded after the March release are 3x more likely to file a billing-related ticket." Restructured with the pyramid/SCQA shape: Situation/Answer-first: "Billing-related support tickets are up 40% quarter over quarter, driven almost entirely by users onboarded after the March release; we recommend a fix to the new billing confirmation step before the next release." Supporting arguments: (1) users onboarded after March file billing tickets at 3x the rate of earlier cohorts, (2) the March release changed the billing confirmation flow, (3) no other cohort or category shows a comparable increase, ruling out a general support-quality issue. Evidence for each argument follows beneath, in the same order, for the reader who wants to verify the claim rather than just act on it.

Trade-offs and pitfalls

• The most common mistake is writing the top line as a topic ("Q3 billing tickets") instead of a complete, decision-relevant sentence with a conclusion in it; a topic doesn't tell the reader anything they can act on.
• Forcing every supporting argument to be truly independent (mutually exclusive) takes real editing; a first draft often has 4-5 overlapping points that should collapse into 2-3 distinct ones.
• The pyramid structure is not a license to omit genuine uncertainty or counter-evidence; the top line should still be honest about confidence and limitations, not just punchy.
• Over-applying the framework to a finding that genuinely has no single clear conclusion (a mixed or inconclusive result) produces a false sense of clarity; in that case the honest top line states the ambiguity itself as the headline, rather than forcing a decisive-sounding conclusion the evidence doesn't support.

Direct answer
Choose histogram bin width using a data-driven rule like the Freedman-Diaconis rule (width = 2 * IQR / n^(1/3)) or Scott's rule (width = 3.49 * std / n^(1/3)), and to make histograms comparable ACROSS groups, compute the bin width (and bin edges) once from the pooled/combined data rather than independently per group, since independently-computed widths will differ purely because of each group's own sample size, even when the underlying distributions are identical.

Structured elaboration

1. Freedman-Diaconis: uses the interquartile range (robust to outliers) and sample size; generally preferred over Scott's rule when the data has heavy tails or outliers, since it doesn't assume a roughly normal shape.
2. Scott's rule: uses the standard deviation and sample size; simpler and works well for roughly normal data, but is more sensitive to outliers than Freedman-Diaconis since standard deviation itself is outlier-sensitive.
3. The cross-group comparability pitfall (verified by execution): computing Freedman-Diaconis bin width independently for two samples of the SAME underlying normal distribution (mean 50, std 10) but different sizes (n=500 vs. n=5000) produced widths of 3.369 and 1.542 respectively, and correspondingly 22 versus 47 bins, purely because of the n^(1/3) term in the formula, not because the underlying distributions actually differ.
4. Fix: compute the bin width from the POOLED data across all groups being compared (or fix explicit shared bin edges), so every group's histogram uses the identical width; in the same executed example, the pooled Freedman-Diaconis width came out to 1.493 (49 shared bins), used consistently for every group's histogram.
5. Pitfalls when distributions differ in scale or sample size: a small-sample group with a genuinely different scale can still end up mis-binned if you use pooled bin width blindly; sanity-check that the shared bin width still resolves the smaller group's shape reasonably (not too few bins) before finalizing.

Worked example (executed)

import numpy as np
def freedman_diaconis_bin_width(x):
    q75, q25 = np.percentile(x, [75, 25])
    iqr = q75 - q25
    return 2 * iqr / (len(x) ** (1/3))

np.random.seed(42)
group_a = np.random.normal(50, 10, 500)
group_b = np.random.normal(50, 10, 5000)
combined = np.concatenate([group_a, group_b])
shared_width = freedman_diaconis_bin_width(combined)  # 1.493

Independently: group A width 3.369 (22 bins), group B width 1.542 (47 bins), for the SAME underlying distribution: comparing the two histograms side by side at these independently-chosen widths would visually suggest they differ in granularity for no real reason. Using the shared pooled width of 1.493 for both fixes the comparison.

Trade-offs and pitfalls
Both rules assume a reasonably continuous, unimodal-ish distribution; for a genuinely multimodal or highly skewed variable, a data-driven bin-width rule can still produce a misleading histogram, and a log-transform or a different chart (like a violin plot) may serve better than tuning bin width alone.

The same "latest order per user" result can be built with a correlated subquery, a derived-table join against a pre-aggregated MAX(created_at), or a ROW_NUMBER() window function; they differ in how much per-row work the engine has to redo and in how they handle ties. On a large orders table, the ROW_NUMBER() form is usually the best default because it computes ranks in one ordered pass and lets you break ties deterministically, but the correlated form is still the right call when the predicate is narrow and well-indexed.

Approach 1: correlated subquery

SELECT o.*
FROM orders o
WHERE o.created_at = (
  SELECT MAX(o2.created_at)
  FROM orders o2
  WHERE o2.user_id = o.user_id
);

Approach 2: derived-table join on MAX(created_at)

SELECT o.*
FROM orders o
JOIN (
  SELECT user_id, MAX(created_at) AS max_created
  FROM orders
  GROUP BY user_id
) m ON o.user_id = m.user_id AND o.created_at = m.max_created;

Approach 3: ROW_NUMBER() window function

SELECT order_id, user_id, amount, created_at
FROM (
  SELECT o.*,
    ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY created_at DESC, order_id DESC) AS rn
  FROM orders o
) t
WHERE rn = 1;

Key points

• Approaches 1 and 2 both compare against MAX(created_at), so if two orders for the same user share the exact same timestamp, both are returned; neither query has a tiebreaker.
• Approach 3 adds order_id DESC as an explicit tiebreaker, so it always returns exactly one row per user even when created_at collides.
• ROW_NUMBER() can't be filtered directly in a WHERE clause (window functions evaluate after WHERE), which is why it has to be wrapped in a subquery, as shown, or filtered with QUALIFY in engines that support it (Snowflake, BigQuery, Databricks; not PostgreSQL or MySQL).

Complexity

The correlated subquery is, logically, one MAX lookup per outer row; whether that's cheap depends entirely on the plan. With an index on (user_id, created_at DESC), a smart optimizer can turn this into an index-descent per user rather than a full re-scan, but many optimizers instead execute it as a nested loop that repeats the inner scan once per outer row, which is the pattern to watch for in EXPLAIN. The derived-table join computes the per-user MAX once, in a single grouped pass over the table (an index or hash aggregate handles this in roughly linear time), then joins that back, which is a bounded amount of work regardless of how many orders a user has. The ROW_NUMBER() form does one sort (or an index-satisfied ordering) per partition, which is generally the same order of work as the grouped aggregate but produces the tiebreaker and the ordering guarantee in the same pass.

Worked example

With five orders across three users, including two orders for user 101 that share the exact same created_at (a same-second data-entry collision), running all three queries against this data shows the difference directly: approaches 1 and 2 both return two rows for user 101 (the tie), while approach 3 with ORDER BY created_at DESC, order_id DESC returns exactly one deterministic row per user, including user 101. (Verified by executing all three queries against SQLite 3.51 with this exact tie case.)

Trade-offs and pitfalls

Genuinely reach for the correlated form when the predicate is highly selective and backed by the right composite index (e.g., you've already filtered down to a handful of users), because a per-row indexed lookup can be cheaper than materializing a full per-user aggregate you don't need elsewhere; some optimizers (PostgreSQL, SQL Server) can automatically rewrite a correlated subquery like this into a semi-join or hash join, but that isn't guaranteed across engines or even across query shapes on the same engine, so don't assume it happened silently. Always check: if EXPLAIN shows a nested loop re-scanning orders once per outer row instead of a hash or merge join, that's the concrete signal to rewrite into the derived-table or ROW_NUMBER() form. The most common bug across all three forms isn't performance, it's silently dropped or duplicated rows from an unhandled tie; if the business logic genuinely needs "the" single latest order, the ROW_NUMBER() form with an explicit tiebreaker is the only one of the three that guarantees it.

Requirements (clarify): fast queries for recent ranges (last 7/90 days), efficient deletes for retention (e.g., keep 1 year), low cost scanning on cloud DWs, and support ad-hoc analysis across history.

High-level strategy

• Partition by event_date (DATE or TIMESTAMP truncated to date).
• Use daily partitions for recent data (last 90–180 days) and monthly partitions for older history (partitioned-by-range or use partitioned table with ingestion-time + virtual month column). This hybrid keeps recent partitions small for fast scans and reduces partition count for old data.

Partition size guidance

• Aim for partitions that contain between ~100MB–10GB compressed (rough guideline), or ~tens to hundreds of millions of rows depending on row width. If daily partitions are too small, switch to weekly.

Partition pruning and query patterns

• Ensure queries include event_date filters (or cast) so the engine can prune partitions. Educate analysts to use WHERE event_date BETWEEN (...) to avoid full-table scans.
• For last 7/90 days the engine will touch only recent daily partitions — very fast.

Indexing / clustering

• Use a clustered/sorted key on (event_date, user_id, event_type) for common predicates and to improve localities (reduces scanned bytes for filters and aggregations).
• On Redshift, use sort keys and zone maps; on Snowflake use clustering keys; on BigQuery use partitioning + clustering.
• Maintain statistics and re-cluster/re-sort periodically for performance.

Retention & fast deletes

• Implement retention by dropping partitions: in systems that support partition DROP/MERGE (BigQuery, Snowflake via time-travel considerations, Redshift Spectrum/managed) remove whole partitions instead of DELETE WHERE — this is cheap.
• For systems that don't support fast physical partition drop, use partition-exchange staging tables or mark-for-delete + background compaction.

Cloud cost considerations

• BigQuery: Partitioning + clustering minimizes bytes scanned (lower cost). Dropping partitions is instant; avoid time-travel retention too long (storage costs).
• Redshift: Use date as sort key and manage vacuum/ANALYZE; UNLOAD older partitions to S3 or use Spectrum for cold data to cut cluster storage costs.
• Snowflake: Partitioning is logical; clustering reduces compute for queries but increases maintenance cost (RECLUSTER). Dropping micro-partitions is automatic when data removed; consider data retention and time-travel (failsafe) costs.

Operational notes

• Automate partition lifecycle (create, expire, drop) and monitor partition sizes.
• Provide query templates and dashboards for analysts to encourage partition-pruning filters.
• Test with representative queries and adjust daily vs weekly vs monthly split based on observed partition sizes and latency.

Extract each required key explicitly and test for its absence with a NULL check on the extraction result, since a naive containment check on the raw JSON text can test for the wrong thing entirely (a VALUE appearing anywhere, rather than a specific KEY existing).

Approach (verified by execution, after fixing a real bug)

```sql
SELECT event_id,
json_extract(payload, '$.user_id') IS NULL AS missing_user_id,
json_extract(payload, '$.order_value') IS NULL AS missing_order_value,
(json_extract(payload, '$.order_value') IS NOT NULL
AND TRY_CAST(payload->>'order_value' AS DOUBLE) IS NULL) AS order_value_not_numeric
FROM events;
```
A common early mistake here is reaching for a generic "does this JSON contain this string" function; that tests whether a given VALUE appears anywhere in the document, not whether a specific KEY exists, and produces wrong results (in one verified test, it incorrectly flagged every row as missing a key that most of them actually had). Extracting the specific path with `json_extract` (or the engine's equivalent `->`/`->>` operators) and testing THAT result for NULL is the correct, unambiguous way to check key presence.

Worked example (verified by execution)

Given three rows, one with both required keys present and a numeric `order_value`, one missing `order_value` entirely, and one with `order_value` present but set to the string `"abc"`: the query correctly reports the second row as `missing_order_value = true` and the third as `order_value_not_numeric = true`, with neither incorrectly flagged for the OTHER condition.

Trade-offs and pitfalls

Distinguish "the key is genuinely absent" from "the key is present but its value is NULL" if that distinction matters to your producers; `json_extract` returning NULL can mean either, depending on the payload and the engine, so if the difference matters, check key presence via the engine's key-listing function (like `json_keys`) rather than relying solely on extraction returning NULL. This validation query is the concrete counterpart to a broader schema-inference pass (discovering what keys exist and their typical types across the whole dataset); once you know the expected shape, THIS is the query that enforces it going forward.

Direct answer

Constructive skepticism means treating a colleague's analysis as something to verify before it reaches people who will make a decision on it, not something to trust blindly or attack. What keeps it collaborative rather than adversarial is that the questions are aimed at the work, in service of the same goal the analyst has (a correct, defensible result), not aimed at their competence.

Structured elaboration

What to actually check

• Data provenance and cleaning: were there filters, joins, or exclusions applied that could bias the result?
• Assumptions and their sensitivity: does the conclusion hold under a slightly different time window, cohort definition, or parameter choice?
• Confounders and alternative explanations: could something else, like seasonality or a cohort mix shift, explain the pattern as well as the stated cause?
• Reproducibility: can someone else rerun the analysis and get the same numbers, and are the metric definitions written down anywhere?

How to raise it without it turning into a credibility fight

The framing matters more than the content. Raise it privately and early, before it's in front of stakeholders, not during the stakeholder meeting itself. Ask it as a question about the data or method ('what date range did you use for this cohort?'), not as a verdict about the person or their competence. Where possible, offer to help verify rather than only pointing out a gap; that keeps the interaction collaborative instead of adversarial. The deeper mechanics of de-escalating a tense disagreement are their own skill; the key move here is simply getting the framing and the timing right before it escalates into one.

Worked example

A colleague's dashboard shows a conversion metric trending in a direction that conflicts with what other data would suggest. Before it goes in front of stakeholders, a private message asks what date range and cohort definition were used, and whether a known seasonal effect was accounted for. It turns out the shift came from a change in how the cohort was defined that week, not a real change in behavior. The colleague fixes the definition before the meeting, and the stakeholder presentation goes out correct, with no public correction needed.

Trade-offs and pitfalls

• Raising a concern only after it's already in front of stakeholders turns a technical question into a public correction, which is exactly where it tends to become a credibility fight.
• Flagging every minor doubt in a public forum regardless of the stakes wears down trust and slows the team; reserve escalation for cases where a private check didn't resolve it and the decision at stake actually matters.
• Being right about a caught issue is not the same as handling it well; how the concern was raised often matters more to the relationship than the fact that it was correct.