Advanced SQL: Window Functions, CTEs, and Subqueries Questions

Clarify first
I would ask whether the product manager wants a time-based window or a calendar window. A 168-hour window means the last 7 times 24 hours from each event timestamp. A calendar window means the current day plus the previous 6 calendar days in the store's business timezone. Those are not the same at midnight boundaries.

Implementation choices

• If they want 168 hours, use a timestamp window.
• If they want calendar days, aggregate by date first, then roll up daily counts.

-- Calendar-day version
WITH daily AS (
  SELECT
    store_id,
    CAST(order_ts AS date) AS order_date,
    COUNT(*) AS orders
  FROM orders
  GROUP BY store_id, CAST(order_ts AS date)
)
SELECT
  store_id,
  order_date,
  SUM(orders) OVER (
    PARTITION BY store_id
    ORDER BY order_date
    ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
  ) AS rolling_7_day_orders
FROM daily;

Boundary example
An order at 2025-01-08 00:05 UTC is inside the calendar-day window for Jan 8 through Jan 2, but in a strict 168-hour window it depends on the exact timestamp cutoff. I would document that choice in the metric definition so analysts get the same answer.

Approach
I would define sessions per canonical user ID, not per device, because the requirement says the same user can act from multiple devices. First dedupe duplicate events, then sort by event timestamp, and start a new session when the gap from the previous event is more than 30 minutes.

WITH deduped AS (
  SELECT *
  FROM (
    SELECT
      e.*,
      ROW_NUMBER() OVER (
        PARTITION BY event_id
        ORDER BY ingest_ts DESC
      ) AS rn
    FROM clickstream e
  ) x
  WHERE rn = 1
), ordered AS (
  SELECT
    user_id,
    event_ts,
    CASE
      WHEN LAG(event_ts) OVER (
        PARTITION BY user_id
        ORDER BY event_ts
      ) IS NULL
      THEN 1
      WHEN event_ts - LAG(event_ts) OVER (
        PARTITION BY user_id
        ORDER BY event_ts
      ) > INTERVAL '30 minutes'
      THEN 1
      ELSE 0
    END AS new_session
  FROM deduped
)
SELECT
  user_id,
  event_ts,
  SUM(new_session) OVER (
    PARTITION BY user_id
    ORDER BY event_ts
  ) AS session_id
FROM ordered;

Session metrics
Group by user_id and session_id to get event count, start time, end time, and duration.

Guardrails
Out-of-order events are handled by sorting on event_ts. Duplicates are removed first so they do not create fake activity or split a session.

What I would look at
Spill to disk usually means the engine ran out of memory for a large sort or window step. Since the query has several window functions, I would inspect whether they share the same PARTITION BY and ORDER BY clauses. If not, the engine may sort the same billion-row fact table multiple times.

Tuning steps

• Keep only the columns needed before the window step.
• Filter as early as possible, especially on date ranges.
• Align window definitions so the engine can reuse one sort.
• Pre-aggregate to the lowest grain that still preserves the result.
• Check table clustering or partitioning on the hottest filters, such as day.

Example
If one window is over (customer_id, day) and another is over (customer_id, country, day), I would ask whether the business result really needs both raw row-level windows. If a daily summary is enough, I would stage one row per customer, country, and day first, then run the windows on that smaller set.

Why this helps
You are not changing the answer, just reducing the amount of data that must be sorted and carried through each window calculation. That is usually the fastest way to stop spills.

One deterministic row per customer
Use ROW_NUMBER(), which assigns 1 to a single row after you specify a full tie-break order. ROW_NUMBER() is a window function, meaning it ranks rows inside each customer group without collapsing them.

WITH ranked AS (
  SELECT
    o.*,
    ROW_NUMBER() OVER (
      PARTITION BY customer_id
      ORDER BY order_ts DESC, order_id DESC
    ) AS rn
  FROM orders o
)
SELECT *
FROM ranked
WHERE rn = 1;

The timestamp decides recency, and order_id DESC makes ties deterministic.

If business later wants every row tied for the latest timestamp
Switch to RANK() and filter on 1.

WITH ranked AS (
  SELECT
    o.*,
    RANK() OVER (
      PARTITION BY customer_id
      ORDER BY order_ts DESC
    ) AS rnk
  FROM orders o
)
SELECT *
FROM ranked
WHERE rnk = 1;

Example
If customer 7 has rows at 10:00, 10:00, and 09:30, the first query returns one row. The second returns both 10:00 rows.

Approach
First aggregate at the raw line-item grain, then rank the product totals within each region. A line item is one row of product revenue, so summing those rows avoids double counting order-level facts. For the tie rule, use DENSE_RANK() or RANK() on the final product totals, then keep ranks <= 3.

WITH product_revenue AS (
  SELECT
    region,
    product_id,
    SUM(line_revenue) AS revenue
  FROM raw_line_items
  GROUP BY region, product_id
), ranked AS (
  SELECT
    region,
    product_id,
    revenue,
    DENSE_RANK() OVER (
      PARTITION BY region
      ORDER BY revenue DESC
    ) AS revenue_rank
  FROM product_revenue
)
SELECT region, product_id, revenue
FROM ranked
WHERE revenue_rank <= 3;

Why this is correct

• The aggregation happens before ranking, so each product appears once per region.
• DENSE_RANK() includes every product tied at the cutoff.
• If you also need order-level facts, aggregate them in a separate CTE at order grain and join the finished summaries, not the raw tables.

Worked example
If a region has revenues 100, 90, 90, 80, the ranks are 1, 2, 2, 3. Filtering <= 3 returns all four products, which matches the tie requirement.