Requirements & approach:- Deliver a repeatable, parameterized weekly automation that slices conversions by campaign and country, refreshes every Monday, and is easy for business to change.- Use: parameterized SQL (view / stored proc), scheduled ETL/refresh (SQL Agent / Airflow), and parameter-driven Power BI or Tableau report with alerts.Template design (SQL + dataset):1. SQL view/stored procedure that accepts date range, campaign list, country list, and granularity.2. Store parameters in a config table or pass via scheduler.3. Materialize a lightweight aggregate table (weekly partitions) for performance.Example parameterized SQL (Postgres / T-SQL style):sql
-- params: @start_date, @end_date, @campaigns_csv, @countries_csv
CREATE OR REPLACE FUNCTION reporting.weekly_conversions(
start_date date, end_date date, campaigns text, countries text
) RETURNS TABLE(campaign text, country text, conversions int, week_start date) AS $$
BEGIN
RETURN QUERY
SELECT c.campaign, t.country, COUNT(*) AS conversions, date_trunc('week', ev.event_date)::date AS week_start
FROM events ev
JOIN campaigns c ON ev.campaign_id = c.id
JOIN territories t ON ev.country_code = t.code
WHERE ev.event_date BETWEEN start_date AND end_date
AND (c.name = ANY(string_to_array(campaigns, ',')) OR campaigns = '')
AND (t.country = ANY(string_to_array(countries, ',')) OR countries = '')
GROUP BY c.campaign, t.country, week_start;
END; $$ LANGUAGE plpgsql;
Power BI / Tableau integration:- Power BI: create a parameterized Query (M) that calls the function via native query; expose parameters (StartDate, EndDate, Campaigns, Countries) in the dataset. Use Incremental Refresh on the aggregate table to speed up loads.- Tableau: create a Custom SQL data source that calls the stored proc; publish as a data source with parameters exposed to workbook.Scheduling & parameter feed:- Use a scheduler (Airflow, Azure Data Factory, or SQL Agent) to: - Compute last Monday / target week and write parameters to a config table or invoke proc with parameters. - Run the aggregation/materialization job. - Trigger dataset refresh in Power BI (Power BI REST API) or Tableau Server extract refresh (TabCmd / REST).- Example flow: Monday 02:00 ET -> run SQL aggregation for prior week -> refresh Power BI dataset -> send report/subscription by 03:30 ET.Monitoring & data freshness:- Add data-quality checks as separate SQL tasks: row counts, null rates, expected conversions delta vs rolling average; write results to monitoring table.- Create a lightweight monitoring dashboard (Power BI/Tableau) or use alerting: - If no rows returned, or conversion delta > threshold, raise an incident (email/Slack) via scheduler. - Log job status, runtime, rows processed; keep last successful run timestamp.- Implement retries with exponential backoff and alert after N failures.Operational notes / best practices:- Parameter defaults: empty campaign/country means “all”.- Use service account with least privilege.- Keep aggregations partitioned by week for fast incremental refresh.- Version control SQL and report templates; document parameters and ownership.- Provide a simple UI (Power App / internal form) if business non-technical users need to change weekly parameters.This design ensures repeatability, parameterization, measurable freshness, and clear alerting so Monday reports are automated and reliable.