I start by decomposing the feature into clear workstreams and estimating each: data discovery & ingestion, data cleaning & labeling, EDA/feature engineering, modeling & experiments, infra/integration, testing/validation, and monitoring. For each stream I list concrete tasks (e.g., “join customer/transactions, handle missing timestamps, write 2 label heuristics, build baseline model, CI/CD pipeline”) and estimate in story-points or days based on historical baselines (past similar projects: average ETL = 3d, labeling = 5d, model prototyping = 7d).
Techniques I use:
- Time-boxed spikes: allocate a short discovery spike (1–3 days) to quantify unknowns in data volume, label availability, class imbalance and easy wins.
- Historical baselines: adjust estimates by comparing to 2–3 previous projects with similar scope and noting variance.
- Probabilistic estimates: give P50/P90 ranges (e.g., 2–4w P50, 4–7w P90) to reflect experimental uncertainty.
- Explicit buffers: add a contingency (typically 20–30% on the total) for data quality surprises, slow labeling, or model iteration.
- Risk-based padding: add extra time to tasks with high unknowns (data cleaning +50% if source is unvalidated).
- Parallelization: identify independent tasks to run concurrently (labeling and feature engineering can often overlap).
I track assumptions (label quality, compute availability), update estimates after the discovery spike, and communicate ranges to stakeholders. Example: for a recommendation-score feature I’d plan: 3d discovery, 5d ingestion/cleaning, 4d labeling heuristics, 10d experiments, 5d infra/integration, 3d validation + 20% buffer → total P50 ~5 weeks, P90 ~7 weeks.
