Assumptions (stated up front)
- Model: 200M-parameter transformer (single-machine, data-parallel)
- Dataset: 5M examples × 3 epochs = 15M samples
- Batch size: 128 samples/GPU
- Throughput: 80 samples/sec per GPU (conservative for a 200M model on NVIDIA A100-class GPU)
- GPU type & price: NVIDIA A100 (on-demand ~$2.50/hr; spot/preemptible ~$1.20/hr)
- Overheads (checkpointing, validation, data I/O): +10% wall-clock
Compute GPU-hours (single GPU)
- Steps = 15,000,000 / 128 ≈ 117,188 steps
- Steps/sec per GPU = 80 / 128 = 0.625 → total seconds ≈ 117,188 / 0.625 ≈ 187,500 s ≈ 52.1 hours
- Add 10% overhead → 57.3 hours ≈ 57.5 GPU-hours (single GPU)
Multi-GPU runtimes
- 4 GPUs (data-parallel) → ~57.5 / 4 ≈ 14.4 wall-clock hours, total 57.5 GPU-hours
Cost estimate
- On-demand (A100 $2.50/hr): 57.5 * $2.50 = $143.75
- Spot (A100 $1.20/hr): 57.5 * $1.20 = $69.00
- Recommend budgeting buffer +30% for retries/experiments/storage: On-demand request ≈ $187; Spot request ≈ $90
How I’d present to Finance
- One-page summary: objective, key assumptions (batch size, throughput, epochs, GPU type), GPU-hours, per-hour rate, subtotal, risk buffer (30%), and total budget request.
- Sensitivity table: show cost range if throughput is ±25% or using 1/4/8 GPUs (so finance sees best/worst cases).
- Operational notes: recommend using spot instances for cost savings with fallback to on-demand; include ancillary costs (storage, egress, monitoring) estimated separately; list experiment plan (dry run on smaller subset to validate throughput before full run).
- Ask: approval for requested amount and permission to use spot with an on-demand fallback.
If you want, I can produce a one-page slide or a short spreadsheet-ready table with the sensitivity breakdown.
