Problem framing & inputs
- Goal: compute energy per request and server thermal load from CPU util U, instruction mix M_int/M_vec, memory bandwidth B, and PUE.
- Outputs: Joules/request, steady-state heat (W), and strategies to lower energy while meeting latency SLO.
Quantitative model (simplified)
- Let P_idle = base power, P_dyn = dynamic power proportional to util and activity factors.
- CPU dynamic power scales with frequency f and activity α (vector instructions have higher α_v).
text
P_cpu = P_idle + C_eff * α * f^3
Plain English: dynamic power ≈ capacitance * activity * frequency^3.
text
α = (frac_int * α_int) + (frac_vec * α_vec)
- Memory power from bandwidth:
text
P_mem = P_mem_idle + k_bw * B
- Total IT power and facility:
text
P_IT = P_cpu + P_mem + P_other
P_total = P_IT * PUE
- Energy per request (E_req):
text
E_req = (P_total) / R
where R = requests/sec at observed util and latency.
Thermal impact
- Waste heat ≈ P_IT (W). Use thermal budget per rack: ΔT ∝ P_rack / airflow_rate. Estimate needed cooling capacity = P_total.
Strategies to reduce energy/request
-
Batching
- How: group small requests to amortize fixed costs (P_idle, RPC overhead).
- Benefit: increases throughput R, lowers E_req.
- Trade-off: adds queuing delay; only acceptable if SLO slack allows.
-
DVFS (frequency scaling)
- How: reduce f when CPU-bound slack exists.
- Benefit: power ∝ f^3 — big savings.
- Trade-off: reduces per-request service rate; may worsen tail latency. Use predictive controllers or SLO-aware governors.
-
Autoscaling (right-sizing)
- How: scale core count or nodes to keep util in efficient range (e.g., 40–70%).
- Benefit: avoid low-util waste; aggregate load to fewer servers with higher efficiency.
- Trade-off: scaling latency, possible transient overload, and orchestration cost.
Operational recipe
- Measure α_int/α_vec (profiling), k_bw, C_eff experimentally.
- Build controller: predict arrival λ, choose batch size, DVFS point, and instance count to minimize E_req subject to latency SLO (use queueing model, e.g., M/M/1 or G/G/1 tail estimates).
- Monitor thermal sensors; enforce rack power caps.
Example trade-off scenario
- If SLO = 50 ms median but 95th needs 200 ms, allow batching up to 20 ms; DVFS can drop f by 20% giving ~50% power cut (f^3), combined yield significant E_req reduction while meeting tail SLO with autoscaling for spikes.
Closing
- Model links utilization, instruction mix, bandwidth to power; controller balances batching, DVFS, autoscaling against latency and thermal constraints.
