Situation: We need to choose between serverless functions (FaaS) and containerized microservices for a new high-throughput ingestion component. I led a cross-functional evaluation to make a data-driven decision.
Evaluation framework (criteria, weighting, rationale)
- Cost (25%): steady-state and burst cost across expected load patterns; includes infra, data egress, and operational staff time.
- Latency (20%): end-to-end ingestion latency and p99; important for near-real-time use cases.
- Cold starts / startup time (10%): affects first-request latency for serverless; less for warm containers.
- Scalability & elasticity (20%): ability to handle spikes, concurrency limits, throttling behavior.
- Operational complexity / SRE burden (15%): deployment, observability, local testing, CI/CD, debugging, and incident recovery.
- Ecosystem fit & feature parity (10%): native integrations (e.g., managed queues, IAM), runtime limits, language/runtime support.
Scoring model
- Score each option 1–5 per criterion, multiply by weighting, sum to 100. Use cost model to convert absolute dollars to normalized score band. Threshold: >60 clear winner; 50–60 requires business trade-off.
Spike plan to gather evidence (2–3 week sprint)
Objectives: quantify cost, latency profile, cold-start behavior, and operational effort under realistic load.
- Prepare baseline
- Implement two minimal ingestion prototypes:
- Serverless version using cloud functions + managed queue (e.g., AWS Lambda + SQS/Kinesis)
- Containerized version using small service in k8s/Fargate + same queue
- Share common business logic and serialization to keep behavior consistent.
- Define test workload
- Use production-like message sizes/distribution, authentication, and downstream writes (mock or staging).
- Scenarios: steady load (X msgs/sec), burst spikes (10x for 5 min), cold-start heavy (low steady then sudden spike), and sustained high load.
- Metrics to collect
- Cost: per-hour infra and per-1000-requests cost; include operational estimates (time to patch, deploy).
- Latency: mean, p95, p99 for enqueue-to-ack and enqueue-to-durable-write.
- Cold-start: first-invocation latency distribution after scale-to-zero or pod eviction.
- Scalability: max concurrent throughput before throttling/errors, time-to-scale.
- Reliability: error rate, retry behavior under backpressure.
- Operational signals: deployment time, local dev feedback loop, debugging effort (qualitative).
- Execution
- Run each scenario 3 times at different times of day; capture resource metrics (CPU, memory, concurrency, queue depths).
- Simulate failures: node restarts, region latency spikes, downstream slowdowns; observe behavior and recovery.
- Analysis & deliverables (end of spike)
- Quantitative report: scored matrix, cost projections for 3 / 12 / 36 months, latency/CDF plots, scaling graphs.
- Qualitative runbook notes: deployment steps, observed operational pain points, monitoring gaps.
- Recommendation: pick option with justification, or hybrid approach (e.g., serverless for bursty lightweight events, containers for heavy/long-running transforms).
- Risk & mitigation: list of risks and remediation (e.g., cold-start mitigation via provisioned concurrency or warmers; autoscaler tuning for k8s).
Stakeholders & timeline
- Team: data engineering (owner), SRE, security, product (requirements).
- Timeline: 2–3 weeks spike + 1 week decision readout.
This approach provides measurable evidence and a clear weighted decision aligned to business priorities.
