Serverless and Function-as-a-Service Architecture Questions

Direct answer

First confirm the spikes really are cold starts: compare p99 (the 99th-percentile latency, the value only 1% of requests exceed) for invocations whose logs carry an Init Duration against those that don't. If they are, three practical strategies, from cheapest to most expensive: (1) make initialization cheaper (smaller package, fewer and lazier imports, more memory for more CPU); (2) restore from a snapshot instead of initializing (AWS Lambda SnapStart); (3) pre-initialize environments so users never hit a cold one (provisioned concurrency, or minimum instances on other platforms). Do (1) always, then pick (2) or (3) based on runtime support and how strict the latency target is.

Why cold starts land exactly in p99

AWS reports cold starts are typically under 1% of invocations. p99 is the boundary of the slowest 1%. If 0.5% of requests are cold, the slowest 1% is half cold and half slow-warm, and p99 sits at the edge of the warm tail. If 1.5% are cold (low traffic, frequent deploys, bursts that force new environments), the p99 value itself is a cold request. A modest rise in cold-start rate can therefore move p99 from roughly "slow warm request" to "full cold start". This is why the fix list starts with cutting the cold-start cost, and why reducing how often they happen also counts.

Strategy 1: make initialization cheaper

Mechanism. Cold-start time is roughly code download + runtime start + your init code. Shrinking any term shrinks the spike.

• Import only what you use (a single SDK service client rather than the whole SDK); remove unused dependencies, test fixtures and docs from the package.
• Lazy-load heavy objects needed only on some code paths.
• Raise memory: Lambda allocates CPU in proportion to memory (1,769 MB equals one full vCPU, one virtual CPU core, a unit of allocated compute), so CPU-bound init (parsing, class loading, JIT warm-up, where JIT is just-in-time compilation of hot code) runs faster.

Trade-offs.

Strategy 2: snapshot restore (Lambda SnapStart)

Mechanism. When you publish a function version, Lambda runs Init once, takes a snapshot of the initialized environment's memory and disk (a Firecracker microVM snapshot, Firecracker being the lightweight virtual machine technology Lambda runs on), and caches it. New environments resume from the snapshot instead of re-running Init.

Trade-offs.

Strategy 3: pre-initialized capacity (provisioned concurrency)

Mechanism. You tell Lambda to keep N environments initialized for a version or alias at all times. Requests up to N concurrent never see a cold start; above N, they spill over (run on ordinary on-demand environments instead of the pre-warmed ones) to normal on-demand environments (and can be cold). The equivalents elsewhere are minimum instances on Google Cloud Run and always-ready instances on Azure Functions.

Trade-offs, with numbers. Price in US East (N. Virginia): $0.0000041667 per GB-second for keeping capacity provisioned, and duration drops to $0.0000097222 per GB-second (from $0.0000166667) for work that runs on it.

• 10 environments at 1 GB for a 30-day month: 10 GB x 2,592,000 s = 25,920,000 GB-s x $0.0000041667 = $108.00 per month, even with no traffic.
• If those environments serve 5 million requests of 80 ms at 1 GB (400,000 GB-s), duration costs $3.89 instead of $6.67 on-demand. The $108 floor dominates unless utilization is high.

Worked example: choosing

A checkout API on Node.js with a 250 ms p50 (median latency: half of requests finish faster than this) and 1.8 s p99, 40 requests/s in business hours, 2 requests/s at night; cold-starting requests show ~1.5 s Init Duration.

1. Trim the bundle and replace a whole-SDK import with single clients: Init Duration falls from ~1.5 s to, say, ~600 ms (measure it; don't assume).
2. SnapStart is not available for Node.js, so for the remaining spikes use provisioned concurrency. By Little's law (concurrency equals arrival rate times how long each request takes; here the 250 ms p50 stands in for that per-request time), 40 req/s x 0.25 s means about 10 requests are in flight at once, so provision ~12 (headroom) during business hours via a schedule and ~2 at night. Priced at the same $0.0000041667 per GB-second as above (1 GB, roughly 12 business hours and 12 night hours a day): 12 environments busy 12 h/day plus 2 environments the other 12 h/day, over a 30-day month, is about $75.60/month, cheaper than holding a flat 10 environments 24/7 ($108, from the pricing above) precisely because it drops to 2 overnight.
3. Re-check p99 and the spillover metric after a week.

Pitfalls

• "Keep-warm" pings (a scheduled event calling the function every few minutes) keep one environment warm. A burst that needs 20 environments still cold-starts 19. Provisioned concurrency is the supported version of this idea.
• Blaming cold starts for everything. Slow downstream calls, network set-up for functions attached to a VPC (Virtual Private Cloud, a private network), or large payloads also show up in p99. Split latency by cold versus warm before paying for a fix.
• Deploy frequency matters. Every deployment replaces all environments, so ten deploys a day means ten waves of cold starts.

Direct answer

At a sustained 100,000 events per second I would build this as Kinesis Data Streams (provisioned shards) feeding a Lambda consumer that enriches and pre-aggregates in batches, with a per-shard checkpoint committed atomically alongside the aggregates, plus a Firehose copy of the raw stream into S3 for replay. The two decisions that matter most are both about batching: producers pack many events into each stream record, and Lambda is invoked once per batch of about 1,600 events rather than once per event. Invoking once per event would cost about $52,560 a month in request fees alone. The batched design runs at about $10,900 a month, and the raw archive (not the compute) is the biggest line item.

Quick glossary for the rest of the answer:

• Serverless / FaaS (functions as a service): you upload a function, the cloud runs as many copies as traffic needs and bills per invocation and per millisecond. AWS Lambda is the example here.
• Kinesis Data Streams (KDS): AWS's managed, ordered, replayable log (similar in spirit to Kafka, Apache Kafka: the most common self-hosted alternative doing the same ordered-log job). It is split into shards; each shard accepts up to 1 MB/s or 1,000 records/s of writes.
• Event source mapping (ESM): the Lambda-managed poller that reads a shard and invokes your function with a batch of records.
• At-least-once delivery: every record is delivered, but some may be delivered twice (after a retry). The pipeline must tolerate duplicates.
• Idempotent: processing the same input twice leaves the same result as processing it once.
• Amazon Data Firehose: a managed service that buffers a stream and writes large files to S3. S3 is object storage; Parquet is a columnar file format; Athena is a serverless SQL engine that queries files in S3.
• DynamoDB: AWS's managed key-value database.

Requirements I am designing to

Architecture

flowchart LR
  P[Devices and gateways] -->|packed 24 KB records| K[Kinesis stream 125 shards]
  K --> L[Lambda consumer: enrich and pre-aggregate]
  L -->|one transaction per batch| D[(DynamoDB: per-shard minute aggregates and checkpoint)]
  L -.->|poison records| Q[SQS on-failure queue]
  D --> R[Rollup job every minute]
  R --> S3A[(S3 aggregates as Parquet, queried by Athena)]
  K --> F[Firehose]
  F --> S3R[(S3 raw archive for replay)]

1. Ingestion and buffering

• Shard count. 100 MB/s divided by 1 MB/s per shard is 100 shards by bytes. With 24 events packed per record, the record rate is 100,000 / 24 = 4,167 records/s, far below the 100,000 records/s limit of 100 shards. Bytes bind, so I provision 125 shards (25% headroom for skew and bursts).
• Why pack events. Unpacked, 100,000 records/s would need 100 shards just for the record-count limit, and every downstream fee charged per record (Kinesis PUT payload units, which bill each record in 25 KB chunks; Firehose's 5 KB billing increment) would be charged 100,000 times a second. Packing is done by the gateway or by the Kinesis Producer Library's (KPL, an AWS client library that batches and compresses records before sending them) aggregation feature.
• Partition key. Device ID, hashed, so load spreads evenly across shards. A partition key concentrated on a few tenants creates a hot shard (one shard at its 1 MB/s ceiling while others idle), which shows up as write throttling.
• Provisioned vs on-demand. On-demand mode (no shard planning, billed per GB) costs about $42,048/month at this volume versus about $1,522 provisioned (arithmetic below). For flat, known 100k/s load, provisioned wins by more than 20x. I would flip to on-demand only for a new stream whose traffic I cannot yet predict.
• Buffering. The stream itself is the buffer: default retention is 24 hours, so a consumer outage of a few hours loses nothing; the backlog is replayed when the consumer recovers.

2. Processing concurrency

• The Kinesis ESM runs one concurrent batch per shard by default. ParallelizationFactor (1 to 10) raises that, but it breaks per-shard ordering (ordering is then only per partition key), and my deduplication below relies on per-shard order. So I keep the factor at 1 and scale by adding shards instead.
• Maximum concurrency is therefore 125 execution environments, well inside Lambda's default regional quota of 1,000 concurrent executions. I set reserved concurrency (a per-function concurrency allotment that is both a guaranteed floor and a hard ceiling) of about 150 on this function so it can never starve other functions, and so nothing else can starve it.
• Batching settings: BatchSize 10,000 (the maximum), MaximumBatchingWindowInSeconds 2. Each shard receives 100,000 / 125 = 800 events/s, so each 2 s batch carries about 1,600 events (about 67 packed records, about 1.6 MB, safely under the 6 MB invocation payload limit even after base64 encoding inflates it by a third).
• That gives 125 / 2 = 62.5 invocations/s. If a batch takes about 120 ms, average concurrency is 62.5 x 0.12 = 7.5 environments. Cold starts (the extra latency when a new execution environment boots) are irrelevant here: environments stay warm under constant load, and a few hundred ms of extra delay on a 2 s batch changes nothing.
• Enrichment (joining each event to device metadata) uses a reference table loaded into memory at initialization and refreshed every few minutes, never a per-event database call. 100,000 lookups/s against a database would itself be a larger system than the pipeline.

3. Idempotency: exactly-once aggregates on at-least-once delivery

Kinesis plus Lambda is at-least-once. The ESM retries a whole batch if the function errors or times out, so if the function wrote aggregates and then timed out before returning, the retry would add the same events again. For aggregates (counters), duplicates silently inflate numbers.

The fix: store a per-shard checkpoint (the highest sequence number already counted) in the same atomic write as the aggregate increments. Each Kinesis record has a sequence number that increases within a shard. On every invocation the function:

1. Reads the checkpoint for its shard (cached in memory, re-read if the conditional write in step 3, a database write that only takes effect if the value has not changed since it was last read, fails).
2. Drops records whose sequence number is at or below the checkpoint (already counted).
3. Commits, in one DynamoDB TransactWriteItems call (an all-or-nothing multi-item write): increment the (shard, minute) aggregate items, and set the checkpoint to the batch's last sequence number, with a condition that the checkpoint still equals the value just read.

Either both the counts and the checkpoint land or neither does, so a retry of any subset of the batch finds its records already covered and adds nothing.

A small simulation proves it. It models 4 shards of 5,000 records, with a 10% chance each attempt crashes before writing and a 10% chance it crashes after writing, and halves the batch after each failure (a simplified model of BisectBatchOnFunctionError):

import random
from collections import Counter

def make_stream(num_shards=4, per_shard=5000, seed=7):
    rng = random.Random(seed)
    shards = {}
    for s in range(num_shards):
        recs = []
        for seq in range(1, per_shard + 1):
            minute = (seq - 1) * 60 // per_shard       # events spread over 60 minutes
            group = rng.choice(["sensor-a", "sensor-b", "sensor-c"])
            recs.append((seq, minute, group))
        shards[s] = recs
    return shards

class Store:
    """Stands in for the database: aggregate counters plus a per-shard checkpoint.
    commit() is atomic, like one TransactWriteItems call."""
    def __init__(self):
        self.agg = Counter()
        self.hwm = {}
    def commit(self, shard, deltas, new_hwm):
        self.agg.update(deltas)
        if new_hwm is not None:
            self.hwm[shard] = new_hwm

def handler(store, shard, batch, use_checkpoint):
    last_done = store.hwm.get(shard, 0) if use_checkpoint else 0
    fresh = [r for r in batch if r[0] > last_done]
    deltas = Counter((shard, minute, group) for _, minute, group in fresh)
    new_hwm = max(last_done, batch[-1][0]) if use_checkpoint else None  # never move backwards
    store.commit(shard, deltas, new_hwm)

def run(use_checkpoint, batch_size=500, p_fail_before=0.10, p_fail_after=0.10, seed=42):
    shards = make_stream()
    rng = random.Random(seed)
    store = Store()
    for shard, recs in shards.items():
        pos, size = 0, batch_size
        while pos < len(recs):
            batch = recs[pos:pos + size]
            roll = rng.random()
            if roll < p_fail_before:                 # crash before the write: nothing committed
                size = max(1, len(batch) // 2)       # simplified bisect-on-error
                continue
            handler(store, shard, batch, use_checkpoint)
            if roll < p_fail_before + p_fail_after:  # write committed, then timeout: batch retried
                size = max(1, len(batch) // 2)
                continue
            pos += len(batch)
            size = batch_size
    truth = Counter((s, m, g) for s, recs in shards.items() for _, m, g in recs)
    return truth, store.agg

for mode in (False, True):
    truth, got = run(use_checkpoint=mode)
    label = "with per-shard checkpoint" if mode else "naive (no checkpoint)   "
    print(f"{label}: true events={sum(truth.values())}, counted={sum(got.values())}, "
          f"every (shard, minute, group) cell exact={truth == got}")

Output:

naive (no checkpoint)   : true events=20000, counted=22250, every (shard, minute, group) cell exact=False
with per-shard checkpoint: true events=20000, counted=20000, every (shard, minute, group) cell exact=True

The naive version over-counts by 11.25%. The max(...) on the checkpoint line is load-bearing: my first version set the checkpoint to the retried half-batch's last sequence number, which moved it backwards after a bisect and still double counted (21,125). A checkpoint must only ever move forward.

Two things this does not cover, stated so nobody assumes otherwise: duplicates created by the producer (a device retrying a PUT gets a new sequence number) need an event ID and a dedupe step, and the raw archive in S3 is at-least-once, so queries over raw events should deduplicate by event ID.

4. Error handling

The DLQ (dead-letter queue) idea here is the on-failure destination: a place failed work goes so it is not lost and does not block the shard.

5. Storage choice

• Aggregates: DynamoDB holds the live per-shard, per-minute partial counts (small, write-heavy, needs atomic conditional writes). A rollup (a job that combines many small partial records into one summarized one) Lambda run every minute by EventBridge Scheduler (AWS's managed cron) sums the 125 shard items for a closed minute and writes one Parquet object per minute to S3 with a deterministic key such as aggregates/dt=2026-09-27/hh=14/mm=05.parquet. Rewriting the same key is naturally idempotent. Analysts query with Athena. A small daily compaction merges per-minute files into hourly ones to avoid a small-file problem (many tiny files make each file's fixed per-file read overhead dominate, slowing every downstream query).
• Raw events: Firehose reads directly from the stream and writes compressed files to S3, so enrichment or aggregation logic can be replayed after a bug fix.
• What would change it: if dashboards need sub-second freshness or high-concurrency interactive queries, I would write aggregates to a real-time OLAP (online analytical processing) database such as ClickHouse or Apache Druid instead of S3 plus Athena.

6. Observability

• Consumer lag: Lambda's IteratorAge metric (how old the last record in each batch was when the batch was sent to the function). This is the single most important alarm: rising iterator age means processing is falling behind. Alarm if it exceeds 60 s for 5 minutes.
• Stream health: Kinesis WriteProvisionedThroughputExceeded (a hot or under-provisioned shard) and GetRecords.IteratorAgeMilliseconds.
• Function health: Errors, Throttles, Duration (p99, meaning the 99th percentile), ConcurrentExecutions.
• Business correctness: a reconciliation job (one that independently checks two derived numbers against each other and flags any mismatch) comparing hourly raw-archive event counts with aggregated counts, alarming on a difference above 0.1%. This catches silent double counting or silent drops that no infrastructure metric will show.
• Structured logs with shard ID and sequence range per batch, so any aggregate can be traced to the records behind it.

7. Cost profile

List prices for us-east-1 as published on the AWS pricing pages; the per-event CPU cost is an assumption to be measured in a load test. DynamoDB is billed in write request units (WRU): under on-demand pricing a normal write costs 1 WRU per KB, and (as the pitfall below explains) a transactional write costs double that per KB:

import math
# us-east-1 list prices (verify on the pricing pages before relying on them)
LAMBDA_GBS = 0.0000166667      # $ per GB-second, x86, first tier
LAMBDA_REQ = 0.20 / 1e6        # $ per request
KDS_SHARD_HR = 0.015           # provisioned shard-hour
KDS_PUT_UNIT = 0.014 / 1e6     # per 25 KB PUT payload unit
KDS_OD_IN, KDS_OD_OUT = 0.08, 0.04   # on-demand $/GB ingested, retrieved
FH_GB = 0.029                  # Firehose ingestion $/GB, billed in 5 KB increments
DDB_WRU = 0.625 / 1e6          # on-demand write request unit

SEC_MONTH = 730 * 3600         # AWS bills a month as 730 hours
events_s, event_kb = 100_000, 1.0
events_per_record = 24         # producers pack ~24 KB records
shards = 125                   # 100 MB/s needs 100 shards at 1 MB/s each, plus 25% headroom

records_s = events_s / events_per_record
mb_s = events_s * event_kb / 1000
gb_month = mb_s / 1000 * SEC_MONTH

kds_prov = shards * 730 * KDS_SHARD_HR + records_s * SEC_MONTH * KDS_PUT_UNIT
kds_od = gb_month * (KDS_OD_IN + 2 * KDS_OD_OUT)   # two consumers: Lambda and Firehose

window_s, mem_gb = 2, 1.0
inv_s = shards / window_s
events_per_inv = events_s / inv_s
duration_s = 0.040 + events_per_inv * 0.00005     # ASSUMED: 40 ms fixed + 0.05 ms per event
lam = inv_s * SEC_MONTH * (LAMBDA_REQ + duration_s * mem_gb * LAMBDA_GBS)
per_event_invoke_requests = events_s * SEC_MONTH * LAMBDA_REQ

wru_per_inv = 2 * 1 + 2 * 4 * 1.05   # txn: 1 KB checkpoint + ~1.05 x 4 KB aggregate item, 2 WRU per KB
ddb = inv_s * SEC_MONTH * wru_per_inv * DDB_WRU

fh_billed_kb = math.ceil(events_per_record * event_kb / 5) * 5
fh = records_s * fh_billed_kb / 1e6 * SEC_MONTH * FH_GB

print(f"records/s={records_s:.0f}  data={mb_s:.0f} MB/s  GB/month={gb_month:,.0f}")
print(f"Kinesis provisioned={kds_prov:,.0f}  on-demand={kds_od:,.0f}")
print(f"Lambda: {inv_s:.1f} inv/s, {events_per_inv:.0f} events/inv, {duration_s*1000:.0f} ms, "
      f"avg concurrency={inv_s*duration_s:.1f}, cost={lam:,.0f}")
print(f"Lambda request fee alone if invoked once per event={per_event_invoke_requests:,.0f}")
print(f"DynamoDB aggregates={ddb:,.0f}")
print(f"Firehose raw archive (billed {fh_billed_kb} KB/record)={fh:,.0f}")
print(f"TOTAL (provisioned Kinesis)={kds_prov + lam + ddb + fh:,.0f} $/month")

Output (dollars per month):

records/s=4167  data=100 MB/s  GB/month=262,800
Kinesis provisioned=1,522  on-demand=42,048
Lambda: 62.5 inv/s, 1600 events/inv, 120 ms, avg concurrency=7.5, cost=361
Lambda request fee alone if invoked once per event=52,560
DynamoDB aggregates=1,068
Firehose raw archive (billed 25 KB/record)=7,939
TOTAL (provisioned Kinesis)=10,890 $/month

The ~4 KB aggregate-item size is an assumption for this example: it is enough room for counts and sums across roughly a few dozen (device group, metric) key combinations packed into one item; size it against your own key cardinality. The ~1.05 multiplier on that item accounts for the occasional batch that straddles a minute boundary and touches two minute items. S3 storage and Athena query charges are excluded because they depend on retention and query volume.

What the numbers say:

• Compute is the cheap part ($361). Lambda cost is dominated by how you batch, not by how fast the code is.
• The raw archive is 73% of the bill. Its value (replay, ad hoc analysis) should be confirmed with the data consumers. If a 7-day replay window is enough, extending the stream's retention instead of archiving everything could remove most of it.
• Firehose bills each record rounded up to 5 KB, so 1 KB unpacked records would be billed at 5x their size. Packing records to just under a 5 KB multiple matters.

Trade-offs and pitfalls

• Is serverless right at this volume? The load is flat and high, which is where serverless is weakest on unit price. It still wins here because the managed pieces (stream, poller, retries, scaling to 125 parallel consumers, no servers to patch) cost less in engineering time than running and patching your own Kafka and stream-processing cluster, while the Lambda line itself is only a few hundred dollars. If enrichment grew heavy, say 2 ms of CPU per event instead of 0.05 ms, two things break. First, one consumer per shard can then process at most 1 / 0.002 = 500 events/s, but each shard receives 800, so the stream falls steadily behind unless you add shards: 100,000 events/s / 500 events/s per shard needs at least 200 shards by that arithmetic (up from 125), before headroom. Second, compute becomes 100,000 x 0.002 = 200 GB-seconds every second at 1 GB, which is 200 x 2,628,000 s/month x $0.0000166667/GB-s, about $8,760 a month in Lambda duration alone. At that point a long-running container consumer (for example Apache Flink, an open-source stream-processing engine that keeps one continuously running process instead of Lambda's start-stop-per-batch model, on a managed service) is the better call, because steady, CPU-heavy work is exactly where per-millisecond billing loses.
• Per-event invocation is the classic mistake: 262.8 billion requests a month.
• ParallelizationFactor as the first scaling lever silently breaks the per-shard ordering the checkpoint relies on. Scale with shards.
• Aggregating in the function's memory across invocations is unsafe: environments are recycled at any time and two environments can serve the same shard over time. State lives in the database or in Lambda's managed tumbling-window state (a built-in feature that carries a small amount of state between consecutive batches from the same shard, for windowed aggregation, which AWS documents as at-least-once and capped at 1 MB per shard, so it does not remove the double-count problem).
• Transactions are not free: each item written in a DynamoDB transaction costs 2 write request units (WRU) per KB instead of 1. That is why there is one transaction per batch, not per event.

Direct answer

The platform's strength is the problem here. A serverless function can scale from 5 to 500 copies in seconds. Each copy has its own retry loop and its own memory, so a traffic spike turns into a flood aimed at a dependency that cannot scale. The fix is to match the rate at which you call the dependency to what it can take:

• put a queue between the trigger and the work
• cap how many function instances may call the dependency at once
• retry with exponential backoff and jitter (a small random extra delay added to each retry, so many instances retrying at once do not all fire again at the same instant) at one layer only
• trip a circuit breaker when the dependency is failing
• decide in advance what a degraded response looks like

On a serverless platform most of this is configuration: the queue trigger's maximum concurrency, reserved concurrency, the visibility timeout (the window during which a message SQS has handed to a consumer is hidden from everyone else; if it is not deleted before the window ends, it reappears on the queue) used as a backoff timer, partial batch responses, and a dead-letter queue.

1. Buffer with a queue

If the caller doesn't need the result immediately, accept the request into a queue (Amazon SQS, Simple Queue Service, here) and return 202 Accepted with a status link. The queue absorbs the spike and holds work while the dependency is down. SQS keeps messages for 4 days by default and up to 14.

2. Rate-match the consumers (the core control)

Set MaximumConcurrency on the SQS event source mapping (the piece of Lambda configuration that polls the queue on your behalf and invokes your function with each batch it reads; allowed range 2 to 1,000). This caps how many function instances the poller (that same polling component) will run at once. Set the function's reserved concurrency to at least that value. Work out the number from the dependency's limit, not from your traffic. See the worked example below.

Reserved concurrency alone is the wrong tool. The poller keeps invoking, Lambda throttles the invocations, and throttled batches go back to the queue and use up their receive count (SQS's own counter of how many times a message has been handed to a consumer without being deleted) on the way to the dead-letter queue (DLQ). MaximumConcurrency stops the poller from invoking in the first place.

If you need an exact global rate across instances (for example, "100 calls per second" rather than "20 in flight"), keep a shared token bucket (a counter that refills at a fixed rate; each call must take one token before it is allowed to proceed, which caps the combined rate no matter how many instances are calling) in a fast store such as Redis or a DynamoDB counter. The cost is one extra round trip per call.

3. Retries: one layer, with backoff and jitter

Retries can happen in five places: the HTTP/SDK client, the function code, Lambda's asynchronous retries (2 by default), SQS redelivery after the visibility timeout, and the original caller. Stacked, they multiply (see the worked example). Choose one:

• Retry only retryable failures: 429, 503 and timeouts. Honour Retry-After when the dependency sends it. Never retry a 400.
• Use the queue as the retry engine. On failure, call ChangeMessageVisibility (the API that resets a message's visibility timeout) with a delay that grows with ApproximateReceiveCount (that same receive-count attribute). This gives exponential backoff without holding a function instance open while it waits.
• Enable partial batch responses (ReportBatchItemFailures), so only the failed messages come back.
• Send poison messages (messages that keep failing no matter how many times they are retried) to a DLQ (dead-letter queue) once their receive count passes maxReceiveCount, a threshold you set to at least 5, and alarm on DLQ depth.

4. Circuit breaker

A circuit breaker counts recent failures. When they cross a threshold it moves from closed (calls flow) to open (fail immediately without calling). After a cool-down it moves to half-open (let a few probe calls through) and closes again if they succeed.

On serverless the breaker's state lives in one environment's memory, so each instance learns about the outage on its own. With concurrency capped at 20, that is acceptable: at most 20 probes per half-open window. For hundreds of instances, share the state in one small record (Redis key or DynamoDB item with a time-to-live), at the price of one read per call.

The serverless-specific trap: when the breaker is open, "fail the batch fast" puts messages back on the queue and increments their receive count. A 30-minute outage then pushes healthy messages into the DLQ. Two options:

• Pause consumption. Automation disables the event source mapping (UpdateEventSourceMapping with Enabled=false) when the breaker opens, and re-enables it after the cool-down.
• Keep the breaker's fast failures below maxReceiveCount by using long visibility delays.

5. Rate limiting at the edge

Reject excess traffic before it costs a function invocation:

• API Gateway stage and method throttling (rate caps set on the whole API stage, or on one method within it), plus usage plans (API Gateway's per-API-key throttle and quota configuration) with per-API-key rate and burst limits (the same token bucket mechanism as above).
• A rate-based rule in the web application firewall (AWS WAF) that limits each source IP.
• A throttled request gets a 429 and never runs your code.

6. Degraded mode when the dependency is down

Timeouts are part of the design. Give the dependency call a short client timeout (for example 2 s) that is well under the function timeout. Otherwise a hung partner keeps all 20 capped instances busy doing nothing.

Worked example

A partner API allows 100 requests per second. Each call takes 200 ms, and a function instance makes its calls one after another. Traffic arrives at 60 messages per second.

vendor_limit_rps = 100      # partner API contract
call_s = 0.200              # one partner call, sequential inside an invocation
calls_per_invocation_per_s = 1 / call_s
max_concurrency = vendor_limit_rps / calls_per_invocation_per_s
print(f"queue consumer MaximumConcurrency = {vendor_limit_rps} / {calls_per_invocation_per_s:.0f} = {max_concurrency:.0f}")

layers, retries = 3, 3
print(f"worst-case attempts reaching the partner per user request: (1+{retries})^{layers} = {(1 + retries) ** layers}")

arrival_rps, outage_s = 60, 600
backlog = arrival_rps * outage_s
drain_s = backlog / (vendor_limit_rps - arrival_rps)
print(f"backlog after a {outage_s // 60}-minute outage: {backlog:,} messages; "
      f"drain time at {vendor_limit_rps}/s while {arrival_rps}/s keep arriving: {drain_s:.0f} s ({drain_s / 60:.0f} min)")

queue consumer MaximumConcurrency = 100 / 5 = 20
worst-case attempts reaching the partner per user request: (1+3)^3 = 64
backlog after a 10-minute outage: 36,000 messages; drain time at 100/s while 60/s keep arriving: 900 s (15 min)

• One instance makes 5 calls per second, so 20 instances exactly fill the 100 RPS contract. Set MaximumConcurrency=20 and reserved concurrency to at least 20. (If a handler makes its calls in parallel, divide by that fan-out as well.)
• Three layers each retrying 3 times means 64 attempts reach the partner for one user request in the worst case. That is how a small blip becomes a retry storm (retries compounding across layers until the call volume reaching the dependency is far higher than the original traffic, which can itself take the dependency down).
• A 10-minute outage leaves 36,000 messages queued, well inside the retention period. Once the dependency recovers, the backlog drains in about 15 minutes, because only 40 RPS of headroom remains while new traffic keeps arriving. Add the 10-minute outage itself and the incident runs for about 25 minutes end to end, from the moment the outage starts to the moment the backlog is fully cleared, which is the number to put in front of the product owner as "how long until we are fully caught up." That is not how long any single message sits stale: SQS works through the backlog in roughly arrival order, so the very first message queued is also first in line once the dependency is back, and its own wait is close to the 10-minute outage, not 25 minutes.

flowchart LR
  client[Client] --> gw[API Gateway, throttled]
  gw --> q[Queue]
  q --> fn[Function, max concurrency 20]
  fn -->|breaker closed| partner[Partner API, 100 RPS]
  fn -->|breaker open| cache[(Last known values)]
  q -.->|after 5 receives| dlq[DLQ]

Pitfalls

• Scaling the function to "handle the load" when the dependency is the bottleneck. More instances produce more throttling.
• Retries at every layer, with no jitter. Every instance retries in the same second.
• In-memory rate limiters that assume a single process.
• A visibility timeout barely above the function timeout: a batch that was throttled or slow reappears while it is still being processed, producing duplicates. Follow AWS's 6x guidance: set the queue's visibility timeout to at least six times the function's timeout (plus any batching window), so a batch that gets retried after a throttle still has room to finish before it goes invisible-then-visible again. (A visibility timeout shorter than the function timeout is rejected by Lambda outright.)
• No DLQ alarm, so failed work sits unseen for days.

Direct answer

"At-least-once" means the platform promises never to lose an event but may deliver it more than once. That happens after a timeout, a crash between processing and acknowledging, or a retry. Since duplicates are guaranteed to happen eventually, the handler must make the effect happen once even when the delivery happens twice. There are four patterns, in order of preference:

1. make the operation naturally idempotent: a plain set, an upsert (insert the row if it is missing, otherwise overwrite it, so replaying it twice changes nothing), or a compare-and-set (write a new value only if the current value still equals what you last read, so a stale retry is rejected instead of double-applying) instead of an increment
2. record an idempotency key (a value that identifies this one specific delivery, so a lookup can answer "have I already handled this?" before doing the work again) in the same transaction as the effect
3. keep an idempotency record with an in-progress lock and a cached result when the effect lives in another system
4. pass the key downstream to APIs that accept one

Broker-level deduplication (for example, the 5-minute window on an Amazon SQS FIFO, first-in first-out, queue) helps, but it does not replace any of these. The trade-offs are one extra conditional write per message (a write the store performs only if a condition on the current data holds, for example "only if this key does not already exist", checked and applied atomically so two concurrent writers cannot both succeed), storage that grows with rate x retention window, and correctness gaps that depend on the order in which you record the key and perform the effect.

Choosing the key

• Use a business identity (orderId + eventType, or the object's bucket + key + version) rather than the broker's message id. A producer that retries a send creates a new message id carrying the same content.
• Never hash the whole payload if it contains timestamps or trace ids. Two copies of the same event would produce different keys.

The ordering problem, demonstrated

The dangerous part is not the lookup. It is what happens when the process dies between the two writes. This demo uses a local SQLite database to model three orderings of "record the key" and "apply the effect". The first delivery crashes, and then the event is delivered twice more:

import sqlite3

class Crash(Exception):
    pass

def fresh():
    db = sqlite3.connect(":memory:", isolation_level=None)
    db.execute("CREATE TABLE account (id TEXT PRIMARY KEY, balance INTEGER)")
    db.execute("CREATE TABLE processed (msg_id TEXT PRIMARY KEY)")
    db.execute("INSERT INTO account VALUES ('A', 0)")
    return db

def check_then_mark(db, msg, crash):
    if db.execute("SELECT 1 FROM processed WHERE msg_id=?", (msg["id"],)).fetchone():
        return
    db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))
    if crash:
        raise Crash()                      # dies after the side effect, before recording it
    db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))

def mark_first(db, msg, crash):
    try:
        db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))
    except sqlite3.IntegrityError:
        return
    if crash:
        raise Crash()                      # dies after claiming, before the side effect
    db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))

def same_transaction(db, msg, crash):
    db.execute("BEGIN")
    try:
        db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))
        db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))
        if crash:
            raise Crash()
        db.execute("COMMIT")
    except sqlite3.IntegrityError:
        db.execute("ROLLBACK")             # duplicate: nothing to do
    except Crash:
        db.execute("ROLLBACK")             # models the transaction dying with the process
        raise

msg = {"id": "m-1", "amount": 50}
for handler in (check_then_mark, mark_first, same_transaction):
    db = fresh()
    try:
        handler(db, msg, crash=True)       # first delivery crashes, so the queue redelivers
    except Crash:
        pass
    handler(db, msg, crash=False)          # redelivery
    handler(db, msg, crash=False)          # plus one ordinary duplicate
    print(f"{handler.__name__:17} balance={db.execute('SELECT balance FROM account').fetchone()[0]} (correct: 50)")

rate, window_s, bytes_per_key = 500, 24 * 3600, 100
keys = rate * window_s
print(f"dedup keys held for a 24 h window at {rate}/s: {keys:,} (~{keys * bytes_per_key / 1e9:.1f} GB at {bytes_per_key} B each)")

check_then_mark   balance=100 (correct: 50)
mark_first        balance=0 (correct: 50)
same_transaction  balance=50 (correct: 50)
dedup keys held for a 24 h window at 500/s: 43,200,000 (~4.3 GB at 100 B each)

• Check, then act, then mark: the crash happens after the money moved but before the key was recorded. The redelivery applies it again, so balance 100: double effect.
• Mark first, then act: the key is recorded and then the crash happens. The redelivery sees the key and skips, so balance 0: lost effect. The message was acknowledged as done when it wasn't.
• Key and effect in one transaction: either both happen or neither does, so balance 50: correct. In DynamoDB (Amazon's managed key-value and document database) the equivalent is TransactWriteItems, which commits several writes as one all-or-nothing operation: a conditional put on the key that only succeeds if attribute_not_exists (the item is not already there) holds, plus the update.

The patterns and their costs

The lock in the third pattern matters because two copies of the same event can run concurrently on two function instances. The conditional put makes exactly one of them the owner. The other gets "already in progress" and should fail and retry later, not proceed.

The remaining gap, with a timeline. Say the in-progress record expires after 1 hour. At t = 0 min instance A wins the conditional put, marks the record INPROGRESS, and calls the external API. At t = 45 min A is still waiting on a slow call. At t = 60 min the record expires, so at t = 61 min a redelivery lands on instance B, sees no live record, wins a fresh conditional put, and starts the same external call again while A's original call may still be in flight. Sizing the expiry comfortably above your slowest realistic call, and above the external system's own retry window, is what keeps this gap rare rather than routine.

Storage and performance

• Storage grows with rate x window. At 500 events per second with a 24-hour window, that is 43,200,000 keys, about 4.3 GB at 100 bytes each (from the script). Expire keys with a time-to-live (TTL). DynamoDB TTL deletion runs in the background and is not immediate, so filter on the expiry attribute when reading.
• The window must cover the longest possible redelivery, not the typical one. The worst case is the queue's retention (up to 14 days on SQS), async retry age (up to 6 hours on Lambda), or a manual redrive (an operator replaying the stuck messages back onto the source queue) from the dead-letter queue (DLQ: a separate queue a message is moved to once it exhausts its retries, so it can be inspected and replayed by hand instead of being silently dropped) days later. The Powertools default expiry is 1 hour. It is too short if someone redrives a DLQ a day later.
• Latency. A conditional write is one round trip to the key-value store, per message. For batches, pre-filter with one batch read and then do conditional writes only for the unseen keys.
• Hot keys. A hot key is one key taking a disproportionate share of traffic. Here, duplicates of the same event land on the same key, and the lock serializes them (each one waits its turn instead of running in parallel), so that single key shows up as contention (many requests queued behind one item) on dashboards even though the behaviour is correct.

Pitfalls

• Deduplicating on message id. Producer retries slip through.
• An in-memory set of seen ids. Every environment has its own, and it is lost on recycle.
• Recording the key before the effect (the lost-update case above).
• A dedup window shorter than your retry and redrive horizon.
• Returning a different response on the duplicate than on the original. Cache the first result so callers see the same answer both times.

Direct answer

A FaaS invocation runs inside an execution environment: an isolated sandbox holding your code, the language runtime and any extensions. The first request that needs a new environment pays a cold start: the platform fetches your code, boots the sandbox, starts the runtime and runs your initialization code (everything outside the handler function). Later requests that land on the same environment are warm: only the handler runs. Between requests the environment is frozen; eventually the platform tears it down. So expensive setup, like loading a large dependency, belongs in initialization code at module scope, where it is paid once per environment, not inside the handler, where it would be paid on every request.

The lifecycle, phase by phase (AWS Lambda)

stateDiagram-v2
    [*] --> Init: first request or scale-out
    Init --> Invoke: handler runs
    Invoke --> Frozen: response returned
    Frozen --> Invoke: next request (warm)
    Invoke --> Init: crash or timeout resets it
    Frozen --> Shutdown: idle or recycled
    Shutdown --> [*]

1. Cold start, platform part. Lambda downloads your code (zip or container image) and starts the sandbox. You cannot run code here; you influence it only through package size and runtime choice.
2. Init phase. Three steps, in order: Extension init (starts any extensions, which are helper processes such as monitoring agents), Runtime init (boots Python, Node.js, the JVM, Java's language runtime, and so on), Function init (runs your module-level code: imports, SDK clients, config loads). On-demand functions get 10 seconds for Init; if it does not finish, Lambda retries the Init as part of the first invocation, bounded by the function timeout. (Provisioned concurrency, a paid feature that keeps a chosen number of environments already initialized and idle so the very first request to each hits a warm environment, and SnapStart, which pre-initialize environments, are allowed a longer Init.) Since 1 August 2025, Init time is billed for on-demand zip functions too, as it already was for container images, custom runtimes (a runtime you bring yourself, via the provided runtime family, instead of one of Lambda's built-in language runtimes) and provisioned concurrency.
3. Invoke phase. Lambda calls your handler with the event. The function timeout (up to 15 minutes) limits this phase. The Duration metric covers this phase only; on a cold invocation the REPORT log line (a summary line Lambda automatically appends to every invocation's logs, with Duration, Billed Duration and Memory Used) adds a separate Init Duration field.
4. Freeze and thaw. After the handler returns, the environment is frozen: no CPU, but memory, open connections and files in /tmp survive. The next request thaws it (a warm start). Background work you started and did not finish resumes on thaw, which is a source of odd bugs.
5. Shutdown. When the environment is idle long enough, or during routine maintenance (Lambda recycles environments every few hours even under constant traffic), Lambda sends a Shutdown event to extensions and kills the runtime. The window is 0 ms with no extensions, 500 ms with an internal extension (one that runs in-process, inside the runtime itself), and 2,000 ms with external extensions (ones that run as their own separate process alongside the runtime, such as a monitoring agent shipped as a layer; Lambda needs longer to signal and wait for a whole separate process to exit cleanly). If an invocation crashes or times out, Lambda resets the environment and the next request pays Init again (the diagram's Invoke --> Init edge above: the same failure path as a normal shutdown, just triggered by a crash instead of idling out).

What runs only on a cold start versus on every request

Where to put expensive setup

boto3 is the AWS SDK for Python, and /opt is the read-only directory where Lambda mounts any layers attached to the function, the natural place a packaged model file would land.

import json
import boto3

s3 = boto3.client("s3")     # module scope: created once per environment
_model = None               # large dependency, loaded lazily and then kept

def get_model():
    global _model
    if _model is None:      # first request in this environment pays; later ones reuse it
        _model = load_large_dependency("/opt/model.bin")
    return _model

def handler(event, context):
    user_id = event["user_id"]          # per-request data stays local, never global
    result = get_model().predict(event["features"])
    return {"user_id": user_id, "score": result}

(load_large_dependency stands for whatever deserializes your dependency.)

Rule of thumb: if every request needs the dependency, load it at module scope so it happens during Init (with provisioned concurrency or SnapStart that means before any user waits). If only some code paths need it, load it lazily as above so the paths that don't need it get a fast cold start.

Worked example: why placement matters

"At module scope" below covers both patterns shown above: loading eagerly when the module is first imported, and loading lazily into a global on first use, the way get_model does. Either way the expensive work runs exactly once per environment, during whichever request first triggers it; only the code path differs.

Loading the dependency takes 1,200 ms; the rest of the handler takes 40 ms. Suppose each environment serves 500 requests before it is recycled.

• At module scope: one request in 500 pays 1,200 ms extra. Averaged, that is 1,200 / 500 = 2.4 ms per request, and 499 of 500 requests take about 40 ms.
• Inside the handler: every request takes 1,200 + 40 = 1,240 ms, 31 times slower, and you pay for 1,240 ms of duration on every call instead of 40 ms.

The cold request still takes about 1,240 ms either way; module placement just stops you paying that cost 500 times.

Pitfalls

• Per-request data in globals leaks between requests. A global current_user set in one invocation is still there in the next. Keep request data local.
• Frozen connections go stale. A database connection opened at Init may be dropped by the server while the environment is frozen. Reuse it, but reconnect on error.
• Hitting the 10-second Init cap turns a slow Init into a retry inside the first invocation, which then pays Init plus handler against the function timeout. If Init is near 10 s, shrink it or pre-initialize.
• Snapshot-based start (SnapStart) restores many environments from one snapshot, so anything "unique" created at Init (random seeds, unique IDs, temporary credentials) is shared by every restored copy. Generate those in the handler or in an after-restore hook (a callback that runs once for each newly restored environment, immediately after SnapStart restores it from the snapshot, the intended place for values that must be unique per environment).
• Don't assume the environment persists. Treat globals and /tmp as a best-effort cache that can vanish at any time.