Serverless and Function-as-a-Service Architecture Questions

Direct answer

"At-least-once" means the platform promises never to lose an event but may deliver it more than once. That happens after a timeout, a crash between processing and acknowledging, or a retry. Since duplicates are guaranteed to happen eventually, the handler must make the effect happen once even when the delivery happens twice. There are four patterns, in order of preference:

1. make the operation naturally idempotent: a plain set, an upsert (insert the row if it is missing, otherwise overwrite it, so replaying it twice changes nothing), or a compare-and-set (write a new value only if the current value still equals what you last read, so a stale retry is rejected instead of double-applying) instead of an increment
2. record an idempotency key (a value that identifies this one specific delivery, so a lookup can answer "have I already handled this?" before doing the work again) in the same transaction as the effect
3. keep an idempotency record with an in-progress lock and a cached result when the effect lives in another system
4. pass the key downstream to APIs that accept one

Broker-level deduplication (for example, the 5-minute window on an Amazon SQS FIFO, first-in first-out, queue) helps, but it does not replace any of these. The trade-offs are one extra conditional write per message (a write the store performs only if a condition on the current data holds, for example "only if this key does not already exist", checked and applied atomically so two concurrent writers cannot both succeed), storage that grows with rate x retention window, and correctness gaps that depend on the order in which you record the key and perform the effect.

Choosing the key

• Use a business identity (orderId + eventType, or the object's bucket + key + version) rather than the broker's message id. A producer that retries a send creates a new message id carrying the same content.
• Never hash the whole payload if it contains timestamps or trace ids. Two copies of the same event would produce different keys.

The ordering problem, demonstrated

The dangerous part is not the lookup. It is what happens when the process dies between the two writes. This demo uses a local SQLite database to model three orderings of "record the key" and "apply the effect". The first delivery crashes, and then the event is delivered twice more:

import sqlite3

class Crash(Exception):
    pass

def fresh():
    db = sqlite3.connect(":memory:", isolation_level=None)
    db.execute("CREATE TABLE account (id TEXT PRIMARY KEY, balance INTEGER)")
    db.execute("CREATE TABLE processed (msg_id TEXT PRIMARY KEY)")
    db.execute("INSERT INTO account VALUES ('A', 0)")
    return db

def check_then_mark(db, msg, crash):
    if db.execute("SELECT 1 FROM processed WHERE msg_id=?", (msg["id"],)).fetchone():
        return
    db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))
    if crash:
        raise Crash()                      # dies after the side effect, before recording it
    db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))

def mark_first(db, msg, crash):
    try:
        db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))
    except sqlite3.IntegrityError:
        return
    if crash:
        raise Crash()                      # dies after claiming, before the side effect
    db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))

def same_transaction(db, msg, crash):
    db.execute("BEGIN")
    try:
        db.execute("INSERT INTO processed VALUES (?)", (msg["id"],))
        db.execute("UPDATE account SET balance = balance + ? WHERE id='A'", (msg["amount"],))
        if crash:
            raise Crash()
        db.execute("COMMIT")
    except sqlite3.IntegrityError:
        db.execute("ROLLBACK")             # duplicate: nothing to do
    except Crash:
        db.execute("ROLLBACK")             # models the transaction dying with the process
        raise

msg = {"id": "m-1", "amount": 50}
for handler in (check_then_mark, mark_first, same_transaction):
    db = fresh()
    try:
        handler(db, msg, crash=True)       # first delivery crashes, so the queue redelivers
    except Crash:
        pass
    handler(db, msg, crash=False)          # redelivery
    handler(db, msg, crash=False)          # plus one ordinary duplicate
    print(f"{handler.__name__:17} balance={db.execute('SELECT balance FROM account').fetchone()[0]} (correct: 50)")

rate, window_s, bytes_per_key = 500, 24 * 3600, 100
keys = rate * window_s
print(f"dedup keys held for a 24 h window at {rate}/s: {keys:,} (~{keys * bytes_per_key / 1e9:.1f} GB at {bytes_per_key} B each)")

check_then_mark   balance=100 (correct: 50)
mark_first        balance=0 (correct: 50)
same_transaction  balance=50 (correct: 50)
dedup keys held for a 24 h window at 500/s: 43,200,000 (~4.3 GB at 100 B each)

• Check, then act, then mark: the crash happens after the money moved but before the key was recorded. The redelivery applies it again, so balance 100: double effect.
• Mark first, then act: the key is recorded and then the crash happens. The redelivery sees the key and skips, so balance 0: lost effect. The message was acknowledged as done when it wasn't.
• Key and effect in one transaction: either both happen or neither does, so balance 50: correct. In DynamoDB (Amazon's managed key-value and document database) the equivalent is TransactWriteItems, which commits several writes as one all-or-nothing operation: a conditional put on the key that only succeeds if attribute_not_exists (the item is not already there) holds, plus the update.

The patterns and their costs

The lock in the third pattern matters because two copies of the same event can run concurrently on two function instances. The conditional put makes exactly one of them the owner. The other gets "already in progress" and should fail and retry later, not proceed.

The remaining gap, with a timeline. Say the in-progress record expires after 1 hour. At t = 0 min instance A wins the conditional put, marks the record INPROGRESS, and calls the external API. At t = 45 min A is still waiting on a slow call. At t = 60 min the record expires, so at t = 61 min a redelivery lands on instance B, sees no live record, wins a fresh conditional put, and starts the same external call again while A's original call may still be in flight. Sizing the expiry comfortably above your slowest realistic call, and above the external system's own retry window, is what keeps this gap rare rather than routine.

Storage and performance

• Storage grows with rate x window. At 500 events per second with a 24-hour window, that is 43,200,000 keys, about 4.3 GB at 100 bytes each (from the script). Expire keys with a time-to-live (TTL). DynamoDB TTL deletion runs in the background and is not immediate, so filter on the expiry attribute when reading.
• The window must cover the longest possible redelivery, not the typical one. The worst case is the queue's retention (up to 14 days on SQS), async retry age (up to 6 hours on Lambda), or a manual redrive (an operator replaying the stuck messages back onto the source queue) from the dead-letter queue (DLQ: a separate queue a message is moved to once it exhausts its retries, so it can be inspected and replayed by hand instead of being silently dropped) days later. The Powertools default expiry is 1 hour. It is too short if someone redrives a DLQ a day later.
• Latency. A conditional write is one round trip to the key-value store, per message. For batches, pre-filter with one batch read and then do conditional writes only for the unseen keys.
• Hot keys. A hot key is one key taking a disproportionate share of traffic. Here, duplicates of the same event land on the same key, and the lock serializes them (each one waits its turn instead of running in parallel), so that single key shows up as contention (many requests queued behind one item) on dashboards even though the behaviour is correct.

Pitfalls

• Deduplicating on message id. Producer retries slip through.
• An in-memory set of seen ids. Every environment has its own, and it is lost on recycle.
• Recording the key before the effect (the lost-update case above).
• A dedup window shorter than your retry and redrive horizon.
• Returning a different response on the duplicate than on the original. Cache the first result so callers see the same answer both times.

Direct answer

A FaaS platform (functions-as-a-service: you upload a function and the provider runs it once per event) scales by creating more execution environments. An execution environment is a small isolated sandbox with your runtime and code loaded, and it handles one request at a time. So 600 requests in flight at the same moment need 600 environments. When a request arrives and no idle environment exists, Lambda builds a new one first. That is a cold start. Growth is bounded by two different limits:

• a per-function scaling rate: how fast new environments can be created
• a Regional account concurrency quota: how many can exist at once, shared by every function in the account

Reserved concurrency gives a function its own slice of that quota and caps it there. Provisioned concurrency keeps environments already initialized, so requests that land on them skip the cold start. For a latency-sensitive endpoint, the cold starts and throttles all arrive in the same first seconds of a burst. The fix is to pre-warm the endpoint with provisioned concurrency sized to the burst and make sure the quota has headroom before the burst starts.

How a new environment gets provisioned

1. Routing. A request arrives. If an idle, already-initialized ("warm") environment exists, Lambda sends the request there. No setup cost.
2. Init phase (the cold start). If there is no idle environment, Lambda creates one: it loads your code package, starts the language runtime and any extensions, then runs your top-level code. That is everything outside the handler: imports, SDK clients, configuration loading.
3. Invoke phase. The handler runs.
4. Freeze and reuse. After the response, the environment is frozen and kept for later requests. How long it is kept is unspecified. It can be reclaimed at any time, so you cannot rely on it staying warm.

Concurrency means the number of requests being processed at the same instant, and it is also the number of execution environments Lambda needs at that instant. You can compute it with Little's law: concurrency = arrival rate (RPS) x the average time each request spends in the system (duration in seconds). That gives you environments directly; there is no hard per-environment throughput ceiling. AWS's own conceptual scaling guide warns against exactly the framing that sounds intuitive here: "it's incorrect to say that each Lambda execution environment can handle only a maximum of 10 requests per second. Instead of observing the load on any individual execution environment, Lambda only considers overall concurrency and overall requests per second when calculating your quotas." A single warm environment reused back to back for a fast function easily clears 10 requests per second in practice (a 20 ms handler run continuously on one environment serves 50 requests per second, not 10).

What Lambda actually enforces alongside the concurrency quota is a separate REQUEST-RATE quota: RPS must stay at or below 10 times whichever concurrency quota is in force (10,000 RPS against the default 1,000-unit account quota; 10 times a function's reserved concurrency if it has one; 10 times its provisioned concurrency for the units running on it). This is a policy ceiling tied to the quota number, not a physical property of any one environment. For a function whose average duration is at or above 100 ms, Little's law alone already keeps the request rate inside that 10x ceiling, so it never binds. It matters specifically for the sub-100-ms case: a 20 ms function serving 30,000 RPS needs a concurrency, and so an environment count, of only 600, comfortably under a 1,000-unit account quota, but its request rate of 30,000 is three times the 10,000 RPS the default quota allows, so it throttles on the rate ceiling even though the environment count looks fine.

with a second, independent check for functions under about 100 ms average duration:

The limits

The effect of a throttle depends on who called the function:

• Synchronous callers (API Gateway, a direct invoke) get the error immediately. Nothing retries for you.
• Asynchronous sources (S3 notifications, SNS, Amazon Simple Notification Service, a managed pub/sub messaging service) go through Lambda's internal queue, which retries throttled events for up to 6 hours by default.
• Queue pollers (Amazon SQS, Simple Queue Service) leave the messages in the queue to be retried.

Reserved versus provisioned concurrency

If both are set, provisioned concurrency cannot exceed reserved concurrency. Setting reserved concurrency to 0 is the documented way to stop a function completely.

Worked example

Assume an endpoint whose handler takes 120 ms. It normally sees 100 RPS with 12 environments warm, and the account quota is the default 1,000.

# Environments needed = RPS x duration (Little's law: this alone gives the concurrency,
# i.e. the environment count). The separate 10x-quota request-rate ceiling only binds
# for average durations under about 100 ms, which this 120 ms example never reaches.
duration_s = 0.120      # assumed average handler duration for the endpoint
bucket = 1000           # scaling rate: up to 1,000 new environments per function...
refill_per_s = 100      # ...refilled continuously (1,000 per 10 s)
warm_now = 12

def report(rps, account_limit):
    need = rps * duration_s
    cold = max(0, min(need, account_limit) - warm_now)
    wait = max(0, cold - bucket) / refill_per_s
    short = max(0, need - account_limit)
    print(f"{rps:>6} RPS, quota {account_limit:>5}: need {need:>5.0f}; new (cold) environments {cold:>5.0f}; "
          f"seconds beyond the first 1,000 new: {wait:>4.1f}; throttled capacity gap: {short:.0f}")

for rps in (100, 5_000, 15_000):
    report(rps, 1_000)          # default Regional concurrency quota
report(15_000, 3_000)           # after a quota increase

pc_units, mem_gb, price_pc = 600, 1.0, 0.0000041667   # USD per GB-second, us-east-1, x86
hour = pc_units * mem_gb * 3600 * price_pc
print(f"provisioned concurrency {pc_units} x {mem_gb} GB: ${hour:.2f}/hour, ${hour * 730:,.0f}/month if always on")

   100 RPS, quota  1000: need    12; new (cold) environments     0; seconds beyond the first 1,000 new:  0.0; throttled capacity gap: 0
  5000 RPS, quota  1000: need   600; new (cold) environments   588; seconds beyond the first 1,000 new:  0.0; throttled capacity gap: 0
 15000 RPS, quota  1000: need  1800; new (cold) environments   988; seconds beyond the first 1,000 new:  0.0; throttled capacity gap: 800
 15000 RPS, quota  3000: need  1800; new (cold) environments  1788; seconds beyond the first 1,000 new:  7.9; throttled capacity gap: 0
provisioned concurrency 600 x 1.0 GB: $9.00/hour, $6,570/month if always on

What the numbers mean:

• Burst to 5,000 RPS. You need 600 environments. The scaling rate is not the problem: its token bucket starts full at 1,000 (see above), so all 588 new environments can be created immediately from that reserve, with no need to wait for the refill. The problem is that 588 requests each wait for a cold start at the same moment, and that is what p99 latency (the latency 99% of requests beat) will show. Init time depends on runtime and package size. Read it from the Init Duration Lambda logs rather than guessing.
• Burst to 15,000 RPS on the default quota. You need 1,800 environments but only 1,000 can exist. About 800/1,800, or 44%, of the load is throttled.
• Same burst after raising the quota to 3,000. The first 1,000 new environments appear immediately. The remaining 788 arrive at the refill rate of about 100 per second, so over about 7.9 seconds. Requests above capacity during those seconds are still throttled.
• Cost of pre-warming 600 x 1 GB: $9.00 an hour, or about $6,570 a month if it is left on all the time. That is why you schedule it rather than leave it on.

What this means for a latency-sensitive endpoint

1. Provisioned concurrency on the alias the API actually invokes (never on $LATEST, which cannot hold provisioned concurrency). Size it to the expected burst plus a buffer. Lambda's own guidance is +10%, so 600 becomes 660. If the burst is predictable, raise it on a schedule.
2. Reserved concurrency at or above that number. This stops other functions in the account from starving the endpoint. Without it, a noisy batch job (an unrelated function in the same account that suddenly scales hard, for a backfill or a retry storm) in the same Region can use up the shared 1,000.
3. Quota headroom requested ahead of time. Check the layers in front of Lambda too. API Gateway (AWS's managed HTTP front door) has its own default account throttle of 10,000 RPS per Region.
4. Make cold starts cheaper for the overflow. Keep the package small, create SDK clients at top level so warm invocations reuse them, and defer rarely used clients. For supported runtimes, snapshot restore (Lambda SnapStart) is a cheaper alternative to keeping environments warm.
5. For bursts beyond anything you pre-warmed, decide the behaviour explicitly. Either return a fast 429 and have clients retry with jittered backoff (each retry waits longer than the last, with a bit of randomness mixed in so many clients do not all retry at the same instant), or, if the work does not need an immediate answer, accept it into a queue.

Pitfalls

• Invoking $LATEST or the wrong alias. Provisioned concurrency sits unused, you still pay for it, and cold starts continue.
• Treating reserved concurrency as a performance feature. It warms nothing.
• Expecting target-tracking auto scaling (a controller that watches a metric and adds or removes capacity to hold it near a target value) to catch a spike. Application Auto Scaling on ProvisionedConcurrencyUtilization needs the load to last about 3 minutes (three data points) before it adds capacity. It handles slow drift, not a sudden burst.
• Forgetting the separate request-rate ceiling for sub-100-ms functions. Little's law alone gives the right environment count even for a fast function (a 20 ms function at 30,000 RPS needs only 600 environments), but request rate can still hit the platform's 10x-of-quota ceiling before concurrency does: at the default 1,000-unit quota that ceiling is 10,000 RPS, so the 30,000-RPS case above throttles despite having plenty of environment headroom.
• Scale-out has side effects downstream. 600 new environments can mean 600 new database connections. Plan the downstream protection together with the scaling.

Direct answer

Instrument every hop with OpenTelemetry (the open standard and SDKs for traces, metrics and logs), propagate one trace context through the API gateway, the functions and every message across async hops, and put a correlation ID in every log line. Keep costs bounded with low head sampling for successful traffic plus guaranteed capture of failures, cheap aggregated metrics for SLOs (service-level objectives) and anomaly detection, and logs with short hot retention and cheap archive. To debug one failed request, walk from the request ID the client saw, to the trace ID in its logs, to the trace, across the queue via its link to the consumer's span.

Requirements I am designing to

• 100 requests per second average at the gateway (259.2M requests per month), each producing a sync function call and an async message to a worker.
• p99 (the 99th-percentile latency: the value only the slowest 1% of requests exceed) latency SLO on the API; error-budget alerting (paging when the service has spent too much of the failure rate its SLO allows, rather than on every individual error; the full mechanism is below).
• Find any specific failed request within minutes, including failures that happen after the async hop.
• Observability spend should be a small fraction of compute spend, with arithmetic below.

Architecture

flowchart LR
  C[Client] --> G[API gateway]
  G --> F1[API function]
  F1 -->|message plus trace context| Q[Queue]
  Q --> F2[Worker function]
  F2 --> D[Downstream API or DB]
  F1 -.spans, EMF metrics, logs.-> T[Telemetry pipeline]
  F2 -.spans, EMF metrics, logs.-> T
  T --> TR[Trace store]
  T --> M[Metrics store]
  T --> L[Log store plus archive]

Components

• Tracing: the AWS Distro for OpenTelemetry Lambda layer (AWS's own supported build of the open-source OpenTelemetry instrumentation, attached to a function without changing its code) or native AWS X-Ray (AWS's distributed tracing service) on each function. Spans (timed records of one unit of work) cover the gateway, handler, each downstream call, and the enqueue.
• Metrics: SLO metrics (requests, errors, latency histograms) emitted as structured logs in the CloudWatch embedded metric format (EMF), which the platform turns into metrics without a synchronous API call inside the function; or a Prometheus-compatible managed store (Prometheus is a widely used open-source metrics system; a managed store that speaks its query language) if the organization standardizes on it.
• Logs: structured JSON with request_id, trace_id, function_version, outcome. Kept in hot retention (the fast, queryable store, as opposed to cheap cold archive) for 14 days, then archived to object storage.
• Anomaly detection: statistical bands (a normal range computed from a metric's own recent history, so an alert fires only once it moves outside that range) on a few high-value series (error rate, p99, throughput per route), plus multi-window burn-rate alerts: an SLO like 99.9% availability allows a fixed amount of failure over its measurement window, the error budget (here, 0.1% of requests over 30 days); a burn-rate alert pages when errors are consuming that budget faster than a sustainable rate, checked over both a short window (to catch a real incident fast) and a long window (so a brief blip does not page anyone) at once, which is why it is "multi-window" (for example, page when 2% of the monthly error budget is consumed in one hour). Anomaly detection runs on aggregated metrics, never on raw traces, which keeps it cheap.

Propagating context across the async hop

The trace context (a trace ID plus parent span ID, the ID of the span that caused this one, in the W3C (World Wide Web Consortium, the standards body) traceparent header or AWS's X-Amzn-Trace-Id) must ride inside the message, because there is no HTTP connection between producer and consumer.

• SQS (Amazon Simple Queue Service) to Lambda on AWS: SQS carries the X-Ray trace header in the reserved AWSTraceHeader message system attribute, and a Lambda consumer picks it up automatically; the console shows the producer's trace linked to the consumer's.
• Other brokers (EventBridge, Kafka (Apache Kafka, a distributed event-streaming log), SNS fan-out, one SNS message delivered to every subscriber at once): inject traceparent into message attributes or headers at publish and extract it in the consumer.
• Batches: one consumer invocation may process 10 messages from 10 different traces. Model the consumer's work as its own span with span links (references from one span to other related spans that are not its direct parent or children, so one span can point back to many origins) to each message's producing span, rather than pretending it has a single parent.
• Redundant safety net (belt and braces): also put a business correlation ID (order ID, request ID) in the message body and in every log line, so the chain can be reconstructed even for unsampled traces.

Sampling strategy and the cost arithmetic

Prices: X-Ray records traces at $5.00 per million, and CloudWatch Logs standard ingestion is $0.50 per GB (US East, current list).

Decision: head-sample 5% of successful requests (the decision is made at the entry point and propagated, so a trace is complete or absent, never half-recorded), and capture 100% of errors and slow requests. Capturing all errors needs tail sampling: deciding after the trace finishes. A function cannot do that alone because it sees only its own spans, so either route spans through an OpenTelemetry Collector gateway (a standalone process that receives every function's spans and applies rules to them centrally, before they reach the trace store) running tail-sampling rules (errors, latency above the SLO threshold, specific tenants), or, if you stay on head sampling only, guarantee that every error log line carries the trace ID and full context, so unsampled failures are still debuggable from logs.

Logs: 1 KB per request is 259.2 GB per month, $129.60 of ingestion. Log at INFO for outcomes only, DEBUG sampled, errors in full.

Metrics cardinality is where observability bills explode. Each unique combination of metric name and dimension values is billed as a separate custom metric ($0.30 per metric per month for the first 10,000, $0.10 for the next tier). Adding a customer_id dimension across 10,000 customers for 3 metrics creates 30,000 metrics:

That is more than all tracing at 5%. Per-customer analysis belongs in logs or traces (queried on demand), not metric dimensions.

Where telemetry is stored

Debugging one specific failed request

1. Client reports failure and returns the request_id from the error response (always return it).
2. Log query on request_id finds the API function's line and its trace_id.
3. Open the trace: gateway span, handler span, enqueue span, status OK. The failure is after the hop.
4. Follow the link from the enqueue span to the worker's span (automatic for SQS to Lambda; via span links for batches).
5. The worker span shows a 5 s downstream timeout; its log lines, filtered by the same trace_id, show the payload that triggered it; the message is in the dead-letter queue, ready to redrive (resend through the same processing path for another attempt) after the fix.

If the request was not in the 5% sample and tail sampling is not deployed, steps 3 and 4 fall back to logs by trace_id and correlation ID, which is why the log discipline is non-negotiable.

Pitfalls

• Head sampling only, then discovering the failure you need was in the 95% dropped.
• Losing trace context at the queue because the producer never injected it.
• High-cardinality metric dimensions.
• Synchronous telemetry exports inside the handler that add latency and billed duration to every request.

Direct answer

First confirm the spikes really are cold starts: compare p99 (the 99th-percentile latency, the value only 1% of requests exceed) for invocations whose logs carry an Init Duration against those that don't. If they are, three practical strategies, from cheapest to most expensive: (1) make initialization cheaper (smaller package, fewer and lazier imports, more memory for more CPU); (2) restore from a snapshot instead of initializing (AWS Lambda SnapStart); (3) pre-initialize environments so users never hit a cold one (provisioned concurrency, or minimum instances on other platforms). Do (1) always, then pick (2) or (3) based on runtime support and how strict the latency target is.

Why cold starts land exactly in p99

AWS reports cold starts are typically under 1% of invocations. p99 is the boundary of the slowest 1%. If 0.5% of requests are cold, the slowest 1% is half cold and half slow-warm, and p99 sits at the edge of the warm tail. If 1.5% are cold (low traffic, frequent deploys, bursts that force new environments), the p99 value itself is a cold request. A modest rise in cold-start rate can therefore move p99 from roughly "slow warm request" to "full cold start". This is why the fix list starts with cutting the cold-start cost, and why reducing how often they happen also counts.

Strategy 1: make initialization cheaper

Mechanism. Cold-start time is roughly code download + runtime start + your init code. Shrinking any term shrinks the spike.

• Import only what you use (a single SDK service client rather than the whole SDK); remove unused dependencies, test fixtures and docs from the package.
• Lazy-load heavy objects needed only on some code paths.
• Raise memory: Lambda allocates CPU in proportion to memory (1,769 MB equals one full vCPU, one virtual CPU core, a unit of allocated compute), so CPU-bound init (parsing, class loading, JIT warm-up, where JIT is just-in-time compilation of hot code) runs faster.

Trade-offs.

Strategy 2: snapshot restore (Lambda SnapStart)

Mechanism. When you publish a function version, Lambda runs Init once, takes a snapshot of the initialized environment's memory and disk (a Firecracker microVM snapshot, Firecracker being the lightweight virtual machine technology Lambda runs on), and caches it. New environments resume from the snapshot instead of re-running Init.

Trade-offs.

Strategy 3: pre-initialized capacity (provisioned concurrency)

Mechanism. You tell Lambda to keep N environments initialized for a version or alias at all times. Requests up to N concurrent never see a cold start; above N, they spill over (run on ordinary on-demand environments instead of the pre-warmed ones) to normal on-demand environments (and can be cold). The equivalents elsewhere are minimum instances on Google Cloud Run and always-ready instances on Azure Functions.

Trade-offs, with numbers. Price in US East (N. Virginia): $0.0000041667 per GB-second for keeping capacity provisioned, and duration drops to $0.0000097222 per GB-second (from $0.0000166667) for work that runs on it.

• 10 environments at 1 GB for a 30-day month: 10 GB x 2,592,000 s = 25,920,000 GB-s x $0.0000041667 = $108.00 per month, even with no traffic.
• If those environments serve 5 million requests of 80 ms at 1 GB (400,000 GB-s), duration costs $3.89 instead of $6.67 on-demand. The $108 floor dominates unless utilization is high.

Worked example: choosing

A checkout API on Node.js with a 250 ms p50 (median latency: half of requests finish faster than this) and 1.8 s p99, 40 requests/s in business hours, 2 requests/s at night; cold-starting requests show ~1.5 s Init Duration.

1. Trim the bundle and replace a whole-SDK import with single clients: Init Duration falls from ~1.5 s to, say, ~600 ms (measure it; don't assume).
2. SnapStart is not available for Node.js, so for the remaining spikes use provisioned concurrency. By Little's law (concurrency equals arrival rate times how long each request takes; here the 250 ms p50 stands in for that per-request time), 40 req/s x 0.25 s means about 10 requests are in flight at once, so provision ~12 (headroom) during business hours via a schedule and ~2 at night. Priced at the same $0.0000041667 per GB-second as above (1 GB, roughly 12 business hours and 12 night hours a day): 12 environments busy 12 h/day plus 2 environments the other 12 h/day, over a 30-day month, is about $75.60/month, cheaper than holding a flat 10 environments 24/7 ($108, from the pricing above) precisely because it drops to 2 overnight.
3. Re-check p99 and the spillover metric after a week.

Pitfalls

• "Keep-warm" pings (a scheduled event calling the function every few minutes) keep one environment warm. A burst that needs 20 environments still cold-starts 19. Provisioned concurrency is the supported version of this idea.
• Blaming cold starts for everything. Slow downstream calls, network set-up for functions attached to a VPC (Virtual Private Cloud, a private network), or large payloads also show up in p99. Split latency by cold versus warm before paying for a fix.
• Deploy frequency matters. Every deployment replaces all environments, so ten deploys a day means ten waves of cold starts.

Direct answer

Give the function its own execution role (the IAM role it runs as, where IAM is AWS Identity and Access Management). That role allows exactly four things:

• read objects under one Amazon S3 (object storage) bucket prefix
• read items from one table
• read one secret
• decrypt with that secret's KMS (AWS Key Management Service) key, and only when the request comes through Secrets Manager. Secrets Manager encrypts every secret at rest with a KMS key, so even though the code only calls secretsmanager:GetSecretValue, Secrets Manager in turn needs the caller allowed to decrypt with that key, or the call fails

Scope each permission to a specific resource ARN (Amazon Resource Name, the unique id of an AWS resource), not *. Wrap every role your deployment pipeline creates in a permissions boundary: a ceiling policy attached to the role that caps what it can ever be granted, so even a mistaken policy attached later still cannot grant more than the ceiling allows.

For secrets, fetch once per execution environment (the sandboxed instance of your function that a cold start creates, and that later requests reuse while it stays warm) and cache in memory with a short TTL (time-to-live), either through the Lambda secrets extension or a library cache. That way warm invocations (requests that land on an already-running environment instead of triggering a new cold start) never pay for a fetch. Make rotation safe for cached copies by using the alternating-users rotation strategy plus refresh-on-authentication-failure.

Structuring the roles

An identity policy is attached to a principal, here the function's role, and says what that principal may call. A resource-based policy is attached to the resource itself, here the function, the secret, or the KMS key, and says who may reach that resource. A call is only allowed when both sides agree.

Effective permission is the intersection of the identity policy, the boundary, and any organization-level service control policies (guardrails set above the account, by the AWS Organization, that cap what any role in the account can ever do no matter what its own policy allows). An explicit deny anywhere wins.

(Each statement below has a Sid, a statement id: just a human-readable label. AWS ignores it when deciding access; only Effect, Action, Resource and Condition do.)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadInputObjects",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::orders-input/incoming/*"
    },
    {
      "Sid": "ReadCustomerLookupTable",
      "Effect": "Allow",
      "Action": ["dynamodb:GetItem", "dynamodb:BatchGetItem"],
      "Resource": "arn:aws:dynamodb:us-east-1:111122223333:table/customer-lookup"
    },
    {
      "Sid": "ReadOnePartnerCredential",
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "arn:aws:secretsmanager:us-east-1:111122223333:secret:prod/order-enricher/partner-api-??????"
    },
    {
      "Sid": "DecryptOnlyViaSecretsManager",
      "Effect": "Allow",
      "Action": "kms:Decrypt",
      "Resource": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
      "Condition": {
        "StringEquals": { "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com" }
      }
    }
  ]
}

Details that separate a real least-privilege policy from a plausible one:

• Read-only verbs. A verb is IAM's term for the action itself, the part of the permission that says what you may do, as opposed to the resource it applies to. No PutObject, PutItem or Scan. GetItem and BatchGetItem cover key lookups. Add Query only if the code does range reads.
• Secret ARN ends in -??????. Secrets Manager appends six random characters to secret ARNs. ?????? matches exactly those six and nothing longer, while a * would also match partner-api-admin-....
• kms:ViaService means the function can use the key only through Secrets Manager, not decrypt arbitrary data with it. If the bucket uses SSE-KMS (server-side encryption with a KMS key), add a second decrypt statement scoped to S3.
• Logs: the basic execution permissions, scoped to this function's log group. A function inside a VPC (Virtual Private Cloud, a private network) also needs the network-interface permissions.
• Multi-tenant tables: add the dynamodb:LeadingKeys condition (a restriction on the table's partition key, the attribute DynamoDB uses to decide which physical partition an item lives on and the first part of every lookup) so the role can only read partition keys belonging to the caller's tenant.
• Checking for unused grants: generate a policy from recorded CloudTrail (AWS's API audit log) activity with IAM Access Analyzer. Its unused-access findings flag permissions the function has but never uses.

Secret caching without the cold-start penalty

Where the cost comes from. Calling Secrets Manager inside the handler adds a network round trip to every invocation and an API charge per call. The fix is to fetch once per execution environment and reuse the value across warm invocations.

Two supported ways to do it:

• The AWS Parameters and Secrets Lambda Extension, added as a layer (a packaged bundle of code or binaries a function attaches without bundling it into its own deployment package). The function calls http://localhost:2773/secretsmanager/get?secretId=... with the header X-Aws-Parameters-Secrets-Token set to the function's session token (the temporary credential Lambda issues alongside the execution role's access key, proving to the local extension that the caller really is this running function). It caches for 300 s by default (configurable 0 to 300 s through SECRETS_MANAGER_TTL), holding up to 1,000 secrets.
• Powertools for AWS Lambda parameters with a max_age. Or a small module-level cache of {value, fetched_at}.

When the fetch happens. Pre-warmed environments (provisioned concurrency) run init code before any request, so fetching at init moves the cost off the request path entirely. With snapshot restore (SnapStart), anything fetched during init is baked into the snapshot and shared by every restored environment. So fetch after restore, not during init.

Never put the secret in an environment variable (readable by anyone with lambda:GetFunctionConfiguration), in /tmp, or in logs. Note that AWS's own sample code prints the retrieved secret. Remove that line.

Cost:

price_per_call = 0.05 / 10_000          # Secrets Manager API calls, USD
invocations_per_s, envs, ttl_s = 200, 50, 300
month_s = 30 * 24 * 3600

no_cache = invocations_per_s * month_s
cached = envs * (month_s / ttl_s)        # worst case: every environment refreshes every TTL
print(f"fetch per invocation: {no_cache:,.0f} calls/month = ${no_cache * price_per_call:,.2f}")
print(f"cache per environment, {ttl_s} s TTL, {envs} environments: {cached:,.0f} calls/month = ${cached * price_per_call:,.2f}")

fetch per invocation: 518,400,000 calls/month = $2,592.00
cache per environment, 300 s TTL, 50 environments: 432,000 calls/month = $2.16

At 200 invocations per second, fetching per invocation costs about $2,592 a month in API calls alone (plus the latency). Caching per environment with a 300 s TTL, even assuming all 50 environments refresh every 5 minutes, costs $2.16.

Rotation that is safe for cached values

Secrets Manager rotation moves three staging labels between versions of the secret: AWSCURRENT marks the version callers get by default, AWSPENDING marks the new version created for an in-progress rotation and not yet promoted, and AWSPREVIOUS marks the version that was AWSCURRENT immediately before the last rotation.

• Single-user rotation changes the password of the one database user in place. Every environment holding the old value in its cache then fails authentication for up to one TTL (up to 5 minutes).
• Alternating-users rotation keeps two users. Rotation changes the password of the inactive user and then makes it AWSCURRENT. The previous credential stays valid until the next rotation, so cached copies keep working. Use this.
• Refresh on authentication failure: if a call fails with an auth error, bypass the cache (for example Powertools' force_fetch=True, or a direct SDK call), fetch AWSCURRENT again, and retry once.
• TTL choice: with alternating users, the TTL no longer has to be short for rotation's sake. It is bounded by how quickly you must stop using a compromised secret. For an emergency revoke, rotate and then publish a new function version or move an alias (a named, mutable pointer to one specific version, used so callers reference a stable name while the version behind it changes) to recycle the environments: pointing callers at a new version or alias forces Lambda to start fresh execution environments for it rather than reuse existing warm ones, so any environment still holding the old cached secret in memory simply stops receiving traffic, rather than waiting out the cache TTL.

Pitfalls

• One shared role for "all the app's functions". One compromised function gets every permission.
• secretsmanager:GetSecretValue on *, or kms:Decrypt with no conditions.
• Caching forever with no TTL, combined with single-user rotation: an outage every rotation.
• Fetching during snapshot init, or logging the secret "just for debugging".