Serverless Cold Start Latency & Concurrency Planner

When a cold start occurs, the request waits for the environment to be created, the runtime to start and your initialisation code to run. The calculator uses typical penalties of 120 ms for Go, 250 ms for Python, 350 ms for Node.js and 2,200 ms for Java; your own figure is the Init Duration AWS Lambda reports in its logs for cold invocations, and it grows with package size and start-up work.

The average latency spreads that penalty across all requests: L_average = L_warm + [ P(Cold) × L_penalty ] For the default Node.js function, 250 ms warm with a cold share of 0.0070%, the average is 250 + 0.00007 × 350 ≈ 250.02 ms. The average hides the problem: the requests that do start cold take 600 ms, and a percentile is only affected once the cold share is larger than the share of requests above it, so cold starts reach p99 only when more than 1% of requests start cold.

Chains make it worse. If a user request calls 3 functions in sequence and each starts cold on 10% of requests, the chance that at least one step is cold is: P(at least one cold) = 1 - (1 - 0.10)³ = 27.1% so low-traffic functions deep inside a chain deserve the closest look.

Provisioned Concurrency keeps a fixed number of environments initialised. Requests that land on them never start cold, but you pay for them every second whether they are busy or not.

The calculator works out the Poisson distribution of busy environments from your request rate and duration, then, for every level above your provisioned pool, how often demand reaches that level and how likely the environment kept for it has been reclaimed. Adding those up gives cold starts per second, which it scales to a 30-day month.

Two assumptions matter. First, the idle timeout: AWS does not publish how long an idle Lambda environment is kept, so treat the 10-minute default as a starting point and adjust it until the monthly cold starts match the cold invocations in your logs. Second, requests are assumed to arrive at random at a steady rate; real traffic with daily peaks and quiet nights will see more cold starts in the quiet hours than a steady average suggests, so run the calculator for your quietest hour as well as your average.

The model also ignores cold starts that platforms cause on their own, such as replacing environments during deployments and maintenance, so treat the result as a floor for steady traffic.

Because purchasing provisioned concurrency adds a fixed hourly cost ($0.015 per GB-hour on AWS Lambda), over-provisioning warm containers will quickly inflate your monthly serverless bill. Developers must design autoscaling policies that adjust provisioned concurrency dynamically based on historical traffic patterns, scaling up before peak hours and down during off-peak windows.

The target provisioned concurrency pool can be adjusted using application auto-scaling rules. Sizing these auto-scaling rules to maintain a target utilization rate (such as keeping active concurrency at 70% of provisioned concurrency) ensures that the system handles minor traffic surges without cold starts, while keeping idle container waste minimal.

To show this trade-off, the calculator prices the provisioned pool and the part of it that sits idle on average, next to the cold starts it would prevent. For a 1024 MB function running 24/7, a provisioned concurrency of 5 containers costs: Provisioned Cost = 5 × 0.015 × 24 × 30 = $54.00/month in addition to standard request fees, showing the financial threshold where dedicated container hosting becomes cheaper.

Additionally, utilizing modern lightweight runtimes (like AWS Lambda SnapStart for Java, which uses VM snapshots to initialize containers in milliseconds) offers a software-level alternative to provisioned concurrency. Sizing these snapshot initializations allows developers to achieve near-zero cold starts without paying for idle warm containers, optimizing infrastructure efficiency.

When designing serverless architectures, developers must coordinate timeouts between the API Gateway and the downstream functions. If your API gateway has a strict timeout of 10 seconds, but your Java function experiences a cold start of 5 seconds followed by a 6-second database query, the gateway will return an HTTP 504 Gateway Timeout error, breaking the transaction.

To prevent these timeout failures, developers configure function execution bounds and client retry limits. Sizing timeouts to accommodate worst-case cold start latencies (e.g., setting function timeouts to 15 seconds) ensures that the execution has time to complete. Combining this with exponential backoff retry schedules in your frontend applications allows client apps to recover from occasional cold start delays without returning error screens to users.

Use this calculator's cold request latency, runtime penalty plus warm duration, as the worst case to budget for when you set those timeouts, and its cold start share to see how often that worst case will happen.

Furthermore, monitoring active invocation histograms using metrics collectors (such as AWS CloudWatch) provides insights into execution times, enabling teams to tune provisioned concurrency parameters dynamically.

For global APIs with strict latency SLAs, deploying serverless functions across multiple geographical regions (such as US-East, EU-West, and AP-East) is a common redundancy strategy. To optimize response times, client traffic is routed to the closest regional endpoint using latency-based DNS routing.

However, multi-region setups split your traffic volume, reducing the request rate per region. A lower regional RPS increases the likelihood that execution environments will go idle and be reclaimed by the platform, leading to higher cold start rates. Sizing a coordinated multi-region warming routine (e.g., pinging all regions periodically) prevents these split-traffic cold starts, maintaining a consistent, high-performance experience worldwide.

Furthermore, deploying global databases with active-active replication ensures that write latency is minimized across all regional nodes. Sizing these database replication channels to sync changes in under 1 second prevents stale read anomalies for global clients routing between different serverless endpoints, optimizing user experience.

Choose your function's runtime (Node.js, Python, Go, or Java; each has a different default cold start penalty, or enter a custom penalty), then enter your request rate (RPS), average execution duration, provisioned concurrency, the idle timeout and the memory size of each environment.

The calculator returns the share of requests that start cold and the monthly count, the average latency and the latency of a cold request, the share of requests that arrive when every provisioned environment is busy, and the monthly cost of the provisioned pool, including the part that is idle on average.

With the default inputs of a Node.js runtime (350 ms cold start penalty), 20 RPS, 250 ms average execution duration, 2 provisioned environments, a 10-minute idle timeout and 1 GB of memory, the average concurrency is λ = 20 × (250/1000) = 5.0 busy environments.

Levels up to 14 busy environments are reached at least once a minute and stay warm. Most cold starts come from bursts to 16 to 18, which happen roughly every 5 minutes to once an hour. Together they give about 3,633 cold starts a month out of 51.84 million requests, a cold share of 0.0070%, an average latency of about 250.02 ms, and 600 ms for each request that does start cold.

The 2 provisioned environments cost $21.60 a month at $0.0000041667 per GB-second and are busy almost all the time: 95.96% of requests arrive while both are in use and run on on-demand environments, which are nearly always already warm. Raising provisioned concurrency to 15 costs $162 a month and only lowers cold starts to about 3,548, because the cold starts come from bursts above 15. For steady traffic like this, provisioned concurrency mostly buys protection for sudden jumps in traffic rather than fewer cold starts at the average rate.

Pair this with the serverless cost calculator to price out the provisioned concurrency tradeoff in dollars, and the Kubernetes capacity planner or Docker image optimizer as alternatives if cold starts are unacceptable for your workload.

For the request-path latency surrounding this function, see the API composite latency & SLA calculator.