The Mathematics of Concurrency: Poisson Distributions in Serverless Scaling
Serverless functions (like AWS Lambda or Google Cloud Functions) scale horizontally to handle concurrent traffic. When a function is triggered, the platform assigns it to an active, warm container. If all existing containers are busy processing other requests, the platform must provision a new container on the fly, triggering a cold start. Sizing this risk requires modeling request arrivals as a Poisson process.
In queueing theory, request arrivals to a web server are modeled using the Poisson distribution, which predicts the probability of receiving a specific number of concurrent requests within a given execution time window. The average number of concurrent requests (concurrency load) is: λ = R_(ingress) × ( T_(duration)/1000 ) where R_(ingress) is the incoming requests per second (RPS), and T_(duration) is the average function execution duration in milliseconds.
To optimize your overall serverless deployment, you can evaluate monthly compute bills with the serverless compute cost calculator or plan host node allocations using the Kubernetes capacity planner. Properly sizing warm container pools is key to maintaining low tail latencies.
The probability that exactly k concurrent requests arrive simultaneously is given by the Poisson Probability Mass Function (PMF): P(X = k) = (λ^k e^(-λ))/k! If we configure a provisioned concurrency pool of c warm containers, a cold start is triggered whenever the concurrent requests exceed this pool. The probability of experiencing at least one cold start is: P(Cold Start) = P(X > c) = 1 - Σ_(k=0)^(c) (λ^k e^(-λ))/k!
Let's calculate this probability for an API receiving 20 RPS, with an average function execution duration of 250 ms, and a provisioned concurrency pool of 2 warm containers. The average concurrency load is: λ = 20 × (250 / 1000) = 5.0 concurrent requests. The probability of experiencing a cold start (concurrent requests exceeding 2) is: P(X > 2) = 1 - ( e^(-5) ( 1 + 5 + 25/2 ) ) = 1 - ( 0.006738 × 18.5 ) = 1 - 0.1246 = 87.54% showing that even with 2 warm containers, 87.5% of requests will experience cold starts due to queue saturation.