16 - Serverless Overviews

Lambda Concurrency

awslambdaconcurrencythrottlingsaa-c03

Overview

The more you invoke Lambda functions, the more concurrent executions you get, because Lambda scales very easily and fast. At a low scale you may have 2 concurrent executions. At a very high event scale, up to 1000 functions may run together.

Reserved Concurrency and Throttling

  • Reserved concurrency is a limit set at the function level. It is recommended.
  • Example: a function limited to 50 concurrent executions.
  • Each invocation over the limit triggers a throttle.
Invocation type Behavior on throttle
Synchronous (direct invoke) Returns throttle error 429
Asynchronous Retries automatically, then goes to the DLQ

To exceed 1000 concurrent executions at a time, open a support ticket to request a higher limit.

Why set it carefully

Scenario with no reserved concurrency: an ALB-fronted Lambda app, an API Gateway-fronted Lambda app, and an app using the SDK/CLI. A huge promotion drives many users to the ALB, which scales Lambda to 1000 concurrent executions. All of them go to the first app, so the API Gateway users and the CLI/SDK callers are throttled.

Account-wide limit

The concurrency limit applies to all functions in the account. If one function goes over the limit, other functions may get throttled.

Reserved concurrency and throttling

Account limit: 1000 concurrent executions, shared Function Areserved concurrency: 50 Function Bno reservation Function Cno reservation B and C share what is left A heavy function can throttle others Over the limit: throttle Synchronous invocation Returns error 429 Asynchronous invocation Retries automatically then goes to the DLQ Beyond 1000: open a support ticket
The account limit is shared. A reservation protects one function. Calls over the limit are throttled.

Concurrency with Asynchronous Invocations

Example: many files uploaded to S3 trigger many Lambda invocations through S3 event notifications. If the function cannot scale because the limit is reached, additional requests are throttled.

  • For throttling errors (429) and system errors (500-series), Lambda returns the event to the internal event queue.
  • Lambda retries the function for up to 6 hours.
  • The retry interval grows exponentially, from 1 second up to a maximum of every 5 minutes.

Cold Starts and Provisioned Concurrency

Cold start: when a new function instance is created, the code is loaded and the code outside the handler (initialization) runs. If init is large (lots of code and dependencies, many database connections, many SDK clients), it takes long. The first request served by a new instance has higher latency than the rest, which can hurt users (for example, a user waiting 3 seconds).

Provisioned concurrency:

  • Allocate concurrency before the function is invoked.
  • The cold start never happens and all invocations have lower latency.
  • Manage it with Application Auto Scaling, for example on a schedule or a target (the transcript wording, "target position", is garbled).
  • The transcript mentions an AWS blog (October/November 2019, the date is slightly unclear) showing improvements that dramatically reduced cold starts for Lambda in a VPC. Cold starts now have minimal impact in a VPC.

The instructor points to two diagrams in the slides that explain reserved and provisioned concurrency.

Cold start vs provisioned concurrency

Without provisioned concurrency New instance is created Code loads, init outside handler runs First request has higher latency This is the cold start With provisioned concurrency Concurrency allocated before invoke Cold start never happens All invocations have lower latency Set on a version or alias, it has a cost Managed with Application Auto Scaling
Provisioned concurrency allocates instances before invocation, so the cold start never happens.

Hands-On Walkthrough

Reserved concurrency

  1. In the function, open Configuration -> Concurrency.
  2. By default the function uses the unreserved account concurrency of 1000 for the entire account, shared by all functions.
AWS update (2025-01)

1,000 is the standard default; new accounts may start with reduced quotas that AWS raises automatically. Separately, each function scales by 1,000 execution environments every 10 s. AWS source

  1. Click Edit and reserve, for example, 20. The function then has 20 reserved and the account has 980 unreserved left for other functions.
  2. To test throttling, reserve 0. The function is always throttled. Clicking Test fails with calling the invoke API action failed because we have exceeded the rate.
  3. Switch back to unreserved account concurrency (or a specific reservation) and the test works.

Provisioned concurrency

  1. In Configuration -> Concurrency -> Provisioned Concurrency Configuration add a configuration.
  2. It applies to an alias or a version, not $LATEST (the transcript says "latest is not working"). You must publish a version first.
  3. Example: 5 provisioned. It has a cost, so choose a sensible number. It is not free. The demo did not apply it.

Questions & Answers

Q1: What happens when a synchronous invocation is throttled?

Answer:

Lambda returns 429.

Q2: What happens when an asynchronous invocation is throttled?

Answer:

It is retried (up to 6 hours, backoff from 1 s to 5 min), then goes to the DLQ.

Q3: How do you stop one function from starving others?

Answer:

Set reserved concurrency on the function. The limit is shared across the account.

Q4: How do you eliminate cold starts?

Answer:

Use provisioned concurrency (applied to a version or alias, costs money). For Java, Python and .NET also see 05-Lambda-SnapStart.

Q5: Why reserve concurrency of 0?

Answer:

It makes the function always throttled, which is useful to test throttling behavior.

Check yourself

Quiz

Flashcards

Click the card (or focus it), then use ← → to move and Space to flip.