15 - Containers on AWS ECS Fargate ECR EKS

Amazon ECS - Solutions Architectures

awscontainersecseventbridgesqssaa-c03

Overview

Four architectures you may encounter with ECS. See 02-Amazon-ECS and 03-ECS-Auto-Scaling.

1. ECS Tasks Invoked by EventBridge

  1. An ECS cluster is backed by Fargate; users upload objects into S3 buckets.
  2. The S3 buckets send events to Amazon EventBridge.
  3. An EventBridge rule runs ECS tasks on the go.
  4. Each task has an ECS task role; it gets the object from S3, processes it and sends results to DynamoDB.

Result: a serverless architecture to process objects from S3 with a Docker container (EventBridge, ECS on Fargate, task role for S3 and DynamoDB).

S3 upload to ECS task via EventBridge

Users S3 bucketobject upload EventBridgerule ECS cluster on Fargate ECS taskECS task role DynamoDB task gets the object from S3 Serverless: process S3 objects with a Docker container.
Serverless processing of S3 objects with a Docker container.

2. ECS Tasks Invoked by an EventBridge Schedule

  • EventBridge triggers a rule every 1 hour, which runs a new ECS task in the Fargate cluster.
  • With a task role that has S3 access, the container can do batch processing on S3 files every hour.
  • Fully serverless.

3. ECS Service with an SQS Queue

  • A service on ECS with two tasks polls messages from an SQS queue and processes them.
  • Enable ECS Service Auto Scaling: the more messages in the queue, the more tasks in the service.

ECS service polling SQS

SQS queuemessages tasks poll messages ECS service Task Task ECS Service Auto Scaling The more messages in the queue, the more tasks in the service.
Service Auto Scaling adds tasks as messages build up.

4. EventBridge Intercepting ECS Events

  • Any task exiting or starting in the cluster can be an event in EventBridge, e.g. ECS task state change for stopped with the stopped reason.
  • Example: alert an SNS topic to email administrators.
  • EventBridge lets you understand the lifecycle of your containers in the cluster.

Questions & Answers

Q1: How do you process S3 uploads serverlessly with a container?

Answer:

S3 events to EventBridge, a rule runs an ECS task on Fargate, and the task role lets it read S3 and write to DynamoDB.

Q2: How do you run a container every hour?

Answer:

An EventBridge schedule rule that runs an ECS task on Fargate.

Q3: How do you scale ECS tasks on queue depth?

Answer:

Poll an SQS queue from the service and enable ECS Service Auto Scaling.

Q4: How do you get notified when a task stops?

Answer:

Use the EventBridge ECS task state change event and target an SNS topic.

Check yourself

Quiz

Flashcards

Click the card (or focus it), then use ← → to move and Space to flip.