14 - Decoupling Applications SQS SNS Kinesis MQ

SQS - Message Visibility Timeout

awssqsvisibility-timeoutsaa-c03

Overview

When a consumer polls a message, it becomes invisible to other consumers. A consumer makes a ReceiveMessage request and the message is returned. The visibility timeout then begins. The default is 30 seconds, so the message has to be processed within those 30 seconds.

During the timeout, if the same or other consumers call ReceiveMessage, the message is not returned. After the timeout elapses and the message has not been deleted, it is put back into the queue, and the same or another consumer receives the same message again.

Exam Pattern

Visibility timeout scenarios come up on the exam. If a message is not processed within the window, it can be processed twice (by two consumers, or twice by the same one).

Visibility timeout timeline

ReceiveMessagemessage returned Visibility timeoutdefault 30 seconds message hidden from other consumers Deleted in timedone Not deletedback in the queue received again: processed twice
A message not deleted within the timeout returns to the queue and is received again.

ChangeMessageVisibility

If a consumer is actively processing a message and needs more time, it calls the ChangeMessageVisibility API. This tells SQS not to make the message visible yet, so it is not processed twice.

Choosing the Value

Setting Problem
Very high (e.g. hours) If the consumer crashes, the message takes hours to become visible again
Very low (e.g. a few seconds) If the consumer does not finish in time, the message is read many times by different consumers -> duplicate processing

Set a reasonable value for your application. Program consumers to call ChangeMessageVisibility when they need more time.

The queue-level default can be edited from 0 seconds (not recommended) up to 12 hours. 30 seconds is fine.

Hands-On Walkthrough

  1. Open two browser windows of Send and receive messages to act as two consumers.
  2. In window one, send hello world. The queue default timeout is 30 seconds.
  3. In window one, Poll for messages. The message appears (received).
  4. In window two, poll. The message does not appear, because it is still in the visibility window.
  5. Stop polling and do not delete. After the timeout, window two receives the message because it was put back into the queue.
  6. Delete the message to finish processing. The message shows receive count 2, i.e. it was received twice.
  7. To change the default: Edit the queue and set the visibility timeout (0 seconds up to 12 hours). For one message needing more time, call ChangeMessageVisibility instead.

Questions & Answers

Q1: What is the default visibility timeout?

Answer:

30 seconds. Configurable from 0 seconds to 12 hours.

Q2: A consumer needs more time to process a message. What should it do?

Answer:

Call the ChangeMessageVisibility API to extend the visibility window for that message.

Q3: Why might a message be processed twice?

Answer:

It was not deleted within the visibility timeout, so it returned to the queue and was received again.

Check yourself

Quiz

Flashcards

Click the card (or focus it), then use ← → to move and Space to flip.