13 - AWS Storage Extras

AWS DataSync

awsdatasyncdata-migrationsnowconesaa-c03

Overview

DataSync appears quite often on the exam. It synchronizes data, moving large amounts of data to and from places such as on premises or other clouds into AWS.

  • Connect to a source server via NFS, SMB, HDFS or other protocols. This needs an agent running on premises or on the other cloud.
  • Move data from one AWS service to another. This needs no agent.
  • Destinations: Amazon S3 (any storage class, even Glacier), Amazon EFS, Amazon FSx (all supported).
  • Replication tasks are scheduled, not continuous: hourly, daily or weekly. There is a lag.
  • Preserves file permissions and metadata (NFS POSIX, SMB permissions).
AWS update (2025-05)

DataSync Enhanced mode (2025-05) can transfer from other-cloud object storage (for example Google Cloud Storage or Azure Blob) to S3 without an agent. The course says an agent is needed for other clouds. An agent is still needed for on-premises NFS/SMB. AWS source

Exam Pattern

DataSync is the unique option that preserves metadata when moving files between locations.

  • One agent runs one task and can use up to 10 Gbit/s. Set a bandwidth limit to avoid maxing out the network.

On Premises to AWS

  1. Install the DataSync agent on premises and point it at your NFS or SMB server.
  2. The agent connects to the DataSync service in encrypted fashion.
  3. DataSync writes to S3 (any storage class), EFS or FSx.

Sync works both ways: on premises to AWS, and AWS back to on premises.

DataSync from on premises to AWS

NFS or SMB serveron premises DataSync agentup to 10 Gbit/s encrypted DataSyncservice S3 EFS FSx Scheduled hourly, daily or weekly, not continuous. Permissions and metadata kept. Sync works both ways: on premises to AWS and AWS back to on premises.
The agent reads NFS or SMB and sends data encrypted to the service, which writes to S3, EFS or FSx.

No Network Capacity: Snowcone

If the exam says to use DataSync but there is no network capacity, use AWS Snowcone. It comes with the DataSync agent pre-installed.

  1. Run Snowcone on premises. It collects your data with the DataSync agent.
  2. Ship it back to your AWS region.
  3. Data is synchronized to AWS storage resources.
AWS update (2024-11)

Snowcone was discontinued on 2024-11-12, so it is no longer the "no network capacity" answer for new customers. The course still uses it. The exam may still use the course figure. AWS source (end of support notice), AWS Snowball resources

DataSync with Snowcone

SnowconeDataSync agent pre-installed 1. collect data Ship backto your AWS region 2. sync AWS storageS3, EFS, FSx Use when the exam says DataSync but there is no network capacity.
Snowcone carries the pre-installed agent when there is no network capacity.

AWS to AWS

DataSync can sync between S3, EFS and FSx in any combination. Data and metadata are kept between the AWS storage services.

Questions & Answers

Q1: Is DataSync continuous?

Answer:

No. It is scheduled (hourly, daily, weekly).

Q2: Which service preserves metadata and permissions during a move?

Answer:

DataSync.

Q3: When is an agent needed?

Answer:

When connecting to an on-premises NFS or SMB server. Not needed between AWS services, nor (Enhanced mode) for other-cloud object storage such as Google Cloud Storage or Azure Blob.

Q4: DataSync but no network capacity?

Answer:

Course answer: Snowcone, which has the DataSync agent pre-installed (discontinued 2024-11-12 for new customers).

Check yourself

Quiz

Flashcards

Click the card (or focus it), then use ← → to move and Space to flip.