13 - AWS Storage Extras

AWS Storage Gateway

awsstorage-gatewayhybrid-cloudfile-gatewayvolume-gatewaytape-gatewaysaa-c03

Overview

Hybrid cloud means part of the infrastructure is on AWS and part stays on premises. Reasons: a long cloud migration, security requirements, or a strategy to use the cloud only for elastic workloads.

S3 is a proprietary storage technology, unlike EFS which is an NFS-compliant file system. To expose S3 data on premises you need a bridge: AWS Storage Gateway.

Cloud-native storage options on AWS:

  • Block: Amazon EBS, EC2 Instance Store
  • File: Amazon EFS, Amazon FSx
  • Object: Amazon S3, Amazon Glacier

Storage Gateway is a bridge between on-premises data and cloud data. Use cases:

  • Disaster recovery: back up on-premises data to the cloud
  • Backup and restore
  • Cloud migration / extend storage to the cloud, with colder data in the cloud and warmer, frequently used data on premises
  • On-premises cache for low-latency file access, with most data stored on AWS

Four gateway types: S3 File Gateway, FSx File Gateway, Volume Gateway, Tape Gateway.

AWS update (2024-09)

FSx File Gateway is no longer available to new customers (2024). The course lists it as one of four gateway types. AWS points to direct access to FSx for Windows File Server. The exam may still use the course figure. AWS source


Amazon S3 File Gateway

  • Connects an on-premises application server to an S3 bucket using NFS or SMB. The gateway translates requests into HTTPS calls to S3. The application sees a normal file share.
  • Supported S3 storage classes: S3 Standard, Standard-IA, One Zone-IA, Intelligent-Tiering, but not Glacier. Use an S3 lifecycle policy to transition objects to Glacier for archive.
  • Most recently used data is cached in the gateway for rapid access, not the whole bucket.
  • Access to the bucket requires IAM roles for each file gateway.
  • With SMB, integrates with Active Directory for user authentication.

Amazon FSx File Gateway

  • Native access to FSx for Windows File Server from SMB clients in your corporate data center.
  • FSx for Windows File Server is already reachable from on premises, so the reason to add the gateway is a local cache of frequently accessed data with low latency.
  • Windows-native compatibility: SMB, NTFS, Active Directory.
  • Useful for group file shares and home directories.

Volume Gateway

  • Block storage over iSCSI, backed by Amazon S3.
  • Volumes are backed up as EBS snapshots, which can restore on-premises volumes.
  • Two types:
    • Cached volumes: low-latency access to the most recent data.
    • Stored volumes: the entire data set is on premises, with scheduled backups to S3.
  • Goal: back up the volumes of on-premises servers.

Tape Gateway

For companies with a physical-tape backup system. The Virtual Tape Library (VTL) is backed by Amazon S3 and Glacier.

  • Back up existing data using tape-based processes over the iSCSI interface.
  • Works with leading backup software vendors.
  • Tapes are stored in S3 and can transition to Glacier and Glacier Deep Archive for archive.

Storage Gateway Hardware Appliance

The gateway must run in your corporate data center. Without virtualization, use the Storage Gateway Hardware Appliance, ordered from amazon.com.

  • A mini server with enough CPU, memory, network and SSD cache.
  • Can be set up as a File, Volume or Tape Gateway.
  • Useful for daily NFS backups in small data centers without virtualization.
AWS update (2025-05)

The Storage Gateway Hardware Appliance is no longer offered (2025-05). The course presents it as the option for sites without virtualization. Existing appliances are supported until May 2028. The exam may still use the course figure. AWS source


Architecture Summary

On premises Gateway Backed by
User group file share over NFS/SMB S3 File Gateway (local cache) S3 (all tiers except Glacier and Glacier Deep Archive; lifecycle policy can move to Glacier)
SMB clients FSx File Gateway FSx for Windows File Server, backed up to S3 once in a while
Application servers mounting iSCSI volumes Volume Gateway S3, convertible to EBS volumes to restore on AWS
Backup applications over iSCSI VTL Tape Gateway S3 as tape library, transition to Glacier and Glacier Deep Archive

A storage gateway VM or hardware appliance is deployed on premises.


Storage Gateway types

User file shareNFS or SMB S3 File Gatewaylocal cache Amazon S3not Glacier SMB clients FSx File Gatewaylocal cache FSx for WindowsFile Server Application serversiSCSI volumes Volume Gatewaycached or stored Amazon S3EBS snapshots Backup applicationsiSCSI VTL Tape Gatewayvirtual tape library S3 and GlacierDeep Archive
What each gateway exposes on premises and what backs it in AWS.

Hands-On Walkthrough

You do not need to configure a gateway for the exam; know the offerings.

  1. In the console, click Create gateway. Four types appear: S3 File Gateway, FSx File Gateway, Volume Gateway, Tape Gateway.
  2. S3 File Gateway: local cache for S3 on premises with the most recently used data. Click Next for platform options:
    • on-premises virtualization: VMware, Hyper-V, Linux KVM
    • Amazon EC2 (loses the caching benefit)
    • hardware appliance, orderable from the console
  3. FSx File Gateway: same idea, caching Windows File Server on premises. Platform options include the hardware appliance.
  4. Volume Gateway: Cached volume (low-latency access to recent data) or Stored volume (on-premises data with scheduled backups). Uses block storage in S3 with point-in-time backups as EBS snapshots.
  5. Tape Gateway: back up to S3 and archive to Glacier using existing tape-based processes, same VTL protocol.

Questions & Answers

Q1: How do you expose S3 objects to on-premises servers over NFS or SMB?

Answer:

S3 File Gateway. Recently used data is cached. Use IAM roles, and Active Directory with SMB.

Q2: Why use FSx File Gateway if FSx for Windows is reachable on premises?

Answer:

For a local cache of frequently accessed files with low latency.

Q3: Cached vs stored volumes?

Answer:

Cached: low-latency access to recent data. Stored: whole data set on premises with scheduled S3 backups.

Q4: Which gateway replaces a physical tape backup system?

Answer:

Tape Gateway (VTL over iSCSI, backed by S3 and Glacier).

Q5: No virtualization on premises?

Answer:

Order the Storage Gateway Hardware Appliance.

Check yourself

Quiz

Flashcards

Click the card (or focus it), then use ← → to move and Space to flip.