Amazon FSx

Overview

Amazon FSx launches third-party high-performance file systems on AWS as a fully managed service. FSx is to file systems what RDS is to databases (RDS launches MySQL or PostgreSQL on AWS). Know these four:

• FSx for Windows File Server
• FSx for Lustre
• FSx for NetApp ONTAP
• FSx for OpenZFS

More may appear over time; the instructor only updates if significant file systems show up on the exam.

FSx for Windows File Server

A fully managed Windows File Server share drive.

• Protocol: SMB and Windows NTFS.
• Microsoft Active Directory integration for user security, plus ACLs and user quotas.
• Can be mounted on Linux EC2 instances too.
• With an existing on-premises Windows File Server, use Microsoft Distributed File System (DFS) to group file systems and join FSx to the on-premises server.
• Scales to tens of GB/s, millions of IOPS, hundreds of PB of data.
• Storage options:
  • SSD: latency-sensitive workloads such as databases, media processing, data analytics.
  • HDD: broad range of workloads, cheaper, e.g. home directories or CMS.
• Reachable from on premises over a private connection.
• Can be Multi-AZ for high availability.
• Data is backed up daily to S3.

FSx for Lustre

Lustre is a distributed file system for large-scale computing. The name derives from Linux + cluster.

• Use cases: video processing, financial modeling, electronic design automation.
• Scales to hundreds of GB/s, millions of IOPS, sub-millisecond latency.
• Storage options:
  • SSD: low-latency, IOPS-intensive workloads, small and random file operations.
  • HDD: throughput-intensive workloads, large and sequential file operations.
  • SSD is more expensive than HDD.
• Seamless S3 integration: read S3 as a file system through FSx, and write computation output back to S3. The exam may ask about this.
• Usable from on-premises servers via VPN or Direct Connect.

Deployment options

Both can have optional underlying S3 buckets as data repository. FSx for Lustre lives in a single AZ; compute instances in AZ1 and AZ2 connect to it. Scratch holds one copy of the data, persistent holds two copies.

FSx for NetApp ONTAP

A managed NetApp ONTAP file system on AWS.

• Protocols: NFS, SMB, iSCSI.
• Use it to move workloads already running on ONTAP or a NAS on premises into AWS.
• Broad compatibility: Linux, Windows, macOS, VMware Cloud on AWS, WorkSpaces, AppStream, EC2, ECS, EKS.
• Storage shrinks or grows automatically.
• Snapshots, replication, low cost, data compression, data de-duplication.
• Point-in-time instantaneous cloning: useful for testing new workloads, e.g. clone a file system into a staging one.

FSx for OpenZFS

A managed OpenZFS file system on AWS.

• Protocol: NFS only (multiple versions).
• Use it to move workloads already running on ZFS to AWS.
• Compatible with Linux, Mac, Windows and more.
• Scales to 1 million IOPS with under 0.5 ms latency.
• Snapshots, compression, low cost, but no data de-duplication.
• Point-in-time instantaneous cloning for testing new workloads.

Comparison

Hands-On Walkthrough

1. Open the FSx console and click Create file system. Choose among the four options.
2. FSx for Lustre: enter file system details, deployment and storage type, throughput, VPC placement and encryption. Know that Lustre has Persistent and Scratch options.
3. FSx for Windows File Server: accessed via SMB. Options: Multi-AZ or Single-AZ (Single-AZ is good for dev), SSD or HDD, storage capacity (which determines the capacity options offered; the lecture audio is unclear on the next setting, likely throughput capacity), VPC, Windows authentication through AWS-managed Active Directory or a self-managed Microsoft Active Directory, encryption, auditing, backup and maintenance.
4. FSx for NetApp ONTAP: compatible with Linux, Windows and macOS. Options: Multi-AZ or Single-AZ, storage type and capacity, storage efficiency (deduplication, compression, compaction). There is a quick create and a standard create.
5. FSx for OpenZFS: ZFS file system compatible with Linux, Windows and macOS.

From an exam perspective, know the high-level differences between the four.

Questions & Answers