Amazon EFS

Home » AWS Cheat Sheets » Amazon EFS
Amazon EFS Services

Amazon EFS is a fully managed service for hosting Network File System (NFS) filesystems in the cloud.

It is an implementation of a NFS file share and is accessed using the NFS protocol.

It provides elastic storage capacity and pay for what you use (in contrast to Amazon EBS with which you pay for what you provision).

You can configure mount-points in one, or many, AZs.

You can mount an AWS EFS filesystem from on-premises systems ONLY if you are using AWS Direct Connect or a VPN connection.

Typical use cases include big data and analytics, media processing workflows, content management, web serving, home directories etc.

Uses a pay for what you use model with no pre-provisioning required.

AWS EFS can scale up to petabytes.

AWS EFS is elastic and grows and shrinks as you add and remove data.

You can concurrently connect up to thousands of Amazon EC2 instances, from multiple AZs.

A file system can be accessed concurrently from all AZs in the region where it is located.

The following diagram depicts the various options for mounting an EFS filesystem:

Amazon EFS File System

Access to AWS EFS file systems from on-premises servers can be enabled via AWS Direct Connect or AWS VPN.

You mount an AWS EFS file system on your on-premises Linux server using the standard Linux mount command for mounting a file system via the NFS protocol.

The Amazon VPC of the connecting instance must have DNS hostnames enabled.

EFS provides a file system interface, file system access semantics (such as strong consistency and file locking).

Data is stored across multiple AZs within a region.

Read after write consistency.

Need to create mount targets and choose AZs to include (recommended to include all AZ’s).

Instances can be behind an Elastic Load Balancer (ELB).

Amazon EFS is compatible with all Linux-based AMIs for Amazon EC2.

Using the EFS-to-EFS Backup solution, you can schedule automatic incremental backups of your Amazon EFS file system.

The following table provides a comparison of the storage characteristics of EFS vs EBS:

 Amazon EFSAmazon EBS Provisioned IOPS
Availability and durabilityData is stored redundantly across multiple AZsData is stored redundantly in a single AZ
AccessUp to thousands of Amazon EC2 instances, from multiple AZs, can connect concurrently to a file systemA single Amazon EC2 instance in a single AZ can connect to a file system
Use casesBig data and analytics, media processing and workflows, content management, web serving and home directoriesBoot volumes, transactional and NoSQL databases, data warehousing and ETL

Backups and Lifecycle Management

Automatic backups are enabled by default and use AWS Backup.

Lifecycle management moves files that have not been accessed for a period of time to the EFS Infrequent Access Storage class.

Amazon EFS Performance

There are two performance modes:

  • “General Purpose” performance mode is appropriate for most file systems.
  • “Max I/O” performance mode is optimized for applications where tens, hundreds, or thousands of EC2 instances are accessing the file system.

Amazon EFS is designed to burst to allow high throughput levels for periods of time.

There are two throughput modes:

  • “Bursting” – throughput scales with file system size.
  • “Provisioned” – Throughput is fixed at the specified amount.

Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow elastically to petabyte scale and allowing massively parallel access from Amazon EC2 instances to your data.

This distributed data storage design means that multithreaded applications and applications that concurrently access data from multiple Amazon EC2 instances can drive substantial levels of aggregate throughput and IOPS.

The table below compares high-level performance and storage characteristics for AWS’s file  (EFS) and block (EBS) cloud storage offerings:

 Amazon EFSAmazon EBS Provisioned IOPS
Per-operation latencyLow, consistent latencyLowest, consistent latency
Throughput scale10+ GB per secondUp to 2 GB per second

Amazon EFS Encryption

EFS offers the ability to encrypt data at rest and in transit.

Encryption keys are managed by the AWS Key Management Service (KMS).

Encryption in transit:

  • Data encryption in transit uses Transport Layer Security (TLS) 1.2 to encrypt data sent between your clients and EFS file systems.
  • Encryption in transit is enabled when mounting the file system.

Encryption at rest:

  • Enable encryption at rest in the EFS console or by using the AWS CLI or SDKs.
  • Encryption at rest MUST be enabled at file system creation time.
  • Data encrypted at rest is transparently encrypted while being written, and transparently decrypted while being read.

Encryption of data at rest and of data in transit can be configured together or separately.

Amazon EFS Access Control

When you create a file system, you create endpoints in your VPC called “mount targets”.

When mounting from an EC2 instance, your file system’s DNS name, which you provide in your mount command, resolves to a mount target’s IP address.

You can control who can administer your file system using IAM (user-based and resource-based policies)

You can control the NFS clients that can access your file systems (resource-based policies).

You can control access to files and directories with POSIX-compliant user and group-level permissions.

POSIX permissions allow you to restrict access from hosts by user and group.

EFS Security Groups act as a firewall, and the rules you add define the traffic flow.

Monitoring and Reporting

The Amazon EFS console shows the following monitoring information for your file systems:

  • The current metered size.
  • The number of mount targets.
  • The lifecycle state.

Amazon EFS reports metrics for Amazon CloudWatch.  A few useful metrics are:

  • TotalIOBytes – use the daily Sum statistic to determine throughput.
  • ClientConnections – use the daily Sum statistic to track the number of connections from EC2 instances.
  • BurstCreditBalance – monitor the burst credit balance.

Logging and Auditing

Amazon EFS is integrated with AWS CloudTrail.

CloudTrail captures all API calls for Amazon EFS as events, including calls from the Amazon EFS console and from code calls to Amazon EFS API operations.

Related posts: