loading...

AWS – Working with network storage provided by EFS

Create a Home page in ASP.Net Core 3.1 MVC

In this recipe, we will use Amazon EFS to provide network-based storage to instances.

Some of the benefits of using EFS compared to other AWS services are as follows:

  • There is a guaranteed write order between distributed clients.
  • There is automatic resizing – no need to pre-allocate and no need to downsize.
  • You only pay for the space you use (per GB) – there’s no transfer or extra costs.

EFS provides a file storage service that can be accessed simultaneously by many instances, similar to Network Attached Storage (NAS). While not as fast as EBS, it still provides low-latency access. Since it may be accessed by multiple clients at a time, it can reach much higher levels of throughput than EBS. EFS filesystems also scale dynamically in size and so do not need to be pre-allocated or modified during use. Filesystems are stored redundantly across AZs.

The following are some recommended use cases for EFS:

  • Home directories
  • Serving shared web content
  • Content management
EFS performance scales according to the filesystem’s size. As the filesystem’s size is not pre-allocated, the only way to increase your performance is to add more data to it.

Getting ready

This recipe works with the default VPC and subnets that are present in all AWS accounts when they are created. Even if you have changed your network configuration, all you need is a working VPC with two or more subnets in different AZs for this recipe.

How to do it…

Follow these steps to create a secure filesystem and mount points that can be attached to your EC2 instances:

  1. Open your favorite text editor and start a new CloudFormation template by defining AWSTemplateFormatVersion and Description:
AWSTemplateFormatVersion: "2010-09-09" 
        Description: Create an EFS file system and endpoints.
  1. Create a top-level Parameters section and define the required parameters, VpcId and SubnetIds, inside it:
        VpcId: 
          Description: VPC ID that contains the subnets that will 
            access the file system 
          Type: AWS::EC2::VPC::Id 
        SubnetIds: 
          Description: Subnet IDs allowed to access the EFS file system 
          Type: List<AWS::EC2::Subnet::Id>
  1. Create a top-level Resources property, which will contain all the resources we’ve defined.
  2. Under the Resources property, add the EFS FileSystem resource:
        FileSystem: 
          Type: AWS::EFS::FileSystem 
          Properties: 
            FileSystemTags: 
              - Key: Name 
                Value: 
                  Fn::Sub: "${AWS::StackName} EFS File System" 
            PerformanceMode: generalPurpose
  1. Add some mount target resources so that they can connect to the filesystem you just created:
      MountTargetA: 
        Type: AWS::EFS::MountTarget 
        Properties: 
          FileSystemId: 
            Ref: FileSystem 
          SecurityGroups: 
            - Fn::GetAtt: MountTargetSecurityGroup.GroupId 
          SubnetId: 
            Fn::Select: [ 0, Ref: SubnetIds  ] 
      MountTargetB: 
        Type: AWS::EFS::MountTarget 
        Properties: 
          FileSystemId: 
            Ref: FileSystem 
          SecurityGroups: 
            - Fn::GetAtt: MountTargetSecurityGroup.GroupId 
          SubnetId: 
            Fn::Select: [ 1, Ref: SubnetIds  ]
  1. Create a security group to control access to the mount targets:
      MountTargetSecurityGroup: 
        Type: AWS::EC2::SecurityGroup 
        Properties: 
          GroupDescription: EFS endpoint security group 
          Tags: 
            - Key: Name 
              Value: MountTargetSecurityGroup 
          VpcId: 
            Ref: VpcId
  1. Create a security group to access the mount target security group you created in the previous step:
      MountTargetAccessSecurityGroup: 
        Type: AWS::EC2::SecurityGroup 
        Properties: 
          GroupDescription: EFS endpoint access security group 
        Tags: 
          - Key: Name 
            Value: MountTargetAccessSecurityGroup 
        VpcId: 
          Ref: VpcId
  1. Define the ingress and egress rules for the mount target security group:
      MountTargetIngress: 
        Type: AWS::EC2::SecurityGroupIngress 
        Properties: 
          FromPort: 2049 
          GroupId: 
            Fn::GetAtt: MountTargetSecurityGroup.GroupId 
          IpProtocol: tcp 
          SourceSecurityGroupId: 
            Fn::GetAtt: MountTargetAccessSecurityGroup.GroupId 
          ToPort: 2049 
      MountTargetEgress: 
        Type: AWS::EC2::SecurityGroupEgress 
        Properties: 
          DestinationSecurityGroupId: 
            Fn::GetAtt: MountTargetAccessSecurityGroup.GroupId 
          FromPort: 2049 
          GroupId: 
            Fn::GetAtt: MountTargetSecurityGroup.GroupId 
          IpProtocol: tcp 
          ToPort: 2049
  1. Define the ingress and egress rules for the mount target access security group:
      MountTargetAccessIngress: 
        Type: AWS::EC2::SecurityGroupIngress 
        Properties: 
          FromPort: 22 
          GroupId: 
            Fn::GetAtt: MountTargetAccessSecurityGroup.GroupId 
          IpProtocol: tcp 
          CidrIp: 0.0.0.0/0 
          ToPort: 22 
      MountTargetAccessEgress: 
        Type: AWS::EC2::SecurityGroupEgress 
        Properties: 
          DestinationSecurityGroupId: 
            Fn::GetAtt: MountTargetSecurityGroup.GroupId 
          FromPort: 2049 
          GroupId: 
            Fn::GetAtt: MountTargetAccessSecurityGroup.GroupId 
          IpProtocol: tcp 
          ToPort: 2049
  1. Save your template with the name 03-03-NetworkStorage.yml.
  2. Launch the CloudFormation stack with the following AWS CLI command, substituting your own VpcId and SubnetIds:
      aws cloudformation create-stack \
        --stack-name wwns1 \
        --template-body file://03-03-NetworkStorage.yml \
        --parameters \
        ParameterKey=VpcId,ParameterValue=<your-vpc-id> \  
        ParameterKey=SubnetIds,ParameterValue="<subnet-id-1>\, \
         <subnet-id-2>"

How it works…

Here is what the created resources will look like at the end of this recipe:

Working with network storage

We start by creating the standard CloudFormation template properties in step 1.

In step 2, we define the template’s parameters that will be used when configuring the resources.

Steps 3 and 4 are where the EFS resources are specified. They consist of an EFS filesystem and mount targets in each of the AZs that will access it.

We then create the security groups in steps 5 and 6 – one for the mount targets, and one for the instances that are allowed to connect to the mount targets.

As these two security groups contain two-way (or circular) references to each other, we must define the rules between them in separate resources, which we did in steps 7 and 8.

In step 9, you save the template with a specific filename so that it can be referenced in the command to launch the stack in step 10.

There’s more…

To confirm that your EFS filesystem, mount targets, and security groups are working, you can also provide some client instances to connect to them. Add the following resources and parameters to the template you have already created:

  1. Add the following parameters to your top-level Parameters section to configure your instances:
      MountPoint: 
        Description: The path on disk to mount the EFS file system 
        Type: String 
        Default: /mnt/efs 
      KeyName: 
        Description: The SSH key pair allowed to connect to the client 
          instance 
        Type: AWS::EC2::KeyPair::KeyName
  1. Add an AutoScalingGroup under the Resources section. Regardless of which AZ your servers are provisioned to, they will have access to the EFS filesystem via the local mount point:
      AutoScalingGroup: 
        Type: AWS::AutoScaling::AutoScalingGroup 
        DependsOn: MountTargetA 
        Properties: 
          MinSize: 2 
          MaxSize: 2 
          LaunchConfigurationName: 
            Ref: LaunchConfiguration 
          Tags: 
            - Key: Name 
              Value: 
                Fn::Sub: "${AWS::StackName} EFS Client" 
              PropagateAtLaunch: true 
          VPCZoneIdentifier: 
            Ref: SubnetIds
  1. While still in the Resources section, add a LaunchConfiguration:
      LaunchConfiguration: 
        Type: AWS::AutoScaling::LaunchConfiguration 
        DependsOn: FileSystem 
        Properties: 
          ImageId: ami-1e299d7e 
        SecurityGroups: 
          - Ref: MountTargetAccessSecurityGroup 
        InstanceType: t2.micro 
        KeyName: 
          Ref: KeyName 
        UserData: 
          Fn::Base64: 
          Fn::Sub: |
            #!/bin/bash -xe
            mkdir -p ${MountPoint}
            echo 'Waiting for mount target DNS to propagate'
            sleep 90
            echo '${FileSystem}.efs.${AWS::Region}.amazonaws.com:/
            ${MountPoint} nfs4
            nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,
            retrans=2 0 0' >> 
            /etc/fstab
            mount -a\nchown ec2-user: ${MountPoint}\n"
  1. Launch the cloudformation stack with the following AWS CLI command, substituting your own parameter values:
      aws cloudformation create-stack \ 
        --stack-name wwns1 \
        --template-body file://03-03-NetworkStorage.yml \
        --parameters \
        ParameterKey=VpcId,ParameterValue=<vpc-id> \
        ParameterKey=SubnetIds,ParameterValue='<subnet-id-1>\, \
          <subnet-id-1>' \  
        ParameterKey=MountPoint,ParameterValue=<local-path-to-mount-efs> \
        ParameterKey=KeyName,ParameterValue=<existing-key-pair-name>

Once the new stack is ready, you will be able to Secure Shell (SSH) to your instances and verify that they have mounted the EFS filesystem.

Comments are closed.

loading...