30 days of AWS - Part 4: AWS Storage

Welcome to part 4 of 8 of the '30 days of AWS'.

Cloud storage is a critical component in cloud computing because it holds information that is accessed by your applications. Cloud storage is typically more reliable, secure, and scalable than on-premise alternatives.

AWS cloud storage offers storage options in 3 categories:

• Object storage i.e S3
• Block storage i.e EBS
• File storage i.e EFS

The most common form of storage is object storage

Cloud storage types

1. Object storage

Data is managed as individual objects.
Each object includes the data itself, the metadata, and a globally unique identifier.
The object storage service in AWS is AWS Simple Storage Service (AWS s3)

2. Block storage

Block storage breaks up data into blocks and then stores those blocks as separate pieces, each with a unique identifier.
Block storage is typically more expensive than object storage and file storage but it provides the following:

• low latency
• High and consistent performance

The block storage service in AWS is the AWS Elastic Block Storage (EBS).

3. File storage

In file storage, data is stored in files and the files are organized in folders, and the folders are organized under a hierarchy of directories and subdirectories.
The file storage service in AWS is the AWS Elastic File System (AWS EFS).

Let's delve deeper into each one of them.

AWS Simple Secure Storage (s3)

Amazon s3, as mentioned earlier, is an object-level storage option.
Amazon s3 stores data within resources called *buckets *

Amazon s3 features

• S3 is designed to deliver 99.999999999% (yap, 11 9's) durability for objects stored in the service and
• S3 supports multiple security and compliance certifications.
• You can store any type of file in s3

Use cases of S3

1. Backup and Disaster Recovery
The Cross Region Replication (CRR) feature in AWS s3 automatically replicates data across regions while maintaining maximum durability and availability.

Amazon S3 versioning feature, when enabled, automatically keeps multiple variants of an object in the same bucket. This makes it easier to recover these files or older versions.

2. Big data and analytics
S3 offers an in-place querying functionality, which is used to run analytics on data stored in S3.

3. Content distribution
S3 can be used to store static content, such as images, videos, and downloadable files. This content can be distributed globally to users by leveraging Amazon CloudFront, a content delivery network (CDN).

4. Data Archiving
Amazon S3 offers several storage class tiers, including the Amazon S3 Glacier, a durable and cost-effective archiving solution.

AWS S3 storage classes

Amazon s3 offers 7 different storage classes that are designed for different use cases.

These classes include:
a. Amazon S3 standard

• it supports frequently accessed data that requires low latency and high throughput.
• It is designed for high durability, availability, and performance
• Use cases include: hosting dynamic websites, content distribution, mobile and gaming applications, and big data analytics.

b. Amazon S3 Intelligent-Tiering

• This storage class is designed to optimize costs by automatically moving data to the most cost-effective tier, without affecting performance impact.
• S3 monitors access patterns of the objects in S3 intelligent-tiering and if objects have not been accessed for 30 consecutive days, the objects are moved to the infrequent access tier (IA).
• If an object in the IA tier is accessed, it is automatically moved back to the frequent access tier.

c. Amazon S3 standard Infrequent-Access (S3-standard-IA)

• It is used for data that is accessed less frequently but requires rapid access when needed.

d. Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)

• It is used for data that is accessed less frequently but requires rapid access when needed.
• The difference between this and S3 standard IA is that S3 One Zone-IA stores data in a single availability zone.

e. Amazon S3 Glacier Instant Retrieval

• It is a secure, durable, and low-cost storage class for data archiving.
• This storage class can retrieve objects within a few milliseconds.
• There is a trade-off to the low-cost storage: Retrieving data from the Glacier Instant Retrieval incurs a higher cost per GB compared to the s3 standard.

f. Amazon S3 Glacier Flexible Retrieval

• It is a low-cost storage designed for data archiving
• This storage class retrieves objects within a few minutes to hours.

g. Amazon S3 Glacier Deep Archive

• This is the lowest-cost storage class for Amazon S3.
• It supports long-term retention and digital preservation of data that might be accessed once or twice a year.
• All objects stored here are replicated and stored across 3 geographically dispersed AZs.
• These objects can be restored within 12 hours.

Naming conventions of S3 buckets

• Names MUST be unique across all AWS accounts; within and without your organization. In other words, there aren't 2 S3 buckets that share a name in AWS.
• Names must be 3 to 63 characters in length
• Names can only contain lowercase letters, numbers, and hyphens

Amazon S3 pricing

You pay for what you use which includes:

• GBs per month (Storage class)
• Transfer OUTs to other regions
• PUT, COPY, POST, GET, and LIST requests

Amazon Elastic Block Storage (EBS)

Amazon EBS provides persistent block storage volumes for use with Amazon EC2 instances.

Persistent storage - It is any data storage device that retains data after power to that device is shut off.

Features of Amazon EBS

• Amazon EBS is automatically replicated within its Availability Zone.
• EBS offers consistent and low-latency performance needed to run workloads.
• EBS volume data persists independently of the life of the instance attached to it.
• You can attach multiple EBS volumes to an instance.
• EBS volumes must be in the same Availability Zone as the instances they are attached to.

A backup of an Amazon EBS is called a snapshot. The first snapshot is called the baseline snapshot

Any other snapshot after the baseline snapshot captures only what is different from the previous snapshot.

Amazon EBS volume types

The volume types differ in performance characteristics and price.

Volume types
1. Solid-state drive (SSD) volumes

• Optimized for transactional workloads involving frequent read/write operations.
• They are usually preferred in cases where high Input/Output Operations per Second (IOPS) is required.
• SSD-backed volume types include:General purpose SSD and Provisioned IOPS SSD

2. Hard Disk Drive (HDD)

• Are optimized for large streaming workloads where the dominant performance attribute is throughput

What is throughput? - It is a measure of the amount of data transferred to and from a storage device per second.

• HDD volume types include: Throughput optimized HDD and Cold HDD

Use Cases of Amazon EBS

a. Boot volumes and storage for Amazon EC2 instances.
b. Database hosts
c. Data storage with a file system

Amazon Elastic File System (EFS)

EFS provides simple, scalable, and elastic file storage for use with AWS services.
EFS uses the Network File System (NFS) protocol

What is a Network File System (NFS) protocol?
It is a file system protocol that allows a user on a client computer to access files over a network the same way they would access a local storage file.

Features of Amazon EFS

• It is built to dynamically scale on demand without causing disruptions.
• It is well suited for big data and analytics, web serving, media processing workflows, and home directories.
• It can scale up to petabytes
• By default, you can create up to 10 file systems per AWS account.
• It can concurrently be connected to 1 to 1000s of EC2 instances from multiple AZs.

Use cases of EFS volumes

• Web serving and content management
• Media and entertainment
• Shared and home directories
• Database backups
• Big data analytics

Creating an EFS

An EFS is created within an Amazon Virtual Private Cloud (VPC) and must be attached to an EC2 instance within the same VPC.

The VPC, EC2 instance, and EFS itself must all reside in the same region.

The EFS file system is mounted on the VPC using the NFS versions 4.0 and 4.1

There are 2 categories of file systems:

• Regional file systems - It stores data redundantly across multiple AZs. You can create a mount target in each AZ in the region.
• One zone file systems - It stores data redundantly within a single AZ. Only a single mount target is created.

What is a mount target? - IIt is an elastic network interface that 'exposes' the EFS to your VPC. Your client accesses the EFS by connecting to the mount target(s).

Amazon EFS infrastructure

In the illustration above, the VPC spans across 3 AZs and each Availability Zone has one mount target created in it.

Accessing the file system from a mount target within the same AZ is recommended for performance and cost reasons.

One of the Availability Zones has 2 subnets. However, a mount target is created in only one of the subnets.

Contents of the file system

a. Mount target

• The mount target ID
• The subnet ID for the subnet where it was created
• The file system ID
• An IP address where the file system is mounted
• The mount target state

b. Tags

• Key-value pair

Thank You for making it to the end 🍾🍾
See you in the next one.