This content originally appeared on DEV Community and was authored by Harshavardhan S U
We will see about how Amazon S3 works.
S3 is based on object-based storage model, where each and everything you put into S3 is considered as an object.
An entity is considered an object if it has an Key (ID), data (blob), metadata and some attributes.
Here the Key is unique inside the bucket and metadata is data about the contents inside the object. The attributes are data about the object itself rather than the content inside.
And it seems, S3 runs on distributed systems and basically every object uploaded is copied across different nodes, said as multi AZ Replication to improve fault tolerance since even if one node fails, the data is available in remaining nodes.
The S3 follows flat storage model where there is no hierarchy in storage like we have in file storage where a folder can exist inside a folder. Here we cannot create a bucket inside a bucket. You can notice that you can upload a folder to S3, let’s see that with an example. When you upload folder with picture1.png inside it. S3 will take it’s path as its key which is images/picture1.png where “/” is just part of the object’s key. Thereby creating an illusion of folder inside S3.
Every object inside a single bucket is indexed using object’s key. When you upload a file into a bucket, the file’s name will be taken as the object’s ID.
Each operation you do in management console or AWS command line, it will fire a API call to the servers. The API is nothing but an access point for servers to utilize the server functions. The simple way for developers to create a bucket is using AWS CLI. Let’s see some commands to get a hands on experience. Please ensure that you have configured the command line with AWS credentials and you have installed AWS CLI.
Bucket Management
# Create a bucket
aws s3 mb s3://<bucket-name>
# Delete a bucket:
aws s3 rb s3://<bucket-name> --force
(Use --force to delete a non-empty bucket)
# List all buckets:
aws s3 ls
Object Management
# Upload a file:
aws s3 cp <local-file-path> s3://<bucket-name>/<key>
# Download a file:
aws s3 cp s3://<bucket-name>/<key> <local-file-path>
# Delete an object:
aws s3 rm s3://<bucket-name>/<key>
Synchronization
# Sync local directory to bucket
aws s3 sync <local-directory> s3://<bucket-name>
# Sync bucket to local directory:
aws s3 sync s3://<bucket-name> <local-directory>
Listing objects
# List objects in a bucket:
aws s3 ls s3://<bucket-name>
# For generating a presigned URL for an object:
aws s3 presign s3://<bucket-name>/<key> --expires-in <seconds>
I have included the link where you can see all the basic commands with arguments to manage buckets and objects.
s3 — AWS CLI 2.28.6 Command Reference
This content originally appeared on DEV Community and was authored by Harshavardhan S U