Introduction to S3 and EC2 file copy

In this article, we are going to see how to copy files from ec2 instance to S3 bucket and vice versa, with the help of IAM roles and without having to use the AWS access key and secrets in the server.

In AWS technical terms.

Copying files from EC2 to S3 is called Uploading the file

Copying files from S3 to EC2 is called Downloading the files

The first three steps are the same for both upload and download and should be performed only once when you are setting up a new EC2 instance or an S3 bucket. the last and the fourth step is same except the change of source and destination.

Hope the following image helps you understand the difference and between upload and download.

Let see how to get both of this done.

Steps to copy files from EC2 instance to S3 bucket (Upload)

1. Create an IAM role with S3 write access or admin access
2. Map the IAM role to an EC2 instance
3. Install AWS CLI in EC2 instance
4. Run the AWS s3 cp command to copy the files to the S3 bucket

1. Create an IAM role with S3 write access or admin access

Login to your AWS Management Console and go to IAM.

Create a new IAM role with S3 Admin Access which can be lateral mapped to the EC2 instance for easy S3 and EC2 integration.

Refer the following GIF to know how to create a new IAM role for S3 access Video Player00:0000:31

2. Map the IAM role to an EC2 instance

Choose the EC2 instance you want to assign this IAM role to.

Click on Actions > Security> Modify IAM Role

Search for the IAM Role we have created in the previous step and select it and hit Save.

3. Install AWS CLI in EC2 instance

Login to your EC2 instance and follow the instructions corresponding to your Operating System and install the AWS CLI

• Installing AWS CLI version 2 on Linux or Unix

4. Run the AWS s3 cp command to copy the files to the S3 bucket

Now as we have installed the aws command line you can simply use the following commands to copy the files to S3 Bucket from EC2.

# To List the S3 Bucket 
aws s3 ls s3://<S3bucketName>

# To copy the files from EC2 to S3
aws s3 cp <Fully Qualified Local filename> s3://<S3BucketName>

Refer the following GIF to know how it works in real-time. Video Player00:0000:31

Hope this quick article helps you to copy files from S3 to EC2 instance directly without any programmatic access and Keys

Steps to copy files from S3 bucket to EC2 instance (Download)

To copy the files from S3 bucket to EC2 instance, you have to follow the same steps except changing the source and destination on the Fourth Step

1. Create an IAM role with S3 write access or admin access
2. Map the IAM role to an EC2 instance
3. Install AWS CLI in EC2 instance
4. Run the AWS s3 cp command to copy the files from S3 to EC2

To copy the files from S3 to EC2, Keep the source as the bucket URL and the destination to your local directory or filename

# To copy the files from S3 to EC2
aws s3 cp

s3://<S3BucketName>

<Fully Qualified Local filename/Directory> 

In the previous command, you can see the difference. Here the source is S3 Bucket URL and the destination is a local file name or directory name.

Hope it helps

About Author