Troubleshooting AWS S3 errors in AWS CLI

AWS S3 is a widely used object storage service that provides developers and IT teams with secure, scalable, and highly available data storage. However, sometimes errors can occur while accessing the service using the AWS Command Line Interface (CLI). These errors can be frustrating and time-consuming to troubleshoot, especially for those who are new to AWS and its command-line interface. Below is a list of common errors.

1. AccessDenied
2. NoSuchBucket
3. NoSuchKey/ Key does not exist
4. InvalidArgument
5. BucketAlreadyExists
6. NoSuchBucketPolicy
7. SlowDown
8. AuthFailure

In this blog, we will discuss these common errors that you may encounter while using AWS CLI to access the AWS S3 service and provide step-by-step instructions on how to diagnose and fix them. We’ll cover everything from authentication issues to bucket policy issues and provide tips on how to prevent these errors from happening in the future. Whether you’re an AWS pro or just starting out, this blog is for you. Let’s dive in and learn how to troubleshoot S3 errors on AWS CLI!

1. AccessDenied

Error Message: upload failed: ./hello.txt to s3://my-k21-bucket/hello.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

Incorrect method:

aws s3 cp hello.txt s3://my-k21-bucket

Explanation:
The above AWS CLI command is trying to upload a file “hello.txt ” to the S3 bucket “my-k21-bucket”. However, if the AWS CLI user does not have permission to access the S3 bucket, this command will result in an “AccessDenied” error.

Solution:
Check your S3 bucket policies and ensure that the IAM user or role you are using to upload the object has the necessary permissions. Check the bucket policy and the access control list (ACL) to verify that the IAM user or role has the necessary permissions to upload objects to the bucket.

Also, you need to grant the PutObject permission to the user or IAM role that is running the AWS CLI command. You can do this by adding the following policy statement to the user or IAM role:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowBucketPutObject",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my-k21-bucket/*"
}
]
}

2. NoSuchBucket

Error Message: An error occurred (NoSuchBucket) when calling the PutBucketWebsite operation: The specified bucket does not exist

Incorrect method:

aws s3 website my-k21-buck --index-document index.html --error-document error.html

Explanation:
This command enables static website hosting on the specified S3 bucket and sets the index document to “index.html” and the error document to “error.html”. If these files do not exist in the bucket, then attempting to access the S3 bucket using an invalid bucket name or URL will result in a “NoSuchBucket” error.

Solution:

aws s3 website my-k21-bucket --index-document index.html --error-document error.html

Make sure that you have spelled the S3 bucket name correctly. If the bucket exists, ensure that the AWS CLI is using the correct region by specifying the –region parameter with the appropriate region name.

3. NoSuchKey/ Key does not exist

Error Message: fatal error: An error occurred (404) when calling the HeadObject operation: Key “hello.txt” does not exist

Incorrect method:

aws s3 cp s3://my-k21-bucket/hello.txt .

Explanation:
The above AWS CLI command is trying to copy the object “hello.txt” from S3 bucket ” my-k21-bucket” to local directory. However, if the object key (file name) provided in the command does not exist in the bucket, this command will result in a “NoSuchKey” error.

Solution:
To resolve this error, check the key or object name that you are trying to access and ensure that it exists in the specified S3 bucket. If the key or object exists, ensure that you have specified the correct S3 bucket name and key name in the AWS CLI command.

4. InvalidArgument

Error Message: aws: error: argument –grant-full-control: expected one argument.

Incorrect method:

aws s3api put-object-acl --bucket my-k21-bucket --key my-file.txt --acl invalid-acl-value --grant-full-control

Explanation:
In this example, we are trying to set an ACL and grant full control to an object (my-file.txt) in an S3 bucket (my-bucket-name) using the aws s3api put-object-acl command. However, we have specified an invalid ACL value (invalid-acl-value) which is not a valid value for the –acl parameter.

Solution:

aws s3api put-object-acl --bucket my-k21-bucket --key my-file.txt --grant-full-control id=0123456789

This command will upload a file to an S3 bucket and grant full control over the object to the AWS account with the specified ID (0123456789). Please note that this will not allow public read and write access to the object. If you also want to grant public read and write access, you can use a canned ACL like public-read-write or public-read instead of the grant-full-control parameter.

5. BucketAlreadyExists

Error message: make_bucket failed: s3://my-k21-bucket An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.

Explanation:
If you encounter this error, it means that the bucket you are trying to create already exists in AWS. To resolve the issue, you can either use the existing bucket, or you can choose a different name for your new bucket.

Solution:
If you want to check the list of your existing buckets:

aws s3 ls

If you want to create a new bucket with a different name:

aws s3api create-bucket --bucket my-new-k21-bucket --region my-region

Make sure to replace my-new-k21-bucket with a unique name for your bucket and my-region with the AWS region where you want to create your bucket. Keep in mind that bucket names must be globally unique across all AWS accounts, so you may need to come up with a more unique name for your bucket.

6. NoSuchBucketPolicy

Error message: An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

Incorrect method:

aws s3api get-bucket-policy --bucket my-k21-bucket

Explanation:
If you are trying to retrieve the bucket policy of an S3 bucket using the “get-bucket-policy” API and the specified bucket does not have a policy configured, you may encounter a “NoSuchBucketPolicy” error.

Solution:
To resolve this error, you can either create a new bucket policy for the bucket or remove the command that is trying to retrieve the bucket policy. If you want to create a new bucket policy for the bucket, you can use the following AWS CLI command:

vi policy.json

Paste below sample policy in the editor:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAllActions",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::my-k21-bucket",
"arn:aws:s3:::my-k21-bucket/*"
]
}
]
}

Type “:wq” (without quotations) and hit enter.

Now attach the policy:

aws s3api put-bucket-policy --bucket my-k21-bucket --policy file://policy.json

The correct way to configure a bucket policy for the S3 bucket “my-k21-bucket” is to use the “put-bucket-policy” API with the bucket name and the policy document file path. The correct AWS CLI command is shown above. Make sure the IAM user or role has the necessary permissions to put a bucket policy on the S3 bucket and public access is allowed to the bucket.

7. SlowDown

Error message: An error occurred (SlowDown) when calling the S3 operation: Please reduce your request rate

Incorrect method:

aws s3 ls s3://my-bucket –recursive

Explanation:
This command lists all the objects in the “my-bucket” S3 bucket, including all subdirectories, which can cause a high rate of requests to the bucket and potentially trigger a “SlowDown” error.

Solution:

aws s3 ls s3://my-bucket --recursive --no-human-readable --page-size 800

To avoid the “SlowDown” error, you can modify the command to use the –page-size option, which limits the number of objects returned per page, and the –no-human-readable option, which avoids displaying the size of the objects shown above. This will limit the number of objects listed per page to 800 and reduce the number of requests made to the bucket, which can help avoid a “SlowDown” error.

8. AuthFailure

Error message: An error occurred (AuthFailure) when calling the S3 operation: AWS was not able to validate the provided access credentials

The config profile (non-existent-profile) could not be found

Incorrect method:

aws s3 ls s3://my-bucket

Explanation:
This command lists the contents of the “my-bucket” S3 bucket, but if the credentials used to access the bucket are incorrect or have expired, or if the IAM user or role associated with the credentials does not have the necessary permissions to access the S3 bucket, it will result in an “AuthFailure” error.

Solution:
To resolve this error, you can verify that the AWS CLI credentials are correct and have not expired and ensure that the IAM user or role associated with the credentials has the necessary permissions to access the S3 bucket. You can also use the aws configure command to update the credentials and region used by the AWS CLI

aws s3 ls s3://my-bucket --profile my-profile

This command uses a special set of information called a “profile” to access the storage space. This profile has the correct login details and permissions needed to look at the files in the storage space. By using the correct profile, you can make sure that the command works properly and that you can see the files in the storage space without any login errors.

Frequently Asked Questions:

1. AccessDenied
2. NoSuchBucket
3. NoSuchKey /Key does not exist
4. InvalidArgument
5. BucketAlreadyExists
6. NoSuchBucketPolicy
7. SlowDown
8. AuthFailure

" } } , { "@type": "Question", "name": "Which AWS CLI command is used to authenticate to aws?", "acceptedAnswer": { "@type": "Answer", "text": "The aws configure command is the quickest method for setting up your AWS CLI installation in ordinary use. The Amazon CLI asks you for the following information when you enter this command:

1. Access key ID
2. Secret Access Key
3. AWS Region
4. Output Format

" } } , { "@type": "Question", "name": "Why does my AWS CLI S3 transfer fail with a 'Connection Timeout' error, and how can I fix it?", "acceptedAnswer": { "@type": "Answer", "text": "The occurrence of a 'Connection Timeout' error could result from problems with the network connection between your local device and the AWS S3 service. Typically, this error suggests that the request sent from the AWS CLI to the S3 service failed to get a response within the pre-set time limit, which is usually around 30 seconds. To fix this issue, try the following troubleshooting steps:

1. Check your network connectivity
2. Check the AWS S3 service status
3. Increase the timeout value
4. Try a different AWS region
5. Check the size of the file being transferred

" } } ] }

Related Links/References:

• AWS Certified Solutions Architect Associate SAA-C03 Exam details
• Amazon S3 Bucket: Overview, Errors & Resolutions
• How to Configure & Install AWS CLI?
• AWS Certified DevOps Engineer Professional DOP-C02
• How to Create a free tier Account in AWS?

Next Task For You

In our AWS Solution Architect Associate training program, we will create a Custom VPC in detail and 30 other Hands-On Labs. If you want to begin your journey towards becoming an AWS Certified Solution Architect Associate, check out our FREE CLASS.

The post Troubleshooting AWS S3 errors in AWS CLI appeared first on Cloud Training Program.