Simple Storage Service

TL;DR
Lifecycle configuration
Further readings
1. Sources

TL;DR

Common usage

# List all buckets.
aws s3 ls
aws s3api list-buckets --output 'json' --query 'Buckets[].Name'
aws s3api list-buckets --output 'yaml-stream' | yq -r '.[].Buckets[].Name' -

# List prefixes and objects in buckets.
# Adding the trailing '/' or '--recurse' lists the content of prefixes.
aws s3 ls 's3://my-bucket'
aws s3 ls --recursive 's3://my-bucket/prefix/'
aws s3 ls 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/'

# Find the size of buckets or objects.
# It will list all the contents *and* give a total size at the end.
aws s3 ls --human-readable --recursive --summarize 's3://my-bucket'
aws s3 ls … 's3://my-bucket/prefix/'

# Create buckets.
aws s3 mb 's3://my-bucket'

# Copy files to or from buckets.
aws s3 cp 'test.txt' 's3://my-bucket/test4.txt'
aws s3 cp 'test.txt' 's3://my-bucket/test2.txt' --expires '2024-10-01T20:30:00Z'
aws s3 cp 's3://my-bucket/test.txt' 'test2.txt'
aws s3 cp 's3://my-bucket/test.txt' 's3://my-bucket/test5.txt'
aws s3 cp 's3://my-bucket/test.txt' 's3://my-other-bucket/'
aws s3 cp 's3://my-bucket' '.' --recursive
aws s3 cp 'myDir' 's3://my-bucket/' --recursive --exclude "*.jpg"
aws s3 cp 's3://my-bucket/logs/' 's3://my-bucket2/logs/' --recursive \
  --exclude "*" --include "*.log"
aws s3 cp 's3://my-bucket/test.txt' 's3://my-bucket/test2.txt' \
    --acl 'public-read-write'
aws s3 cp 'file.txt' 's3://my-bucket/' \
  --grants read=uri='http://acs.amazonaws.com/groups/global/AllUsers' \
    'full=id=79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be'
aws s3 cp 'mydoc.txt' 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/mykey'

# Handling file streams.
aws s3 cp - 's3://my-bucket/stream.txt'
aws s3 cp - 's3://my-bucket/stream.txt' --expected-size '54760833024'
aws s3 cp 's3://my-bucket/stream.txt' -

# Sync buckets.
aws s3 sync '.' 's3://my-bucket'
aws s3 sync 's3://my-bucket' '.' --delete
aws s3 sync 's3://my-bucket' 's3://my-other-bucket' --exclude "*.jpg"
aws s3 sync 's3://my-us-west-2-bucket' 's3://my-eu-east-1-bucket' \
  --source-region 'us-west-2' --region 'eu-east-1'
aws s3 sync '.' 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/'

# Delete buckets.
aws s3 rb 's3://my-bucket'
aws s3 rb 's3://my-bucket' --force


# Lifecycle configurations.
aws s3api get-bucket-lifecycle-configuration --bucket 'bucketName'
aws s3api put-bucket-lifecycle-configuration --bucket 'bucketName' \
  --lifecycle-configuration 'file://lifecycle.definition.json'
aws s3api delete-bucket-lifecycle-configuration --bucket 'bucketName'


# Show tags on objects.
aws s3api list-objects-v2 \
  --bucket 'my-bucket' --prefix 'someObjectsInHereAreTagged' \
  --query 'Contents[*].Key' --output text \
| xargs -n 1 \
    aws s3api get-object-tagging --bucket 'my-bucket' --query 'TagSet[*]' --key

Lifecycle configurations

# Manage lifecycle configurations.
aws s3 get-bucket-lifecycle-configuration --bucket 'batman'
aws s3 put-bucket-lifecycle-configuration --bucket 'batman'  \
  --lifecycle-configuration 'file://lifecycle-batman.json'
aws s3 delete-bucket-lifecycle --bucket 'batman'

Lifecycle configuration

Adding, removing or changing lifecycle rules takes a while.
Wait a couple of minutes after the operation to make sure all the bucket's properties are synced.

When one has multiple rules in an S3 Lifecycle configuration, an object can become eligible for multiple S3 Lifecycle actions. In such cases, Amazon S3 follows these general rules:

Permanent deletion takes precedence over transition.
Transition takes precedence over creation of delete markers.
When an object is eligible for both a S3 Glacier Flexible Retrieval and S3 Standard-IA (or S3 One Zone-IA) transition, Amazon S3 chooses the S3 Glacier Flexible Retrieval transition.

Propagation delay: When you add an S3 Lifecycle configuration to a bucket, there is usually some lag before a new or updated Lifecycle configuration is fully propagated to all the Amazon S3 systems. Expect a delay of a few minutes before the configuration fully takes effect. This delay can also occur when you delete an S3 Lifecycle configuration.

Objects can only go down the tiers, not up. Many other constraints apply, like no transition done for objects <128KiB.
See General considerations for transitions.

Examples: 1, 2

6.2 KiB Raw Blame History

Simple Storage Service

TL;DR

Lifecycle configuration

Further readings

Sources

6.2 KiB

Raw Blame History