# Simple Storage Service
1. [TL;DR](#tldr)
1. [Storage tiers](#storage-tiers)
1. [Lifecycle configuration](#lifecycle-configuration)
1. [Further readings](#further-readings)
1. [Sources](#sources)
## TL;DR
Usage
```sh
# List all buckets.
aws s3 ls
aws s3api list-buckets --output 'json' --query 'Buckets[].Name'
aws s3api list-buckets --output 'yaml-stream' | yq -r '.[].Buckets[].Name' -
# List prefixes and objects in buckets.
# Adding the trailing '/' or '--recurse' lists the content of prefixes.
aws s3 ls 's3://my-bucket'
aws s3 ls --recursive 's3://my-bucket/prefix/'
aws s3 ls 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/'
# Find the size of buckets or objects.
# It will list all the contents *and* give a total size at the end.
aws s3 ls --human-readable --recursive --summarize 's3://my-bucket'
aws s3 ls … 's3://my-bucket/prefix/'
# Create buckets.
aws s3 mb 's3://my-bucket'
# Copy files to or from buckets.
aws s3 cp 'test.txt' 's3://my-bucket/test4.txt'
aws s3 cp 'test.txt' 's3://my-bucket/test2.txt' --expires '2024-10-01T20:30:00Z'
aws s3 cp 's3://my-bucket/test.txt' 'test2.txt'
aws s3 cp 's3://my-bucket/test.txt' 's3://my-bucket/test5.txt'
aws s3 cp 's3://my-bucket/test.txt' 's3://my-other-bucket/'
aws s3 cp 's3://my-bucket' '.' --recursive
aws s3 cp 'myDir' 's3://my-bucket/' --recursive --exclude "*.jpg"
aws s3 cp 's3://my-bucket/logs/' 's3://my-bucket2/logs/' --recursive \
--exclude "*" --include "*.log"
aws s3 cp 's3://my-bucket/test.txt' 's3://my-bucket/test2.txt' \
--acl 'public-read-write'
aws s3 cp 'file.txt' 's3://my-bucket/' \
--grants read=uri='http://acs.amazonaws.com/groups/global/AllUsers' \
'full=id=79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be'
aws s3 cp 'mydoc.txt' 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/mykey'
# Handle file streams.
# Useful for piping:
# - setting the source to '-' sends data from stdin
# - setting the destination to '-' sends data to stdout
aws s3 cp - 's3://my-bucket/stream.txt'
aws s3 cp - 's3://my-bucket/stream.txt' --expected-size '54760833024'
aws s3 cp 's3://my-bucket/stream.txt' -
# Directly print the contents of files to stdout.
aws s3 cp --quiet 's3://my-bucket/file.txt' '-'
aws s3 cp --quiet 's3://my-bucket/file.txt' '/dev/stdout'
# Remove objects.
aws s3 rm 's3://my-bucket/prefix-name' --recursive --dryrun
# Sync buckets.
aws s3 sync '.' 's3://my-bucket'
aws s3 sync 's3://my-bucket' '.' --delete
aws s3 sync 's3://my-bucket' 's3://my-other-bucket' --exclude "*.jpg"
aws s3 sync 's3://my-us-west-2-bucket' 's3://my-eu-east-1-bucket' \
--source-region 'us-west-2' --region 'eu-east-1'
aws s3 sync '.' 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/'
# Delete buckets.
aws s3 rb 's3://my-bucket'
aws s3 rb 's3://my-bucket' --force
# Check permissions.
aws s3api get-bucket-acl --bucket 'my-bucket'
```
Lifecycle configurations
```sh
# Manage lifecycle configurations.
# Operations on lifecycle rules take a while.
aws s3api get-bucket-lifecycle-configuration --bucket 'bucketName'
aws s3api put-bucket-lifecycle-configuration --bucket 'bucketName' \
--lifecycle-configuration 'file://lifecycle.definition.json'
aws s3api delete-bucket-lifecycle-configuration --bucket 'bucketName'
```
Real life use cases
```sh
# Get objects with their storage class.
aws s3api list-objects --bucket 'my-bucket' \
--query 'Contents[].{Key: Key, StorageClass: StorageClass}'
# Show tags on objects.
aws s3api list-objects-v2 \
--bucket 'my-bucket' --prefix 'someObjectsInHereAreTagged' \
--query 'Contents[*].Key' --output text \
| xargs -n 1 \
aws s3api get-object-tagging --bucket 'my-bucket' --query 'TagSet[*]' --key
```
## Storage tiers
| | Standard | Intelligent-Tiering | Express One Zone | Standard Infrequent Access | One Zone Infrequent Access | Glacier Instant Retrieval | Glacier Flexible Retrieval | Glacier Deep Archive |
| ---------------------- | ------------ | ------------------- | ------------------------- | -------------------------- | -------------------------- | ------------------------- | -------------------------- | -------------------- |
| Retrieval charge | ✗ | ✗ | ✗ | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved |
| Latency | milliseconds | milliseconds | single-digit milliseconds | milliseconds | milliseconds | milliseconds | minutes to hours | hours |
| Minimum storage charge | ✗ | ✗ | 1 hour | 30 days | 30 days | 90 days | 90 days | 180 days |
| Availability Zones | 3+ | 3+ | 1 | 3+ | 1 | 3+ | 3+ | 3+ |
## Lifecycle configuration
> Adding, removing or changing lifecycle rules takes a while.
> Wait a couple of minutes after the operation to make sure all the bucket's properties are synced.
When multiple rules are applied through S3 Lifecycle configurations, objects can become eligible for multiple S3
Lifecycle actions. In such cases:
1. Permanent deletion takes precedence over transitions.
1. Transitions takes precedence over creation of delete markers.
1. When objects are eligible for transition to both S3 Glacier Flexible Retrieval and S3 Standard-IA (or One Zone-IA),
precedence is given to S3 Glacier Flexible Retrieval transition.
When adding S3 Lifecycle configurations to buckets, there is usually some lag before a new or updated Lifecycle
configuration is fully propagated to all the S3's systems.
Expect a delay of a few minutes before any change in configuration fully takes effect. This includes configuration
deletions.
Objects can only go down the tiers, not up.
Other constraints apply, like no transition done for objects smaller than 128KiB.
See [General considerations for transitions][lifecycle general considerations for transitions].
Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules examples]
## Further readings
- [Amazon Web Services]
- [Configure notification for lifecycle rules][lifecycle configure notification]
- AWS' [CLI]
- [Expiring Amazon S3 objects based on last accessed date to decrease costs]
### Sources
- [Amazon S3 Storage Classes]
- [General considerations for transitions][lifecycle general considerations for transitions]
- [Lifecycle configuration examples][lifecycle configuration examples]
- [CLI subcommand reference]
- [Find out the size of your Amazon S3 buckets]
- [How S3 Intelligent-Tiering works]
[amazon web services]: README.md
[cli]: cli.md
[s3 lifecycle rules examples]: ../../../examples/aws/s3.lifecycle-rules
[amazon s3 storage classes]: https://aws.amazon.com/s3/storage-classes/
[cli subcommand reference]: https://docs.aws.amazon.com/cli/latest/reference/s3/
[expiring amazon s3 objects based on last accessed date to decrease costs]: https://aws.amazon.com/blogs/architecture/expiring-amazon-s3-objects-based-on-last-accessed-date-to-decrease-costs/
[find out the size of your amazon s3 buckets]: https://aws.amazon.com/blogs/storage/find-out-the-size-of-your-amazon-s3-buckets/
[how s3 intelligent-tiering works]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/intelligent-tiering-overview.html
[lifecycle configuration examples]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html
[lifecycle configure notification]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configure-notification.html
[lifecycle general considerations for transitions]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html