brandon/oam
1
0
Fork 0
mirror of https://gitea.com/mcereda/oam.git synced 2026-02-08 21:34:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(kb/aws/s3): expand on storage classes

Browse Source
This commit is contained in:
Michele Cereda
2025-07-05 22:06:03 +02:00
parent 87d7fa7f0d
commit 57b99e526d
2 changed files with 89 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
knowledge base/cloud computing/aws/s3 supported storage classes transitions.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 37 KiB

109
knowledge base/cloud computing/aws/s3.md
View File

@@ -1,7 +1,7 @@
# Simple Storage Service
1. [TL;DR](#tldr)
1. [Storage tiers](#storage-tiers)
1. [Storage classes](#storage-classes)
1. [Lifecycle configuration](#lifecycle-configuration)
1. [Further readings](#further-readings)
1. [Sources](#sources)
@@ -113,36 +113,105 @@ aws s3api list-objects-v2 \
</details>
## Storage tiers
## Storage classes
| | Standard | Intelligent-Tiering | Express One Zone | Standard Infrequent Access | One Zone Infrequent Access | Glacier Instant Retrieval | Glacier Flexible Retrieval | Glacier Deep Archive |
| ---------------------- | ------------ | ------------------- | ------------------------- | -------------------------- | -------------------------- | ------------------------- | -------------------------- | -------------------- |
| Retrieval charge | ✗ | ✗ | ✗ | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved |
| Latency | milliseconds | milliseconds | single-digit milliseconds | milliseconds | milliseconds | milliseconds | minutes to hours | hours |
| Minimum storage charge | ✗ | ✗ | 1 hour | 30 days | 30 days | 90 days | 90 days | 180 days |
| Availability Zones | 3+ | 3+ | 1 | 3+ | 1 | 3+ | 3+ | 3+ |
| Class name | Console name | Fees | Latency | Minimum storage charge | Minimum billed object size | # of AZs |
| -------------------------- | --------------------- | -------------------- | ---------------- | ---------------------- | -------------------------- | -------- |
| Standard | `STANDARD` | ✗ | milliseconds | ✗ | | 3+ |
| Express One Zone | `EXPRESS_ONEZONE` | ✗ | single-digit ms | 1 hour | | 1 |
| Intelligent Tiering | `INTELLIGENT_TIERING` | per monitored object | milliseconds | ✗ | | 3+ |
| Standard Infrequent Access | `STANDARD_IA` | per GB retrieved | milliseconds | 30 days | 128 KB | 3+ |
| One Zone Infrequent Access | `ONEZONE_IA` | per GB retrieved | milliseconds | 30 days | 128 KB | 1 |
| Glacier Instant Retrieval | `GLACIER_IR` | per GB retrieved | milliseconds | 90 days | 128 KB | 3+ |
| Glacier Flexible Retrieval | `GLACIER` | per GB retrieved | minutes to hours | 90 days | | 3+ |
| Glacier Deep Archive | `DEEP_ARCHIVE` | per GB retrieved | hours | 180 days | | 3+ |
_Standard_ is the storage class used by default if none is specified when uploading objects.
_Express One Zone_ is purpose-built for consistency and low latency. It has the highest performance, and lower request
costs than standard, but is only available within a single Availability Zone at a time.
_Intelligent Tiering_ optimizes storage costs by automatically moving data between access tiers depending on its usage,
without performance impact or operational overhead.<br/>
Ideal for data that has unknown or changing access patterns.
Intelligent Tiering automatically moves objects that have not been accessed in some time to lower-cost access tiers that
still offer low-latency and high-throughput.
<details style='padding: 0 0 1rem 1rem'>
Objects in Intelligent Tiering are stored automatically in the following tiers:
- _Frequent Access_: contains objects that are uploaded, or transitioned, to the storage class.
- _Infrequent Access_: contains objects that have not been accessed for **30 consecutive days**.
- _Archive Instant Access_: contains objects that have not been accessed for **90 consecutive days**.
> [!important]
> Object less than 128 KB in size are **not** eligible for auto-tiering. These objects are kept in the Frequent Access
> tier at all times.
</details>
One can also enable automatic archiving capabilities within Intelligent Tiering for data that can be accessed
**asynchronously**. In this case, it will eventually move objects to access tiers with even lower costs, but that
require explicit retrieval processes.
<details style='padding: 0 0 1rem 1rem'>
The optional archive access tiers are the following:
- _Archive Access_: archives objects that have not been accessed for **at least 90 consecutive days**.
- _Deep Archive Access_: archives objects that have not been accessed for **at least 180 consecutive days**.
Objects in the Archive Access or Deep Archive Access tiers **must first be restored** to higher tiers by using the
`RestoreObject` action.
</details>
_Standard Infrequent Access_ and _One Zone Infrequent Access_ are designed for data that is both **long-lived** and
**infrequently accessed**, but still requires millisecond access.<br/>
Suitable for objects larger than 128 KB that are needed for at least 30 days.
> [!important]
> S3 charges for object smaller than 128 KB as if they were of 128 KB.<br/>
> Objects deleted, overwritten, or transitioned to a different storage class before the end of the 30-day minimum
> storage duration period will still incur in charges for the full 30 days.
_Glacier Instant Retrieval_, _Glacier Flexible Retrieval_, and _Glacier Deep Archive_ are designed for low-cost,
long-term data storage and data archiving.<br/>
All these storage classes require minimum storage durations and charge retrieval fees.
Glacier Instant Retrieval is the only one in the Glacier set that offers milliseconds retrieval and real-time
access.<br/>
Glacier Flexible Retrieval and Glacier Deep Archive archive the data they receive, making it **not** available for
real-time access.
## Lifecycle configuration
> Adding, removing or changing lifecycle rules takes a while.<br/>
> Wait a couple of minutes after the operation to make sure all the bucket's properties are synced.
S3 supports specific lifecycle transitions between storage classes using Lifecycle configurations:
When multiple rules are applied through S3 Lifecycle configurations, objects can become eligible for multiple S3
Lifecycle actions. In such cases:
![supported storage classes transitions](s3%20supported%20storage%20classes%20transitions.png)
Objects can be transitioned **down** the storage classes, but **not** up.<br/>
Objects in need to be moved to a higher storage class need to be **_recreated_** in that storage class. This means that
they will take new metadata.
Other constraints apply, e.g., objects smaller than 128KiB are not usually transitioned in tier.<br/>
See [General considerations for transitions][lifecycle general considerations for transitions].
When multiple rules are applied through Lifecycle configurations, objects can become eligible for multiple Lifecycle
actions. In such cases:
1. Permanent deletion takes precedence over transitions.
1. Transitions takes precedence over creation of delete markers.
1. When objects are eligible for transition to both S3 Glacier Flexible Retrieval and S3 Standard-IA (or One Zone-IA),
precedence is given to S3 Glacier Flexible Retrieval transition.
When adding S3 Lifecycle configurations to buckets, there is usually some lag before a new or updated Lifecycle
configuration is fully propagated to all the S3's systems.<br/>
Expect a delay of a few minutes before any change in configuration fully takes effect. This includes configuration
deletions.
Objects can only go down the tiers, not up.<br/>
Other constraints apply, like no transition done for objects smaller than 128KiB.<br/>
See [General considerations for transitions][lifecycle general considerations for transitions].
> [!important]
> When adding Lifecycle configurations to buckets, there is usually some lag before a new, or updated, Lifecycle
> configuration is fully propagated to all S3's systems.<br/>
> Expect a delay of a few minutes before any change in configuration starts taking effect. This includes configuration
> deletions.
Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules examples]