diff --git a/knowledge base/cloud computing/aws/s3 supported storage classes transitions.png b/knowledge base/cloud computing/aws/s3 supported storage classes transitions.png new file mode 100644 index 0000000..b77ae95 Binary files /dev/null and b/knowledge base/cloud computing/aws/s3 supported storage classes transitions.png differ diff --git a/knowledge base/cloud computing/aws/s3.md b/knowledge base/cloud computing/aws/s3.md index b55dcff..05283c2 100644 --- a/knowledge base/cloud computing/aws/s3.md +++ b/knowledge base/cloud computing/aws/s3.md @@ -1,7 +1,7 @@ # Simple Storage Service 1. [TL;DR](#tldr) -1. [Storage tiers](#storage-tiers) +1. [Storage classes](#storage-classes) 1. [Lifecycle configuration](#lifecycle-configuration) 1. [Further readings](#further-readings) 1. [Sources](#sources) @@ -113,36 +113,105 @@ aws s3api list-objects-v2 \ -## Storage tiers +## Storage classes -| | Standard | Intelligent-Tiering | Express One Zone | Standard Infrequent Access | One Zone Infrequent Access | Glacier Instant Retrieval | Glacier Flexible Retrieval | Glacier Deep Archive | -| ---------------------- | ------------ | ------------------- | ------------------------- | -------------------------- | -------------------------- | ------------------------- | -------------------------- | -------------------- | -| Retrieval charge | ✗ | ✗ | ✗ | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | -| Latency | milliseconds | milliseconds | single-digit milliseconds | milliseconds | milliseconds | milliseconds | minutes to hours | hours | -| Minimum storage charge | ✗ | ✗ | 1 hour | 30 days | 30 days | 90 days | 90 days | 180 days | -| Availability Zones | 3+ | 3+ | 1 | 3+ | 1 | 3+ | 3+ | 3+ | +| Class name | Console name | Fees | Latency | Minimum storage charge | Minimum billed object size | # of AZs | +| -------------------------- | --------------------- | -------------------- | ---------------- | ---------------------- | -------------------------- | -------- | +| Standard | `STANDARD` | ✗ | milliseconds | ✗ | | 3+ | +| Express One Zone | `EXPRESS_ONEZONE` | ✗ | single-digit ms | 1 hour | | 1 | +| Intelligent Tiering | `INTELLIGENT_TIERING` | per monitored object | milliseconds | ✗ | | 3+ | +| Standard Infrequent Access | `STANDARD_IA` | per GB retrieved | milliseconds | 30 days | 128 KB | 3+ | +| One Zone Infrequent Access | `ONEZONE_IA` | per GB retrieved | milliseconds | 30 days | 128 KB | 1 | +| Glacier Instant Retrieval | `GLACIER_IR` | per GB retrieved | milliseconds | 90 days | 128 KB | 3+ | +| Glacier Flexible Retrieval | `GLACIER` | per GB retrieved | minutes to hours | 90 days | | 3+ | +| Glacier Deep Archive | `DEEP_ARCHIVE` | per GB retrieved | hours | 180 days | | 3+ | + +_Standard_ is the storage class used by default if none is specified when uploading objects. + +_Express One Zone_ is purpose-built for consistency and low latency. It has the highest performance, and lower request +costs than standard, but is only available within a single Availability Zone at a time. + +_Intelligent Tiering_ optimizes storage costs by automatically moving data between access tiers depending on its usage, +without performance impact or operational overhead.
+Ideal for data that has unknown or changing access patterns. + +Intelligent Tiering automatically moves objects that have not been accessed in some time to lower-cost access tiers that +still offer low-latency and high-throughput. + +
+ +Objects in Intelligent Tiering are stored automatically in the following tiers: + +- _Frequent Access_: contains objects that are uploaded, or transitioned, to the storage class. +- _Infrequent Access_: contains objects that have not been accessed for **30 consecutive days**. +- _Archive Instant Access_: contains objects that have not been accessed for **90 consecutive days**. + +> [!important] +> Object less than 128 KB in size are **not** eligible for auto-tiering. These objects are kept in the Frequent Access +> tier at all times. + +
+ +One can also enable automatic archiving capabilities within Intelligent Tiering for data that can be accessed +**asynchronously**. In this case, it will eventually move objects to access tiers with even lower costs, but that +require explicit retrieval processes. + +
+ +The optional archive access tiers are the following: + +- _Archive Access_: archives objects that have not been accessed for **at least 90 consecutive days**. +- _Deep Archive Access_: archives objects that have not been accessed for **at least 180 consecutive days**. + +Objects in the Archive Access or Deep Archive Access tiers **must first be restored** to higher tiers by using the +`RestoreObject` action. + +
+ +_Standard Infrequent Access_ and _One Zone Infrequent Access_ are designed for data that is both **long-lived** and +**infrequently accessed**, but still requires millisecond access.
+Suitable for objects larger than 128 KB that are needed for at least 30 days. + +> [!important] +> S3 charges for object smaller than 128 KB as if they were of 128 KB.
+> Objects deleted, overwritten, or transitioned to a different storage class before the end of the 30-day minimum +> storage duration period will still incur in charges for the full 30 days. + +_Glacier Instant Retrieval_, _Glacier Flexible Retrieval_, and _Glacier Deep Archive_ are designed for low-cost, +long-term data storage and data archiving.
+All these storage classes require minimum storage durations and charge retrieval fees. + +Glacier Instant Retrieval is the only one in the Glacier set that offers milliseconds retrieval and real-time +access.
+Glacier Flexible Retrieval and Glacier Deep Archive archive the data they receive, making it **not** available for +real-time access. ## Lifecycle configuration -> Adding, removing or changing lifecycle rules takes a while.
-> Wait a couple of minutes after the operation to make sure all the bucket's properties are synced. +S3 supports specific lifecycle transitions between storage classes using Lifecycle configurations: -When multiple rules are applied through S3 Lifecycle configurations, objects can become eligible for multiple S3 -Lifecycle actions. In such cases: +![supported storage classes transitions](s3%20supported%20storage%20classes%20transitions.png) + +Objects can be transitioned **down** the storage classes, but **not** up.
+Objects in need to be moved to a higher storage class need to be **_recreated_** in that storage class. This means that +they will take new metadata. + +Other constraints apply, e.g., objects smaller than 128KiB are not usually transitioned in tier.
+See [General considerations for transitions][lifecycle general considerations for transitions]. + +When multiple rules are applied through Lifecycle configurations, objects can become eligible for multiple Lifecycle +actions. In such cases: 1. Permanent deletion takes precedence over transitions. 1. Transitions takes precedence over creation of delete markers. 1. When objects are eligible for transition to both S3 Glacier Flexible Retrieval and S3 Standard-IA (or One Zone-IA), precedence is given to S3 Glacier Flexible Retrieval transition. -When adding S3 Lifecycle configurations to buckets, there is usually some lag before a new or updated Lifecycle -configuration is fully propagated to all the S3's systems.
-Expect a delay of a few minutes before any change in configuration fully takes effect. This includes configuration -deletions. - -Objects can only go down the tiers, not up.
-Other constraints apply, like no transition done for objects smaller than 128KiB.
-See [General considerations for transitions][lifecycle general considerations for transitions]. +> [!important] +> When adding Lifecycle configurations to buckets, there is usually some lag before a new, or updated, Lifecycle +> configuration is fully propagated to all S3's systems.
+> Expect a delay of a few minutes before any change in configuration starts taking effect. This includes configuration +> deletions. Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules examples]