mirror of
https://gitea.com/mcereda/oam.git
synced 2026-02-09 05:44:23 +00:00
chore(aws): give rds its own article and start kms' section
This commit is contained in:
Michele Cereda
@@ -10,7 +10,7 @@
|
||||
1. [GuardDuty](#guardduty)
|
||||
1. [EventBridge](#eventbridge)
|
||||
1. [Inspector](#inspector)
|
||||
1. [RDS](#rds)
|
||||
1. [KMS](#kms)
|
||||
1. [Security Hub](#security-hub)
|
||||
1. [Resource constraints](#resource-constraints)
|
||||
1. [Access control](#access-control)
|
||||
@@ -60,6 +60,7 @@ One can can rapidly remapping addresses to other instances in one's account and
|
||||
| [EventBridge] | FIXME |
|
||||
| [GuardDuty] | Threat detection |
|
||||
| [Inspector] | FIXME |
|
||||
| [KMS] | Key management |
|
||||
| [OpenSearch] | ELK, logging |
|
||||
| [RDS] | Databases |
|
||||
| [S3] | Storage |
|
||||
@@ -89,8 +90,6 @@ This feature is automatically enabled by default when one starts using one of th
|
||||
|
||||
### Config
|
||||
|
||||
FIXME
|
||||
|
||||
Compliance service for assessing and auditing AWS resources.
|
||||
|
||||
Provides an inventory of resources.<br/>
|
||||
@@ -113,15 +112,11 @@ Sample templates for compliance standards and benchmarks are available.
|
||||
|
||||
### Detective
|
||||
|
||||
FIXME
|
||||
|
||||
Uses ML and graphs to try and identify the root cause of security issues.<br/>
|
||||
Creates visualizations with details and context by leveraging events from VPC Flow Logs, CloudTrail and GuardDuty.
|
||||
|
||||
### GuardDuty
|
||||
|
||||
FIXME
|
||||
|
||||
Threat detection service.
|
||||
|
||||
It continuously monitors accounts and workloads for malicious activity and delivers security findings for visibility and
|
||||
@@ -139,73 +134,62 @@ _Threat IP List_ is a blacklist of **public IPs and CIDRs** that will be used by
|
||||
|
||||
### EventBridge
|
||||
|
||||
FIXME
|
||||
TODO
|
||||
|
||||
### Inspector
|
||||
|
||||
FIXME
|
||||
TODO
|
||||
|
||||
### RDS
|
||||
### KMS
|
||||
|
||||
Read replicas **can** be promoted to standalone DB instances.<br/>
|
||||
See [Working with DB instance read replicas].
|
||||
_Key material_ is the cryptographic secret of Keys that is used in encryption operations.
|
||||
|
||||
Disk free metrics are available in CloudWatch.
|
||||
Enabling automatic key rotation for a KMS key makes the service generate new cryptographic material for the key every
|
||||
year by default.<br/>
|
||||
Specify a custom rotation period to customize that time frame.
|
||||
|
||||
Automatic backups are **enabled** by default.<br/>
|
||||
RDS will automatically create storage volume snapshots of the **entire** DB instances.<br/>
|
||||
Backups occur during a daily user-configurable 30 minute period backup window and are kept for a configurable number of
|
||||
up to 35 days (_backup retention period_). One can recover DB instances to any point in time during the backup retention
|
||||
period.
|
||||
Perform on-demand rotation should you need to immediately initiate key material rotation.<br/>
|
||||
This works regardless of whether the automatic key rotation is enabled or not. On-demand rotations do **not** change
|
||||
existing automatic rotation schedules.
|
||||
|
||||
DB instances must be in the `available` state for automated backups to occur.<br/>
|
||||
Automated backups don't occur while DB instances are in other states (i.e., `storage_full`).
|
||||
KMS saves **all** previous versions of the cryptographic material in perpetuity to allow decryption of any data
|
||||
encrypted with keys.<br/>
|
||||
Rotated key material is **not** deleted until the key itself is deleted.
|
||||
|
||||
Automated backups don't occur while a DB snapshot copy is running in the same AWS Region for the same database.
|
||||
Track the rotation of key material [CloudWatch], CloudTrail, and the KMS console.<br/>
|
||||
Alternatively, use the `GetKeyRotationStatus` operation to verify whether automatic rotation is enabled for a key and
|
||||
identify any in progress on-demand rotations. Use the `ListKeyRotations` operation to view the details of completed
|
||||
rotations.
|
||||
|
||||
Back up DB instances manually by creating DB snapshots.<br/>
|
||||
The first snapshot contains the data for the full database. Subsequent snapshots of the same database are incremental.
|
||||
When using a rotated KMS key to encrypt data, KMS uses the **current** key material.<br/>
|
||||
When using the same rotated KMS key to decrypt ciphertext, KMS uses the version of the key material that was used for
|
||||
encryption.<br/>
|
||||
One **cannot** select a particular version of key materials for decrypt operations. This automation allows to safely use
|
||||
rotated KMS keys in applications and AWS services without code changes.
|
||||
|
||||
One can copy both automatic and manual DB snapshots, but only share manual DB snapshots.
|
||||
Automatic key rotation has no effect on the data that KMS keys protect: it does **not** rotate the data generated by
|
||||
rotated keys, re-encrypts any data protected by the keys, nor it will mitigate the effect of compromised data keys.
|
||||
|
||||
RDS backup storage for each Region is composed of both the automated backups and manual DB snapshots for that
|
||||
Region.<br/>
|
||||
Moving snapshots to other Regions increases the backup storage in the destination Regions.
|
||||
KMS supports automatic and on-demand key rotation only for symmetric encryption keys with key material that KMS itself
|
||||
creates.<br/>
|
||||
Automatic rotation is optional for customer managed KMS keys. KMS rotates the key material for AWS managed keys on an
|
||||
yearly basis. Rotation of AWS owned KMS keys is managed by the AWS service that owns the key.
|
||||
|
||||
Backups are stored in [S3].
|
||||
Key rotation only changes the key material, not the key's properties.<br/>
|
||||
The key is considered the same logical resource, regardless of whether or how many times its key material changes.
|
||||
|
||||
Should one choose to retain automated backups when deleting DB instances, its automated backups are saved for the full
|
||||
retention period, otherwise all automated backups are deleted with the instance.<br/>
|
||||
After they are deleted, automated backups can't be recovered.
|
||||
Creating a new key and using it in place of the original one has the same effect as rotating the key material in an
|
||||
existing key.<br/>
|
||||
This is considered a _manual_ key rotation and is a good choice to rotate keys that are not eligible for automatic key
|
||||
rotation.
|
||||
|
||||
Should one choose to have RDS create a final DB snapshot before deleting a DB instance, one can use that or previously
|
||||
created manual snapshots to recover it.
|
||||
AWS charges a monthly fee for the first and second rotation of key material maintained for each key.<br/>
|
||||
This price increase is capped at the second rotation. Any subsequent rotations will **not** be billed.
|
||||
|
||||
Manual snapshots are **not** deleted.
|
||||
|
||||
One can store up to 100 manual snapshots per Region.
|
||||
|
||||
One can choose any of the following retention periods for instances' Performance Insights data:
|
||||
|
||||
- 7 days (default, free tier).
|
||||
- _n_ months, where n is a number from 1 to 24.<br/>
|
||||
In CLI and IaC, this number must be _n*31_.
|
||||
|
||||
One can export DB snapshot data to [S3] buckets.
|
||||
|
||||
The export process runs in the background and does **not** affect the performance of active DB instances.
|
||||
|
||||
RDS extracts data from snapshots and stores it in Apache Parquet format.<br/>
|
||||
By default, **all** data in the snapshots is exported; one can export specific sets of databases, schemas, or tables.
|
||||
|
||||
After the export, one can analyze the data directly through
|
||||
[Athena](https://docs.aws.amazon.com/athena/latest/ug/parquet-serde.html) or
|
||||
[Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/copy-usage_notes-copy-from-columnar.html).
|
||||
Each key counts as one when calculating key resource quotas, regardless of the number of rotated key material versions.
|
||||
|
||||
### Security Hub
|
||||
|
||||
FIXME
|
||||
|
||||
Aggregator of findings for security auditing.
|
||||
|
||||
> Uses [Config] to check resources' configuration by leveraging compliancy rules.
|
||||
@@ -328,7 +312,6 @@ Examples:
|
||||
- [Using service-linked roles]
|
||||
- [Best Practices for Tagging AWS Resources]
|
||||
- [Automating DNS-challenge based LetsEncrypt certificates with AWS Route 53]
|
||||
- [Working with DB instance read replicas]
|
||||
- AWS' [CLI]
|
||||
- [Configuring EC2 Disk alert using Amazon CloudWatch]
|
||||
- [aws.permissions.cloud]
|
||||
@@ -357,11 +340,9 @@ Examples:
|
||||
- [Use an IAM role in the AWS CLI]
|
||||
- [Creating a role to delegate permissions to an IAM user]
|
||||
- [How to use the PassRole permission with IAM roles]
|
||||
- [Introduction to backups][rds introduction to backups] for RDS
|
||||
- [Pricing and data retention for Performance Insights][rds pricing and data retention for performance insights] for
|
||||
RDS
|
||||
- [Exporting DB snapshot data to Amazon S3]
|
||||
- [I'm trying to export a snapshot from Amazon RDS MySQL to Amazon S3, but I'm receiving an error. Why is this happening?]
|
||||
- [Rotating AWS KMS keys]
|
||||
|
||||
<!--
|
||||
Reference
|
||||
@@ -376,7 +357,7 @@ Examples:
|
||||
[eventbridge]: #eventbridge
|
||||
[guardduty]: #guardduty
|
||||
[inspector]: #inspector
|
||||
[rds]: #rds
|
||||
[kms]: #kms
|
||||
[security hub]: #security-hub
|
||||
|
||||
<!-- Knowledge base -->
|
||||
@@ -386,6 +367,7 @@ Examples:
|
||||
[ecs]: ecs.md
|
||||
[eks]: eks.md
|
||||
[opensearch]: opensearch.md
|
||||
[rds]: rds.md
|
||||
[s3]: s3.md
|
||||
[sagemaker]: sagemaker.md
|
||||
|
||||
@@ -406,8 +388,7 @@ Examples:
|
||||
[iam json policy elements: sid]: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_sid.html
|
||||
[nat gateways]: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html
|
||||
[not authorized to perform: sts:assumerole]: https://repost.aws/questions/QUOY5XngCtRyOX4Desaygz8Q/not-authorized-to-perform-sts-assumerole
|
||||
[rds introduction to backups]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html
|
||||
[rds pricing and data retention for performance insights]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PerfInsights.Overview.cost.html
|
||||
[rotating aws kms keys]: https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html
|
||||
[services that publish cloudwatch metrics]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html
|
||||
[subnets for your vpc]: https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html
|
||||
[test your roles' access policies using the aws identity and access management policy simulator]: https://aws.amazon.com/blogs/security/test-your-roles-access-policies-using-the-aws-identity-and-access-management-policy-simulator/
|
||||
@@ -419,7 +400,6 @@ Examples:
|
||||
[what is amazon vpc?]: https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html
|
||||
[what is aws config?]: https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html
|
||||
[what is cloudwatch]: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html
|
||||
[working with db instance read replicas]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html
|
||||
|
||||
<!-- Others -->
|
||||
[automating dns-challenge based letsencrypt certificates with aws route 53]: https://johnrix.medium.com/automating-dns-challenge-based-letsencrypt-certificates-with-aws-route-53-8ba799dd207b
|
||||
|
||||
@@ -108,11 +108,6 @@ aws iam detach-user-policy --user-name 'me-user' \
|
||||
aws iam delete-user-policy --user-name 'me-user' --policy-name 'user-ro-access-bucket'
|
||||
|
||||
|
||||
# Show RDS instances.
|
||||
aws rds describe-db-instances
|
||||
aws rds describe-db-instances --output 'json' --query "DBInstances[?(DBInstanceIdentifier=='master-prod')]"
|
||||
|
||||
|
||||
# List hosted zones.
|
||||
aws route53 list-hosted-zones
|
||||
|
||||
@@ -147,6 +142,7 @@ Subcommands not listed here are in their own service-specific article:
|
||||
[`ec2`][ec2 tldr] |
|
||||
[`ecr`][ecr tldr] |
|
||||
[`eks`][eks tldr] |
|
||||
[`rds`][rds tldr] |
|
||||
[`s3`][s3 tldr] |
|
||||
[`ssm`][ssm tldr]
|
||||
|
||||
@@ -280,6 +276,7 @@ Solutions:
|
||||
[ec2 tldr]: ec2.md#tldr
|
||||
[ecr tldr]: ecr.md#tldr
|
||||
[eks tldr]: eks.md#tldr
|
||||
[rds tldr]: rds.md#tldr
|
||||
[s3 tldr]: s3.md#tldr
|
||||
[ssm tldr]: ssm.md#tldr
|
||||
|
||||
|
||||
294
knowledge base/cloud computing/aws/rds.md
Normal file
294
knowledge base/cloud computing/aws/rds.md
Normal file
@@ -0,0 +1,294 @@
|
||||
# Amazon Relational Database Service
|
||||
|
||||
1. [TL;DR](#tldr)
|
||||
1. [Backup](#backup)
|
||||
1. [Automatic backups](#automatic-backups)
|
||||
1. [Manual backups](#manual-backups)
|
||||
1. [Export snapshots to S3](#export-snapshots-to-s3)
|
||||
1. [Restore](#restore)
|
||||
1. [Encryption](#encryption)
|
||||
1. [Further readings](#further-readings)
|
||||
1. [Sources](#sources)
|
||||
|
||||
## TL;DR
|
||||
|
||||
<details>
|
||||
<summary>Usage</summary>
|
||||
|
||||
```sh
|
||||
# Show RDS instances.
|
||||
aws rds describe-db-instances
|
||||
aws rds describe-db-instances --output 'json' --query "DBInstances[?(DBInstanceIdentifier=='master-prod')]"
|
||||
|
||||
# Restore instances from snapshots.
|
||||
aws rds restore-db-instance-from-db-snapshot \
|
||||
--db-instance-identifier 'mynewdbinstance' \
|
||||
--db-snapshot-identifier 'mydbsnapshot'
|
||||
|
||||
# Start export tasks.
|
||||
aws rds start-export-task \
|
||||
--export-task-identifier 'db-finalSnapshot-2024' \
|
||||
--source-arn 'arn:aws:rds:eu-west-1:012345678901:snapshot:db-prod-final-2024' \
|
||||
--s3-bucket-name 'backups' \
|
||||
--iam-role-arn 'arn:aws:iam::012345678901:role/CustomRdsS3Exporter' \
|
||||
--kms-key-id 'arn:aws:kms:eu-west-1:012345678901:key/abcdef01-2345-6789-abcd-ef0123456789'
|
||||
|
||||
# Get export tasks' status.
|
||||
aws rds describe-export-tasks
|
||||
aws rds describe-export-tasks --export-task-identifier 'my-snapshot-export'
|
||||
|
||||
# Cancel tasks.
|
||||
aws rds cancel-export-task --export-task-identifier 'my_export'
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
Read replicas **can** be promoted to standalone DB instances.<br/>
|
||||
See [Working with DB instance read replicas].
|
||||
|
||||
Disk free metrics are available in CloudWatch.
|
||||
|
||||
One can choose any of the following retention periods for instances' Performance Insights data:
|
||||
|
||||
- 7 days (default, free tier).
|
||||
- _n_ months, where n is a number from 1 to 24.<br/>
|
||||
In CLI and IaC, this number must be _n*31_.
|
||||
|
||||
## Backup
|
||||
|
||||
RDS backup storage for each Region is calculated from both the automated backups and manual DB snapshots for that
|
||||
Region.<br/>
|
||||
Moving snapshots to other Regions increases the backup storage in the destination Regions.
|
||||
|
||||
Backups are stored in [S3].
|
||||
|
||||
Should one choose to retain automated backups when deleting DB instances, those backups are saved for the full retention
|
||||
period; otherwise, all automated backups are deleted with the instance.<br/>
|
||||
After automated backups are deleted, they **cannot** be recovered.
|
||||
|
||||
Should one choose to have RDS create a final DB snapshot before deleting a DB instance, one can use that or previously
|
||||
created manual snapshots to recover it.
|
||||
|
||||
### Automatic backups
|
||||
|
||||
Automatic backups are storage volume snapshots of **entire** DB instances.
|
||||
|
||||
Automatic backups are **enabled** by default.<br/>
|
||||
Setting the backup retention period to 0 disables them, setting it to a nonzero value (re)enables them.
|
||||
|
||||
> Enabling automatic backups takes the affected instances offline to have a backup created immediately.<br/>
|
||||
> It **will** cause outages.
|
||||
|
||||
Automatic backups occur **daily** during the instances' backup window, configured in 30 minute periods. Should backups
|
||||
require more time than allotted to the backup window, they will continue after the window ends and until they finish.
|
||||
|
||||
Backups are retained for up to 35 days (_backup retention period_).<br/>
|
||||
One can recover DB instances to any point in time from the backup retention period.
|
||||
|
||||
The backup window can't overlap with the weekly maintenance window for DB instance or Multi-AZ DB cluster.<br/>
|
||||
During automatic backup windows storage I/O might be suspended briefly while the backup process initializes.
|
||||
Initialization typically takes up to a few seconds. One might also experience elevated latencies for a few minutes
|
||||
during backups for Multi-AZ deployments.<br/>
|
||||
For MariaDB, MySQL, Oracle and PostgreSQL Multi-AZ deployments, I/O activity isn't suspended on the primary instance as
|
||||
the backup is taken from the standby.<br/>
|
||||
Automated backups might occasionally be skipped if instances or clusters are running heavy workloads at the time backups
|
||||
are supposed to start.
|
||||
|
||||
DB instances must be in the `available` state for automated backups to occur.<br/>
|
||||
Automated backups don't occur while DB instances are in other states (i.e., `storage_full`).
|
||||
|
||||
Automated backups aren't created while a DB instance or cluster is stopped.<br/>
|
||||
RDS doesn't include time spent in the stopped state when the backup retention window is calculated. This means backups
|
||||
can be retained longer than the backup retention period if a DB instance has been stopped.
|
||||
|
||||
Automated backups don't occur while a DB snapshot copy is running in the same AWS Region for the same database.
|
||||
|
||||
### Manual backups
|
||||
|
||||
Back up DB instances manually by creating DB snapshots.<br/>
|
||||
The first snapshot contains the data for the full database. Subsequent snapshots of the same database are incremental.
|
||||
|
||||
One can copy both automatic and manual DB snapshots, but only share manual DB snapshots.
|
||||
|
||||
Manual snapshots **never** expire and are retained indefinitely.
|
||||
|
||||
One can store up to 100 manual snapshots per Region.
|
||||
|
||||
### Export snapshots to S3
|
||||
|
||||
One can export DB snapshot data to [S3] buckets.<br/>
|
||||
RDS spins up an instance from the snapshot, extracts data from it and stores the data in Apache Parquet format.<br/>
|
||||
By default **all** data in the snapshots is exported, but one can specify specific sets of databases, schemas, or tables
|
||||
to export.
|
||||
|
||||
- The export process runs in the background and does **not** affect the performance of active DB instances.
|
||||
- Multiple export tasks for the same DB snapshot cannot run simultaneously. This applies to both full and partial
|
||||
exports.
|
||||
- Exporting snapshots from DB instances that use magnetic storage isn't supported.
|
||||
- The following characters aren't supported in table column names:
|
||||
|
||||
```plaintext
|
||||
, ; { } ( ) \n \t = (space) /
|
||||
```
|
||||
|
||||
Tables containing those characters in column names are skipped during export.
|
||||
- PostgreSQL _temporary_ and _unlogged_ tables are skipped during export.
|
||||
- Large objects in the data, like BLOBs or CLOBs, close to or greater than 500 MB will make the export fail.
|
||||
- Large rows close to or greater than 2 GB will make their table being skipped during export.
|
||||
- Data exported from snapshots to S3 cannot be restored to new DB instances.
|
||||
- The snapshot export tasks require a role with write-access permission to the destination S3 bucket:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [{
|
||||
"Effect": "Allow",
|
||||
"Action": "sts:AssumeRole",
|
||||
"Principal": {
|
||||
"Service": "export.rds.amazonaws.com"
|
||||
}
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"s3:PutObject*",
|
||||
"s3:ListBucket",
|
||||
"s3:GetObject*",
|
||||
"s3:DeleteObject*",
|
||||
"s3:GetBucketLocation"
|
||||
],
|
||||
"Resource": [
|
||||
"arn:aws:s3:::bucket",
|
||||
"arn:aws:s3:::bucket/*"
|
||||
]
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
After the export, one can analyze the data directly through
|
||||
[Athena](https://docs.aws.amazon.com/athena/latest/ug/parquet-serde.html) or
|
||||
[Redshift Spectrum](https://docs.aws.amazon.com/redshift/latest/dg/copy-usage_notes-copy-from-columnar.html).
|
||||
|
||||
<details>
|
||||
<summary>In the Console</summary>
|
||||
|
||||
The _Export to Amazon S3_ console option appears only for snapshots that can be exported to Amazon S3.<br/>
|
||||
Snapshots might not be available for export because of the following reasons:
|
||||
|
||||
- The DB engine isn't supported for S3 export.
|
||||
- The DB instance version isn't supported for S3 export.
|
||||
- S3 export isn't supported in the AWS Region where the snapshot was created.
|
||||
|
||||
</details>
|
||||
<details>
|
||||
<summary>Using the CLI</summary>
|
||||
|
||||
```sh
|
||||
# Start new tasks.
|
||||
$ aws rds start-export-task \
|
||||
--export-task-identifier 'db-finalSnapshot-2024' \
|
||||
--source-arn 'arn:aws:rds:eu-west-1:012345678901:snapshot:db-prod-final-2024' \
|
||||
--s3-bucket-name 'backups' --s3-prefix 'rds' \
|
||||
--iam-role-arn 'arn:aws:iam::012345678901:role/CustomRdsS3Exporter' \
|
||||
--kms-key-id 'arn:aws:kms:eu-west-1:012345678901:key/abcdef01-2345-6789-abcd-ef0123456789'
|
||||
{
|
||||
"ExportTaskIdentifier": "db-finalSnapshot-2024",
|
||||
"IamRoleArn": "arn:aws:iam::012345678901:role/CustomRdsS3Exporter",
|
||||
"KmsKeyId": "arn:aws:kms:eu-west-1:012345678901:key/abcdef01-2345-6789-abcd-ef0123456789",
|
||||
"PercentProgress": 0,
|
||||
"S3Bucket": "backups",
|
||||
"S3Prefix": "rds",
|
||||
"SnapshotTime": "2024-06-17T09:04:41.387000+00:00",
|
||||
"SourceArn": "arn:aws:rds:eu-west-1:012345678901:snapshot:db-prod-final-2024",
|
||||
"Status": "STARTING",
|
||||
"TotalExtractedDataInGB": 0
|
||||
}
|
||||
|
||||
# Get tasks' status.
|
||||
$ aws rds describe-export-tasks
|
||||
$ aws rds describe-export-tasks --export-task-identifier 'db-finalSnapshot-2024'
|
||||
$ aws rds describe-export-tasks --query 'ExportTasks[].WarningMessage' --output 'yaml'
|
||||
|
||||
# Cancel tasks.
|
||||
$ aws rds cancel-export-task --export-task-identifier 'my_export'
|
||||
{
|
||||
"Status": "CANCELING",
|
||||
"S3Prefix": "",
|
||||
"ExportTime": "2019-08-12T01:23:53.109Z",
|
||||
"S3Bucket": "DOC-EXAMPLE-BUCKET",
|
||||
"PercentProgress": 0,
|
||||
"KmsKeyId": "arn:aws:kms:AWS_Region:123456789012:key/K7MDENG/bPxRfiCYEXAMPLEKEY",
|
||||
"ExportTaskIdentifier": "my_export",
|
||||
"IamRoleArn": "arn:aws:iam::123456789012:role/export-to-s3",
|
||||
"TotalExtractedDataInGB": 0,
|
||||
"TaskStartTime": "2019-11-13T19:46:00.173Z",
|
||||
"SourceArn": "arn:aws:rds:AWS_Region:123456789012:snapshot:export-example-1"
|
||||
}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
## Restore
|
||||
|
||||
DB instances **can** be restored from DB snapshots.<br/>
|
||||
Instances **cannot** be restored with less storage.
|
||||
|
||||
When increasing allocated storage, increases must be by at least of 10%. Trying to increase the value by less than 10%
|
||||
will result in an error.<br/>
|
||||
The allocated storage **cannot** be increased when restoring RDS for SQL Server DB instances.
|
||||
|
||||
```sh
|
||||
aws rds restore-db-instance-from-db-snapshot \
|
||||
--db-instance-identifier 'mynewdbinstance' \
|
||||
--db-snapshot-identifier 'mydbsnapshot'
|
||||
```
|
||||
|
||||
## Encryption
|
||||
|
||||
RDS automatically integrates with AWS KMS for key management.
|
||||
|
||||
By default, RDS uses the RDS AWS managed key (`aws/rds`) for encryption.<br/>
|
||||
This key can't be managed, rotated, nor deleted by users.
|
||||
|
||||
RDS will automatically put databases into a terminal state when access to the KMS key is required but the key has been
|
||||
disabled or deleted, or its permissions have been somehow revoked.<br/>
|
||||
This change could be immediate or deferred depending on the use case that required access to the KMS key.<br/>
|
||||
In this terminal state, DB instances are no longer available and their databases' current state can't be recovered. To
|
||||
restore DB instances, one must first re-enable access to the KMS key for RDS, and then restore the instances from their
|
||||
latest available backup.
|
||||
|
||||
## Further readings
|
||||
|
||||
- [Working with DB instance read replicas]
|
||||
|
||||
### Sources
|
||||
|
||||
- [Pricing and data retention for Performance Insights]
|
||||
- [Introduction to backups]
|
||||
- [Restoring from a DB snapshot]
|
||||
- [AWS KMS key management]
|
||||
|
||||
<!--
|
||||
Reference
|
||||
═╬═Time══
|
||||
-->
|
||||
|
||||
<!-- In-article sections -->
|
||||
<!-- Knowledge base -->
|
||||
[s3]: s3.md
|
||||
|
||||
<!-- Files -->
|
||||
<!-- Upstream -->
|
||||
[aws kms key management]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Encryption.Keys.html
|
||||
[introduction to backups]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html
|
||||
[pricing and data retention for performance insights]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PerfInsights.Overview.cost.html
|
||||
[restoring from a db snapshot]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html
|
||||
[working with db instance read replicas]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html
|
||||
|
||||
<!-- Others -->
|
||||
@@ -1,6 +1,7 @@
|
||||
# Simple Storage Service
|
||||
|
||||
1. [TL;DR](#tldr)
|
||||
1. [Storage tiers](#storage-tiers)
|
||||
1. [Lifecycle configuration](#lifecycle-configuration)
|
||||
1. [Further readings](#further-readings)
|
||||
1. [Sources](#sources)
|
||||
@@ -8,7 +9,7 @@
|
||||
## TL;DR
|
||||
|
||||
<details>
|
||||
<summary>Common usage</summary>
|
||||
<summary>Usage</summary>
|
||||
|
||||
```sh
|
||||
# List all buckets.
|
||||
@@ -47,11 +48,18 @@ aws s3 cp 'file.txt' 's3://my-bucket/' \
|
||||
'full=id=79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be'
|
||||
aws s3 cp 'mydoc.txt' 's3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/mykey'
|
||||
|
||||
# Handling file streams.
|
||||
# Handle file streams.
|
||||
# Useful for piping:
|
||||
# - setting the source to '-' sends data from stdin
|
||||
# - setting the destination to '-' sends data to stdout
|
||||
aws s3 cp - 's3://my-bucket/stream.txt'
|
||||
aws s3 cp - 's3://my-bucket/stream.txt' --expected-size '54760833024'
|
||||
aws s3 cp 's3://my-bucket/stream.txt' -
|
||||
|
||||
# Directly print the contents of files to stdout.
|
||||
aws s3 cp --quiet 's3://my-bucket/file.txt' '-'
|
||||
aws s3 cp --quiet 's3://my-bucket/file.txt' '/dev/stdout'
|
||||
|
||||
# Remove objects.
|
||||
aws s3 rm 's3://my-bucket/prefix-name' --recursive --dryrun
|
||||
|
||||
@@ -105,20 +113,35 @@ aws s3api list-objects-v2 \
|
||||
|
||||
</details>
|
||||
|
||||
## Storage tiers
|
||||
|
||||
| | Standard | Intelligent-Tiering | Express One Zone | Standard Infrequent Access | One Zone Infrequent Access | Glacier Instant Retrieval | Glacier Flexible Retrieval | Glacier Deep Archive |
|
||||
| ---------------------- | ------------ | ------------------- | ------------------------- | -------------------------- | -------------------------- | ------------------------- | -------------------------- | -------------------- |
|
||||
| Retrieval charge | ✗ | ✗ | ✗ | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved | per GB retrieved |
|
||||
| Latency | milliseconds | milliseconds | single-digit milliseconds | milliseconds | milliseconds | milliseconds | minutes to hours | hours |
|
||||
| Minimum storage charge | ✗ | ✗ | 1 hour | 30 days | 30 days | 90 days | 90 days | 180 days |
|
||||
| Availability Zones | 3+ | 3+ | 1 | 3+ | 1 | 3+ | 3+ | 3+ |
|
||||
|
||||
## Lifecycle configuration
|
||||
|
||||
> Adding, removing or changing lifecycle rules takes a while.<br/>
|
||||
> Wait a couple of minutes after the operation to make sure all the bucket's properties are synced.
|
||||
|
||||
When one has multiple rules in an S3 Lifecycle configuration, an object can become eligible for multiple S3 Lifecycle actions. In such cases, Amazon S3 follows these general rules:
|
||||
When multiple rules are applied through S3 Lifecycle configurations, objects can become eligible for multiple S3
|
||||
Lifecycle actions. In such cases:
|
||||
|
||||
1. Permanent deletion takes precedence over transition.
|
||||
1. Transition takes precedence over creation of delete markers.
|
||||
1. When an object is eligible for both a S3 Glacier Flexible Retrieval and S3 Standard-IA (or S3 One Zone-IA) transition, Amazon S3 chooses the S3 Glacier Flexible Retrieval transition.
|
||||
1. Permanent deletion takes precedence over transitions.
|
||||
1. Transitions takes precedence over creation of delete markers.
|
||||
1. When objects are eligible for transition to both S3 Glacier Flexible Retrieval and S3 Standard-IA (or One Zone-IA),
|
||||
precedence is given to S3 Glacier Flexible Retrieval transition.
|
||||
|
||||
Propagation delay: When you add an S3 Lifecycle configuration to a bucket, there is usually some lag before a new or updated Lifecycle configuration is fully propagated to all the Amazon S3 systems. Expect a delay of a few minutes before the configuration fully takes effect. This delay can also occur when you delete an S3 Lifecycle configuration.
|
||||
When adding S3 Lifecycle configurations to buckets, there is usually some lag before a new or updated Lifecycle
|
||||
configuration is fully propagated to all the S3's systems.<br/>
|
||||
Expect a delay of a few minutes before any change in configuration fully takes effect. This includes configuration
|
||||
deletions.
|
||||
|
||||
Objects can only go down the tiers, not up. Many other constraints apply, like no transition done for objects <128KiB.<br/>
|
||||
Objects can only go down the tiers, not up.<br/>
|
||||
Other constraints apply, like no transition done for objects smaller than 128KiB.<br/>
|
||||
See [General considerations for transitions][lifecycle general considerations for transitions].
|
||||
|
||||
Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules examples]
|
||||
@@ -132,6 +155,7 @@ Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules example
|
||||
|
||||
### Sources
|
||||
|
||||
- [Amazon S3 Storage Classes]
|
||||
- [General considerations for transitions][lifecycle general considerations for transitions]
|
||||
- [Lifecycle configuration examples][lifecycle configuration examples]
|
||||
- [CLI subcommand reference]
|
||||
@@ -139,7 +163,8 @@ Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules example
|
||||
- [How S3 Intelligent-Tiering works]
|
||||
|
||||
<!--
|
||||
References
|
||||
Reference
|
||||
═╬═Time══
|
||||
-->
|
||||
|
||||
<!-- In-article sections -->
|
||||
@@ -151,6 +176,7 @@ Examples: [1][lifecycle configuration examples], [2][s3 lifecycle rules example
|
||||
[s3 lifecycle rules examples]: ../../../examples/aws/s3.lifecycle-rules
|
||||
|
||||
<!-- Upstream -->
|
||||
[amazon s3 storage classes]: https://aws.amazon.com/s3/storage-classes/
|
||||
[cli subcommand reference]: https://docs.aws.amazon.com/cli/latest/reference/s3/
|
||||
[expiring amazon s3 objects based on last accessed date to decrease costs]: https://aws.amazon.com/blogs/architecture/expiring-amazon-s3-objects-based-on-last-accessed-date-to-decrease-costs/
|
||||
[find out the size of your amazon s3 buckets]: https://aws.amazon.com/blogs/storage/find-out-the-size-of-your-amazon-s3-buckets/
|
||||
|
||||
@@ -99,3 +99,16 @@ set instance_id 'i-0915612f182914822' \
|
||||
|
||||
aws imagebuilder list-image-recipes
|
||||
aws imagebuilder get-image-recipe --image-recipe-arn 'arn:aws:imagebuilder:eu-west-1:012345678901:image-recipe/my-custom-image/1.0.12'
|
||||
|
||||
|
||||
aws rds start-export-task \
|
||||
--export-task-identifier 'db-finalSnapshot-2024' \
|
||||
--source-arn 'arn:aws:rds:eu-west-1:012345678901:snapshot:db-prod-final-2024' \
|
||||
--s3-bucket-name 'backups' --s3-prefix 'rds' \
|
||||
--iam-role-arn 'arn:aws:iam::012345678901:role/CustomRdsS3Exporter' \
|
||||
--kms-key-id 'arn:aws:kms:eu-west-1:012345678901:key/abcdef01-2345-6789-abcd-ef0123456789'
|
||||
|
||||
# Max 5 running at any given time, RDS cannot queue
|
||||
echo {1..5} | xargs -p -n '1' -I '{}' aws rds start-export-task …
|
||||
|
||||
aws rds describe-export-tasks --query 'ExportTasks[].WarningMessage' --output 'json'
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
/**
|
||||
* https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html
|
||||
* https://repost.aws/knowledge-center/rds-mysql-export-snapshot
|
||||
*/
|
||||
|
||||
|
||||
Reference in New Issue
Block a user