diff --git a/knowledge base/cloud computing/aws/rds.md b/knowledge base/cloud computing/aws/rds.md
index 66a0537..21c3c2e 100644
--- a/knowledge base/cloud computing/aws/rds.md
+++ b/knowledge base/cloud computing/aws/rds.md
@@ -13,13 +13,15 @@
1. [Operations](#operations)
1. [PostgreSQL](#postgresql)
1. [Reduce allocated storage by migrating using transportable databases](#reduce-allocated-storage-by-migrating-using-transportable-databases)
+1. [Troubleshooting](#troubleshooting)
+ 1. [ERROR: extension must be loaded via shared\_preload\_libraries](#error-extension-must-be-loaded-via-shared_preload_libraries)
1. [Further readings](#further-readings)
- 1. [Sources](#sources)
+ 1. [Sources](#sources)
## TL;DR
- Usage
+ CLI usage
```sh
# Show RDS instances.
@@ -64,6 +66,7 @@ aws rds cancel-export-task --export-task-identifier 'my_export'
```
+
Read replicas **can** be promoted to standalone DB instances.
See [Working with DB instance read replicas].
@@ -108,12 +111,14 @@ Decrease the storage size of DB instances by creating a new instance with lower
the data into the new instance.
Use one of the following methods:
-- Use the database engine's native dump and restore method.
- Consider using [transportable DBs][migrating databases using rds postgresql transportable databases] when dealing with
+- Use the database engine's native dump and restore method.
+ This **will** require **long** downtime.
+- Consider using [transportable DBs][migrating databases using rds postgresql transportable databases] when dealing with
PostgreSQL DBs should the requirements match.
- This **will** require downtime.
+ This **will** require **_some_** downtime.
- [Perform an homogeneous data migration][migrating databases to their amazon rds equivalents with aws dms] using AWS's
- [DMS][what is aws database migration service?] for minimal downtime.
+ [DMS][what is aws database migration service?]
+ This **should** require **minimal** downtime.
## Parameter Groups
@@ -351,8 +356,41 @@ Refer [Migrating databases using RDS PostgreSQL Transportable Databases],
[Transporting PostgreSQL databases between DB instances] and
[Transport PostgreSQL databases between two Amazon RDS DB instances using pg_transport].
-Requires a **source** DB and a **target** DB.
-Suggested the use of an EC2 instance in the middle to operate on both DBs.
+> When the transport begins, all current sessions on the **source** database are ended and the DB is put in ReadOnly
+> mode.
+> Only the specific source database that is being transported is affected. Others are **not** affected.
+>
+> The in-transit database will be **inaccessible** on the **target** DB instance for the duration of the transport.
+> During transport, the target DB instance **cannot be restored** to a point in time, as the transport is **not**
+> transactional and does **not** use the PostgreSQL write-ahead log to record changes.
+
+A test transfer of a ~350 GB database between two `db.t4g.medium` instances using gp3 storage took FIXME minutes.
+
+
+ Requirements
+
+- A **source** DB to copy from.
+- A **target** instance to copy the DB to.
+
+ > Since the transport will create the DB on the target, the target instance must **not** contain the database that
+ > needs to be transported.
+ > Should the target contain the DB already, it **will** need to be dropped beforehand.
+
+- The transported DB (but not _other_ DBs on the same source instance) to:
+
+ - Be put in **Read Only** mode.
+ - Have all installed extensions **removed**.
+
+To avoid locking the operator's machine for the time needed by the transport, it is suggested the use of an EC2 instance
+in the middle to operate on both DBs.
+
+> Try and keep the DBs identifiers under 22 characters.
+> PostgreSQL will try and truncate the identifier after 63 characters, and AWS will add something like
+> `.{{12-char-id}}.{{region}}.rds.amazonaws.com` to it.
+
+
+
+ Procedure
1. Enable the required configuration parameters and `pg_transport` extension on the source and target RDS instances.
Create a new RDS Parameter Group or modify the existing one used by the source.
@@ -380,33 +418,35 @@ Suggested the use of an EC2 instance in the middle to operate on both DBs.
1. Connect to the DB:
```sh
- psql -h 'source-instance.111122223333.eu-west-1.rds.amazonaws.com' -p '5432' -U 'admin' --password
+ psql -h 'source-instance.5f7mp3pt3n6e.eu-west-1.rds.amazonaws.com' -p '5432' -U 'admin' --password
+ psql -h 'target-instance.5f7mp3pt3n6e.eu-west-1.rds.amazonaws.com' -p '5432' -U 'admin' --password 'postgres'
```
1. **Remove** all extensions but `pg_transport` from the public schema of the DB instance.
Only the `pg_transport` extension is allowed during the actual transport operation.
```sql
- SELECT * FROM pg_extension;
- DROP EXTENSION 'pggeo', '…';
+ SELECT "extname" FROM "pg_extension";
+ DROP EXTENSION "plpgsql", "postgis", "…" CASCADE;
```
1. Install the `pg_transport` extension if missing:
```sql
- CREATE EXTENSION pg_transport;
- CREATE EXTENSION;
+ CREATE EXTENSION "pg_transport";
```
1. \[optional] Test the transport by running the `transport.import_from_server` function on the **target** DB instance:
```sql
+ -- Keep arguments in *single* quotes here
SELECT transport.import_from_server(
- 'source-instance.111122223333.eu-west-1.rds.amazonaws.com', 5432,
- 'admin', 'SourcePassword', 'mySourceDb',
- 'destination-user-password',
+ 'source-instance.5f7mp3pt3n6e.eu-west-1.rds.amazonaws.com', 5432,
+ 'admin', 'source-user-password', 'mySourceDb',
+ 'target-user-password',
true
);
+ ```
1. Run the transport by running the `transport.import_from_server` function on the **target** DB instance:
@@ -414,13 +454,14 @@ Suggested the use of an EC2 instance in the middle to operate on both DBs.
SELECT transport.import_from_server( …, …, …, …, …, …, false);
```
-> When the transport begins, all current sessions on the **source** database are ended and the DB is put in ReadOnly
-> mode.
-> Only the specific source database that is being transported is affected.
->
-> The in-transit database will be **inaccessible** on the **target** DB instance for the duration of the transport.
-> During transport, the target DB instance **cannot be restored** to a point in time: the transport is **not**
-> transactional and does **not** use the PostgreSQL write-ahead log to record changes.
+1. Validate the data in the target.
+1. Add all the needed roles and permissions to the target.
+1. Restore uninstalled extensions in the public schema of **both** DB instances.
+ `pg_transport` _can_ be uninstalled.
+1. Revert the value of the max_worker_processes parameter.
+
+
+
If the target DB instance has automatic backups enabled, a backup is automatically taken after transport completes.
Point-in-time restores will be available for times after the backup finishes.
@@ -434,10 +475,17 @@ happen, allow write-enabled queries manually:
ALTER DATABASE db-name SET default_transaction_read_only = false;
```
+## Troubleshooting
+
+### ERROR: extension must be loaded via shared_preload_libraries
+
+Refer [How can I resolve the "ERROR: must be loaded via shared_preload_libraries" error?]
+
## Further readings
- [Working with DB instance read replicas]
- [Working with parameter groups]
+- [How can I resolve the "ERROR: must be loaded via shared_preload_libraries" error?]
### Sources
@@ -466,6 +514,7 @@ ALTER DATABASE db-name SET default_transaction_read_only = false;
[amazon rds db instance storage]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html
[aws kms key management]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Encryption.Keys.html
[how can i decrease the total provisioned storage size of my amazon rds db instance?]: https://repost.aws/knowledge-center/rds-db-storage-size
+[how can i resolve the "error: must be loaded via shared_preload_libraries" error?]: https://repost.aws/knowledge-center/rds-postgresql-resolve-preload-error
[introduction to backups]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html
[migrating databases to their amazon rds equivalents with aws dms]: https://docs.aws.amazon.com/dms/latest/userguide/data-migrations.html
[migrating databases using rds postgresql transportable databases]: https://aws.amazon.com/blogs/database/migrating-databases-using-rds-postgresql-transportable-databases/
diff --git a/snippets/postgresql.sql b/snippets/postgresql.sql
new file mode 100644
index 0000000..91a6bc4
--- /dev/null
+++ b/snippets/postgresql.sql
@@ -0,0 +1,33 @@
+-- Close the connection to the current DB
+\q
+
+-- Show extensions.
+SELECT "*" FROM "pg_extension";
+SELECT "extname" FROM "pg_extension";
+
+-- Add extensions
+CREATE EXTENSION "pg_transport";
+CREATE EXTENSION IF NOT EXISTS "pgaudit";
+
+-- Remove extensions
+DROP EXTENSION "plpgsql", "btree_gist", … CASCADE;
+
+
+-- Simulate DB transfers
+-- Requires 'pg_transport' to be installed on both the source and destination DBs
+-- Requires 'pg_transport' to be the *only* extension active on the source
+SELECT transport.import_from_server(
+ 'up.to.63.chars.source.fqdn', 5432,
+ 'source.username', 'source.password', 'source.db',
+ 'destination.password',
+ true
+);
+-- Run DB transfers
+-- Requires 'pg_transport' to be installed on both the source and destination DBs
+-- Requires 'pg_transport' to be the *only* extension active on the source
+SELECT transport.import_from_server(
+ 'up.to.63.chars.source.fqdn', 5432,
+ 'source.username', 'source.password', 'source.db',
+ 'destination.password',
+ false
+);