diff --git a/knowledge base/peerdb.md b/knowledge base/peerdb.md new file mode 100644 index 0000000..fe44a2a --- /dev/null +++ b/knowledge base/peerdb.md @@ -0,0 +1,65 @@ +# PeerDB + +Fast, simple, and cost effective Postgres replication. + +1. [TL;DR](#tldr) +1. [Further readings](#further-readings) + 1. [Sources](#sources) + +## TL;DR + + + + + + + +## Further readings + +- [Website] +- [Main repository] +- [Documentation] + +### Sources + +- [Public IPs For PeerDB Cloud] + + + + + + + +[documentation]: https://docs.peerdb.io/ +[main repository]: https://github.com/PeerDB-io/peerdb +[website]: https://www.peerdb.io/ +[public ips for peerdb cloud]: https://docs.peerdb.io/peerdb-cloud/ip-table + + diff --git a/knowledge base/pg_flo.md b/knowledge base/pg_flo.md new file mode 100644 index 0000000..1e87b3e --- /dev/null +++ b/knowledge base/pg_flo.md @@ -0,0 +1,117 @@ +# `pg_flo` + +Move and transform data between PostgreSQL databases using Logical Replication. + +1. [TL;DR](#tldr) +1. [Further readings](#further-readings) + 1. [Sources](#sources) + +## TL;DR + +
+ Setup + +
+ Check requirements + +```sql +sourceDb=> SELECT name,setting FROM pg_settings WHERE name IN ('wal_level','rds.logical_replication'); + name | setting +-------------------------+--------- + rds.logical_replication | on + wal_level | logical +(2 rows) +``` + +
+ +```sh +docker pull 'nats' && docker pull 'shayonj/pg_flo' +``` + +
+ Configuration file + +[Reference][configuration file reference] + +```yaml +# Replicator settings +host: "localhost" +port: 5432 +dbname: "myapp" +user: "replicator" +password: "secret" +group: "users" +tables: + - "users" + +# Worker settings (postgres sink) +target-host: "dest-db" +target-dbname: "myapp" +target-user: "writer" +target-password: "secret" + +# Common settings +nats-url: "nats://localhost:4222" +``` + +
+ +
+ +
+ Usage + +```sh +# Start NATS server +docker run -d --name 'pg_flo_nats' --network 'host' -v "$PWD/config/nats-server.conf:/etc/nats/nats-server.conf" \ + 'nats' -c '/etc/nats/nats-server.conf' + +# Start replicator (using config file) +docker run -d --name 'pg_flo_replicator' --network 'host' -v "$PWD/config/pg_flo.yaml:/etc/pg_flo/config.yaml" \ + 'shayonj/pg_flo' replicator --config '/etc/pg_flo/config.yaml' + +# Start worker +docker run -d --name 'pg_flo_worker' --network 'host' -v "$PWD/config/pg_flo.yaml:/etc/pg_flo/config.yaml" \ + 'shayonj/pg_flo' worker postgres --config '/etc/pg_flo/config.yaml' +``` + +
+ + + +## Further readings + +- [Website] +- [Main repository] +- [Transformation rules] + +### Sources + +- [How to set the wal_level in AWS RDS Postgresql?] +- [Configuration file reference] + + + + + + + +[configuration file reference]: https://github.com/shayonj/pg_flo/blob/main/internal/pg-flo.yaml +[main repository]: https://github.com/shayonj/pg_flo +[transformation rules]: https://github.com/shayonj/pg_flo/blob/main/pkg/rules/README.md +[website]: https://www.pgflo.io/ + + +[How to set the wal_level in AWS RDS Postgresql?]: https://dba.stackexchange.com/questions/238686/how-to-set-the-wal-level-in-aws-rds-postgresql#243576