diff --git a/knowledge base/gcp/cloud sql.md b/knowledge base/gcp/cloud sql.md new file mode 100644 index 0000000..8ea9885 --- /dev/null +++ b/knowledge base/gcp/cloud sql.md @@ -0,0 +1,115 @@ +# Cloud SQL + +## Table of contents + +1. [TL;DR](#tldr) +1. [Connect to a cloud SQL instance](#connect-to-a-cloud-sql-instance) +1. [Create users in a SQL instance from the MySQL shell](#create-users-in-a-sql-instance-from-the-mysql-shell) +1. [Manually execute a `terraform plan` or `apply` on a project defining Cloud SQL instances and users](#manually-execute-a-terraform-plan-or-apply-on-a-project-defining-cloud-sql-instances-and-users) +1. [Gotchas](#gotchas) +1. [Further readings](#further-readings) +1. [Sources](#sources) + +## TL;DR + +```sh +# Connect to cloud SQL instances. +gcloud sql connect 'instance-name' --user='root' --quiet + +# Connect to cloud SQL instances trough local proxy. +# brew install 'cloud_sql_proxy' +cloud_sql_proxy -instances=project-name:region:instance-name=tcp:3306 +cloud_sql_proxy -instances=project-name:region:instance-name -dir=/tmp \ + -verbose -log_debug_stdout +``` + +## Connect to a cloud SQL instance + +```sh +$ gcloud sql connect $INSTANCE_NAME --user=root --quiet +Allowlisting your IP for incoming connection for 5 minutes...done. +Connecting to database with SQL user [root].Enter password: +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 293 +Server version: 8.0.18-google (Google) + +Copyright (c) 2000, 2021, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> +``` + +## Create users in a SQL instance from the MySQL shell + +1. create an administrative user for the instance using `gcloud`, the APIs or the console +1. use this administrative user to connect to the MySQL console: + + ```shell + mysql -h $HOST -u admin -p + ``` + +1. create the new users from there + +## Manually execute a `terraform plan` or `apply` on a project defining Cloud SQL instances and users + +- make sure the SQL instance has been created (using a IaC tool or not, it doesn't matter) +- install `cloud_sql_proxy` on your machine: + + ```sh + brew install 'cloud_sql_proxy' + ``` + +- start the proxy and point it to the SQL instance the code needs to connect to + + ```sh + $ cloud_sql_proxy -instances=${PROJECT}:${REGION}:${INSTANCE}=tcp:3306 -verbose -log_debug_stdout + 2021/04/20 10:49:03 Rlimits for file descriptors set to {Current = 8500, Max = 9223372036854775807} + 2021/04/20 10:49:05 Listening on 127.0.0.1:3306 for myAwesomeProject:europe-west4:sqlInstance + 2021/04/20 10:49:05 Ready for new connections + + # or, using sockets + $ cloud_sql_proxy -instances=${PROJECT}:${REGION}:${INSTANCE} -dir=/tmp -verbose -log_debug_stdout + 2021/05/19 23:13:40 Rlimits for file descriptors set to {Current = 8500, Max = 9223372036854775807} + 2021/05/19 23:13:41 Listening on /tmp/myAwesomeProject:europe-west4:sqlInstance for myAwesomeProject:europe-west4:sqlInstance + 2021/05/19 23:13:41 Ready for new connections + ``` + +- make the Terraform SQL provider point to localhost + + ```hcl + provider "mysql" { + # endpoint = google_sql_database_instance.sqlInstance.first_ip_address + # endpoint = "127.0.0.1" + endpoint = "/tmp/myAwesomeProject:europe-west4:sqlInstance" + username = "admin" + password = var.sql_password + version = "~> 1.9" + } + ``` + +- execute `terraform plan` from your machine + +Terraform will use the provider to connect to the proxy and operate on the SQL instance. + +## Gotchas + +- As of 2021-05-18 the `root` user will **not be able** to create other users from the MySQL shell because it will lack `CREATE USER` permissions. +- The documentation says that SQL users created using `gcloud`, the APIs or the cloud console will have the same permissions of the `root` user; in reality, those administrative entities will be able to create users only from the MySQL shell. + +## Further readings + +## Sources + +All the references in the [further readings] section, plus the following: + + + + +[further readings]: #further-readings + + diff --git a/knowledge base/gcp/config connector.md b/knowledge base/gcp/config connector.md new file mode 100644 index 0000000..8a7200f --- /dev/null +++ b/knowledge base/gcp/config connector.md @@ -0,0 +1,65 @@ +# Config Connector + +Kubernetes addon to manage Google Cloud resources from inside Kubernetes clusters. + +Provides a collection of Custom Resource Definitions and controllers. + +## Table of contents + +## TL:DR + +```sh +# List gcp resources one can create using config connector. +# Requires config connector to be installed. +kubectl get crds --selector 'cnrm.cloud.google.com/managed-by-kcc=true' +``` + +## Installation + +1. Refer to: + + - the [installation howto] for details and updated instructions if you are using GKE; + - the [installation types] page for details and updated instructions for other K8S clusters. + +1. Enable the Resource Manager API: + + ```sh + gcloud services enable 'cloudresourcemanager.googleapis.com' + ``` + +## Resources management + +List what Google Cloud [resources] you can create with Config Connector: + +```sh +kubectl get crds --selector cnrm.cloud.google.com/managed-by-kcc=true +``` + +## Gotchas + +- Service accounts can be granted _editor_ access by replacing `--role="roles/owner"` with `--role="roles/editor"`; this allows **most** Config Connector functionality, except project and organization wide configurations such as IAM modifications. +- When creating a resource, Config Connector creates it if it doesn't exist; if a resource already exists with the same name, then Config Connector acquires and manages it instead. + +## Further readings + +- [Website] +- [Getting started] + +## Sources + +All the references in the [further readings] section, plus the following: + + +[getting started]: https://cloud.google.com/config-connector/docs/how-to/getting-started +[installation howto]: https://cloud.google.com/config-connector/docs/how-to/install-upgrade-uninstall +[installation types]: https://cloud.google.com/config-connector/docs/concepts/installation-types +[overview]: https://cloud.google.com/config-connector/docs/overview +[resources]: https://cloud.google.com/config-connector/docs/reference/overview +[stackdriver]: https://cloud.google.com/stackdriver/docs/solutions/gke +[website]: https://cloud.google.com/config-connector +[workload identity]: https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity + + +[further readings]: #further-readings + + diff --git a/knowledge base/gcp/gcloud.md b/knowledge base/gcp/gcloud.md new file mode 100644 index 0000000..0adc27e --- /dev/null +++ b/knowledge base/gcp/gcloud.md @@ -0,0 +1,69 @@ +# Title + +Intro + +## Table of contents + +1. [TL:DR](#tldr) +1. [Further readings](#further-readings) +1. [Sources](#sources) + +## TL:DR + +```sh +# List all project the current user has access to. +gcloud projects list --sort-by=projectId + +# Delete projects. +gcloud projects delete 'project-name' + +# Undo delete project operations. +# Available for a limited period of time. +gcloud projects undelete 'project-name' + +# Add the pubsub admin role to the 'awesome-sa' service account in the +# 'gcp-project' project. +gcloud projects add-iam-policy-binding 'gcp-project' \ + --member "serviceAccount:awesome-sa@gcp-project.iam.gserviceaccount.com" \ + --role "roles/pubsub.admin" + +# Remove the pubsub subscriber role from the 'awesome-sa' service account in the gcpproject project +gcloud projects remove-iam-policy-binding 'gcp-project' \ + --member="serviceAccount:awesome-sa@gcp-project.iam.gserviceaccount.com" \ + --role="roles/pubsub.subscriber" + +# Get all Kubernetes versions available for use in gke clusters. +gcloud container get-server-config --format "yaml(validNodeVersions)" +gcloud container get-server-config --format "yaml(validMasterVersions)" --zone 'compute-zone' +gcloud container get-server-config --flatten="channels" --filter="channels.channel=RAPID" --format="yaml(channels.channel,channels.validVersions)" + +# Generate 'kubeconfig' entries for gke clusters. +gcloud container clusters get-credentials 'cluster-name' +gcloud container clusters get-credentials 'cluster-name' --region 'region' + +# SSH into compute instances. +# Includes gke clusters' compute instances. +gcloud compute ssh 'instance-name' --zone 'zone' + +# Connect to cloud SQL instances. +gcloud sql connect 'instance-name' --user='root' --quiet +``` + +## Further readings + +- [Creating and managing projects] + +## Sources + +All the references in the [further readings] section, plus the following: + +- [`gcloud projects`][gcloud projects] + + +[creating and managing projects]: https://cloud.google.com/resource-manager/docs/creating-managing-projects +[gcloud projects]: https://cloud.google.com/sdk/gcloud/reference/projects + + +[further readings]: #further-readings + + diff --git a/knowledge base/gcp/gke.md b/knowledge base/gcp/gke.md new file mode 100644 index 0000000..c66164e --- /dev/null +++ b/knowledge base/gcp/gke.md @@ -0,0 +1,112 @@ +# Google Kubernetes Engine + +Managed Kubernetes solution offered by the Google Cloud Platform. + +## Table of contents + +1. [TL:DR](#tldr) +1. [Gotchas](#gotchas) +1. [SSH into GKE clusters' compute instances](#ssh-into-gke-clusters-compute-instances) +1. [Further readings](#further-readings) +1. [Sources](#sources) + +## TL:DR + +```sh +# Generate 'kubeconfig' entries for gke clusters. +gcloud container clusters get-credentials 'cluster-name' +gcloud container clusters get-credentials 'cluster-name' --region 'region' + +# Get all Kubernetes versions available for use in gke clusters. +gcloud container get-server-config --format "yaml(validNodeVersions)" +gcloud container get-server-config --format "yaml(validMasterVersions)" --zone 'compute-zone' +gcloud container get-server-config --flatten="channels" --filter="channels.channel=RAPID" --format="yaml(channels.channel,channels.validVersions)" + +# SSH into gke clusters' compute instances. +gcloud compute ssh 'instance-name' --zone 'zone' +``` + +## Gotchas + +- When creating admission webhooks, either make sure to expose your webhook service and deployments on port 443 or poke a hole in the firewall for the port they are listening to.
+ By default, firewall rules restrict the cluster's masters communication to nodes only on ports 443 (HTTPS) and 10250 (kubelet). Additionally, GKE enables the `enable-aggregator-routing` option by default, which makes the master to bypass the service and communicate straight to pods. + +## SSH into GKE clusters' compute instances + +Use the same procedure to connect to any other compute instance: + +```sh +$ gcloud compute ssh 'gke-euwe4-my-instance' +WARNING: The private SSH key file for gcloud does not exist. +WARNING: The public SSH key file for gcloud does not exist. +WARNING: You do not have an SSH key for gcloud. +WARNING: SSH keygen will be executed to generate a key. +Generating public/private rsa key pair. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /Users/you/.ssh/google_compute_engine. +Your public key has been saved in /Users/you/.ssh/google_compute_engine.pub. +The key fingerprint is: +SHA256:cbYuJKZROlbzX2wuzzN4zd3OGu6m7CupYKJHdiYOxVw you@machine +The key's randomart image is: ++---[RSA 3072]----+ +| | +| E | +| o .+ . o | +| ++ o + o | +| .= o S . + | +| ..+=oo o + | +| =o+o . +o.o...| +| .oo . .+=+.+oo| +| .. .. +BB+oo| ++----[SHA256]-----+ +No zone specified. Using zone [europe-west4-c] for instance: [gke-euwe4-my-instance]. +External IP address was not found; defaulting to using IAP tunneling. +Updating project ssh metadata...â ¹Updated [https://www.googleapis.com/compute/v1/projects/gcp-project]. +Updating project ssh metadata...done. +Waiting for SSH key to propagate. +Warning: Permanently added 'compute.4401449885042934396' (ED25519) to the list of known hosts. +Enter passphrase for key '/Users/you/.ssh/google_compute_engine': +Enter passphrase for key '/Users/you/.ssh/google_compute_engine': + +Welcome to Kubernetes v1.16.15-gke.6000! + +You can find documentation for Kubernetes at: + http://docs.kubernetes.io/ + +The source for this release can be found at: + /home/kubernetes/kubernetes-src.tar.gz +Or you can download it at: + https://storage.googleapis.com/kubernetes-release-gke/release/v1.16.15-gke.6000/kubernetes-src.tar.gz + +It is based on the Kubernetes source at: + https://github.com/kubernetes/kubernetes/tree/v1.16.15-gke.6000 + +For Kubernetes copyright and licensing information, see: + /home/kubernetes/LICENSES + +[instance]$ +``` + +## Further readings + +- [How to Master Admission Webhooks In Kubernetes] +- [Kubectl cluster access] + +## Sources + +All the references in the [further readings] section, plus the following: + +- [Connect to a compute instance] +- [Preparing a Google Kubernetes Engine environment for production] + + +[connect to a compute instance]: https://cloud.google.com/compute/docs/instances/connecting-to-instance +[kubectl cluster access]: https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl +[preparing a google kubernetes engine environment for production]: https://cloud.google.com/solutions/prep-kubernetes-engine-for-prod + + +[further readings]: #further-readings + + +[how to master admission webhooks in kubernetes]: https://digizoo.com.au/1376/mastering-admission-webhooks-in-kubernetes-gke-part-1/