chore(gitlab): signals, improve upgrade commands

This commit is contained in:
Michele Cereda
2024-10-29 23:33:54 +01:00
parent 226c3e20f0
commit 2e5e2920fc
7 changed files with 104 additions and 3 deletions

View File

@@ -186,6 +186,14 @@ sudo yum check-update
sudo gitlab-backup create
tmux new-session -As 'gitlab-upgrade' "sudo yum update 'gitlab-ee'"
# DB version upgrade
sudo gitlab-ctl pg-upgrade
sudo gitlab-ctl pg-upgrade -V '16'
# Check there is enough disk space for two copies of the database
test $(( $(sudo du -s '/var/opt/gitlab/postgresql/data' | awk '{print $1}') * 2 )) -lt \
$(sudo df --output='avail' --direct '/var/opt/gitlab/postgresql/data' | tail -n 1) \
&& sudo gitlab-ctl pg-upgrade -V '16'
# Reset the root user's password.
sudo gitlab-rake 'gitlab:password:reset[root]'
sudo gitlab-rails console
@@ -752,6 +760,7 @@ Solution: set the correct ownership with
- [GitLab HA Scaling Runner Vending Machine for AWS EC2 ASG]
- [GitLab maintenance mode]
- [Forks]
- [Upgrade packaged PostgreSQL server]
<!--
Reference
@@ -803,6 +812,7 @@ Solution: set the correct ownership with
[tls]: https://docs.gitlab.com/charts/installation/tls.html
[tutorial: use buildah in a rootless container with gitlab runner operator on openshift]: https://docs.gitlab.com/ee/ci/docker/buildah_rootless_tutorial.html
[uninstall the linux package (omnibus)]: https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/doc/installation/index.md#uninstall-the-linux-package-omnibus
[upgrade packaged postgresql server]: https://docs.gitlab.com/omnibus/settings/database.html#upgrade-packaged-postgresql-server
[upgrade path tool]: https://gitlab-com.gitlab.io/support/toolbox/upgrade-path/
[use kaniko to build docker images]: https://docs.gitlab.com/ee/ci/docker/using_kaniko.html

View File

@@ -42,13 +42,17 @@ gitlab-runner exec docker \
--env 'DOCKER_AUTH_CONFIG={ "credsStore": "ecr-login" }' \
--docker-volumes "$HOME/.aws/credentials:/root/.aws/credentials:ro"
'job-requiring-ecr-access'
# Force a configuration file reload.
sudo kill -HUP $(pidof 'gitlab-runner')
sudo kill -s 'SIGHUP' $(pidof 'gitlab-runner')
```
</details>
Each runner executor is assigned 1 task at a time by default.
Runners seem to require the main instance to give the full certificate chain upon connection.
Runners require the main instance to give the full certificate chain upon connection.
The `runners.autoscaler.policy.periods` setting appears to be a full blown cron job, not just a time frame.
@@ -78,6 +82,23 @@ Meaning it will reapply policy 1 at the 31st minute of every hour in the period
</details>
One can use system signals to interact with runners.
<details style="margin-top: -1em; padding: 0 0 1em 1em;">
| Signal | Command it operates on | Effect | Example |
| ---------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------- | -------------------------------------- |
| `SIGINT` | `register` | Cancel ongoing runner registrations.<br/>Delete runners if already registered. | |
| `SIGINT`<br/>`SIGTERM` | `run`<br/>`run-single` | Abort all running builds and exit as soon as possible.<br/>Use twice to exit immediately (forceful shutdown). | |
| `SIGQUIT` | `run`<br/>`run-single` | Stop accepting new builds and exit as soon as currently running builds finish (graceful shutdown). | `sudo kill -SIGQUIT <main_runner_pid>` |
| `SIGHUP` | run | Force reloading the configuration file. | `sudo kill -SIGHUP <main_runner_pid>` |
> Do **not** use `killall` or `pkill` for graceful shutdowns if one is using the `shell` or `docker` executors.<br/>
> This causes improper handling of the signals due to subprocesses being killed as well. Only use it on the main process
> handling the jobs.
</details>
## Pull images from private AWS ECR registries
1. Create an IAM Role in one's AWS account and attach it the
@@ -313,8 +334,8 @@ Procedure:
</details>
1. Install the gitlab runner on the **manager** instance.<br/>
Configure it to use the `docker-autoscaler` executor.
1. Install the gitlab runner on the **manager** instance.
1. Configure the runner to use the `docker-autoscaler` executor.
<details style="margin-top: -1em; padding-bottom: 1em;">
@@ -849,6 +870,7 @@ Improvements:
- [Autoscaling GitLab Runner on AWS EC2]
- [Instance executor]
- [Docker Autoscaler executor]
- [Signals]
<!--
Reference
@@ -876,6 +898,7 @@ Improvements:
[install and register gitlab runner for autoscaling with docker machine]: https://docs.gitlab.com/runner/executors/docker_machine.html
[install gitlab runner]: https://docs.gitlab.com/runner/install/
[instance executor]: https://docs.gitlab.com/runner/executors/instance.html
[signals]: https://docs.gitlab.com/runner/commands/#signals
[store registration tokens or runner tokens in secrets]: https://docs.gitlab.com/runner/install/kubernetes.html#store-registration-tokens-or-runner-tokens-in-secrets
<!-- Others -->