brandon/oam

Fork 0

mirror of https://gitea.com/mcereda/oam.git synced 2026-02-09 05:44:23 +00:00

Files

Michele Cereda f466fca3e6 chore(ansible): improve notes for awx setup on eks

2024-08-07 22:28:53 +02:00

35 KiB

Raw Blame History

Ansible

TL;DR
Configuration
1. Performance tuning
Inventories
1. AWS
Templating
1. Tests
2. Loops
Validation
1. Assertions
Error handling
1. Using blocks
Output formatting
Roles
Create custom filter plugins
Execution environments
Troubleshooting
Further readings
1. Sources

TL;DR

# Install.
pip3 install --user 'ansible'
brew install 'ansible' 'sshpass'         # darwin
sudo pamac install 'ansible' 'sshpass'   # manjaro linux

# Generate example configuration files with entries disabled.
ansible-config init --disabled > 'ansible.cfg'
ansible-config init --disabled -t 'all' > 'ansible.cfg'

# Show the current configuration.
ansible-config dump

# Show hosts' ansible facts.
ansible -i 'path/to/hosts/file' -m 'setup' all
ansible -i 'host1,hostN,' -m 'setup' 'host1' -u 'remote-user'
ansible -i 'localhost,' -c 'local' -km 'setup' 'localhost'

# List hosts.
ansible-inventory -i 'inventory' --list
ansible-playbook -i 'inventory' 'playbook.yml' --list-hosts
ansible -i 'inventory' all --list-hosts

# Check the syntax of a playbook.
# This will *not* execute the plays inside it.
ansible-playbook 'path/to/playbook.yml' --syntax-check

# Execute playbooks.
ansible-playbook 'path/to/playbook.yml' -i 'hosts.list'
ansible-playbook … -i 'host1,host2,hostN,' -l 'hosts,list'
ansible-playbook … -i 'host1,host2,other,' -l 'hosts-pattern' --step

# Show what changes (with details) a play would apply to the local machine.
ansible-playbook 'path/to/playbook.yml' -i 'localhost,' -c 'local' -vvC

# Only execute tasks with specific tags.
ansible-playbook 'path/to/playbook.yml' --tags 'configuration,packages'

# Avoid executing tasks with specific tags.
ansible-playbook 'path/to/playbook.yml' --skip-tags 'system,user'

# Check what tasks will be executed.
ansible-playbook 'path/to/playbook.yml' --list-tasks
ansible-playbook … --list-tasks --tags 'configuration,packages'
ansible-playbook … --list-tasks --skip-tags 'system,user'

# Debug playbooks.
ANSIBLE_ENABLE_TASK_DEBUGGER=True ansible-playbook …

# List roles installed from Galaxy.
ansible-galaxy list

# Install roles from Galaxy.
ansible-galaxy install 'namespace.role'
ansible-galaxy install --roles-path 'path/to/ansible/roles' 'namespace.role'
ansible-galaxy install 'namespace.role,v1.0.0'
ansible-galaxy install 'git+https://github.com/namespace/role.git,commit-hash'
ansible-galaxy install -r 'requirements.yml'

# Create new roles.
ansible-galaxy init 'role_name'
ansible-galaxy role init --type 'container' --init-path 'path/to/role' 'name'

# Remove roles installed from Galaxy.
ansible-galaxy remove 'namespace.role'

Galaxy collections and roles worth a check:

ID	Type	Description
sivel.toiletwater	collection	Extra filters, mostly

UIs:

UI	Static inventories	Dynamic inventories
AWX	✓	✓
Rundeck	✓	?
Semaphore	✓	✗
Zuul	?	?

Configuration

Ansible can be configured using INI files named ansible.cfg, environment variables, command-line options, playbook keywords, and variables.

The ansible-config utility allows to see all the configuration settings available, their defaults, how to set them and where their current value comes from.

Ansible will process the following list and use the first file found; all the other files are ignored even if existing:

the ANSIBLE_CONFIG environment variable;
the ansible.cfg file in the current directory;
the ~/.ansible.cfg file in the user's home directory;
the /etc/ansible/ansible.cfg file.

Generate a fully commented-out example of the ansible.cfg file:

ansible-config init --disabled > 'ansible.cfg'

# Includes existing plugins.
ansible-config init --disabled -t all > 'ansible.cfg'

Performance tuning

Refer the following:

Suggestions:

Optimize fact gathering:

Disable fact gathering when not used.

Consider using smart fact gathering:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible/facts.json  ; /tmp/ansible to use the directory and have a file per host
fact_caching_timeout = 86400

Only gather subsets of facts:

- name: Play with selected facts
  gather_facts: true
  gather_subset:
    - '!all'
    - '!min'
    - system

Refer the setup module for more information, and the setup module source code for available keys.

Consider increasing the number of forks when dealing with lots of managed hosts:
```
[defaults]
forks = 25
```
Set independent tasks as async.

Optimize SSH connections:

Prefer key-based authentication if used:

[ssh_connection]
ssh_args = -o PreferredAuthentications=publickey

Use pipelining:
```
[ssh_connection]
pipelining = True
```

Consider using multiplexing:

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=3600s

Consider installing and using the Mitogen plugin on the controller:

curl -fsLO 'https://github.com/mitogen-hq/mitogen/releases/download/v0.3.7/mitogen-0.3.7.tar.gz'
tar -xaf 'mitogen-0.3.7.tar.gz'

[defaults]
strategy_plugins = mitogen-0.3.7/ansible_mitogen/plugins/strategy
strategy = mitogen_linear

Be advised that mitogen is not really supported by Ansible and has some issues with privilege escalation (1).

Improve the code:
- Bundle up package installations together.
- Beware of expensive calls.

Inventories

saturn ansible_python_interpreter=/usr/bin/python3.12 ansible_connection=local
jupiter.lan ansible_python_interpreter=/usr/bin/python3 ansible_port=4444

[accessed_remotely]
saturn
jupiter.lan
uranus.example.com ansible_port=5987

[swap_resistent]
jupiter.lan
saturn

[workstations]
saturn
; mars.lan ansible_port=4444

AWS

Refer Integrate with AWS SSM.

Templating

Ansible leverages Jinja2 templating, which can be used directly in tasks or through the template module.

All Jinja2's standard filters and tests can be used, with the addition of:

specialized filters for selecting and transforming data
tests for evaluating template expressions
lookup plugins for retrieving data from external sources for use in templating

All templating happens on the Ansible controller, before the task is sent and executed on the target machine.

Updated examples are available.

# Remove empty or false values from a list piping it to 'select()'.
# Returns ["string"].
- vars:
    list: ["", "string", 0, false]
  ansible.builtin.debug:
    var: list | select

# Remove only empty strings from a list 'reject()'ing them.
# Returns ["string", 0, false].
- vars:
    list: ["", "string", 0, false]
  ansible.builtin.debug:
    var: list | reject('match', '^$')

# Merge two lists.
# Returns ["a", "b", "c", "d"].
- vars:
    list1: ["a", "b"]
    list2: ["c", "d"]
  ansible.builtin.debug:
    var: list1 + list2

# Dedupe elements in a list.
# Returns ["a", "b"].
- vars:
    list: ["a", "b", "b", "a"]
  ansible.builtin.debug:
    var: list | unique

# Sort a list by version number (not lexicographically).
# Returns ['2.7.0', '2.8.0', '2.9.0', '2.10.0' '2.11.0'].
- vars:
    list: ['2.8.0', '2.11.0', '2.7.0', '2.10.0', '2.9.0']
  ansible.builtin.debug:
    var: list | community.general.version_sort

# Generate a random password.
# Returns a random string following the specifications.
- vars:
    password: "{{ lookup('password', '/dev/null length=32 chars=ascii_letters,digits,punctuation') }}"
  ansible.builtin.debug:
    var: password

# Hash a password.
# Returns a hash of the requested type.
- vars:
    password: abcd
    salt: "{{ lookup('community.general.random_string', special=false) }}"
  ansible.builtin.debug:
    var: password | password_hash('sha512', salt)

# Get a variable's type.
- ansible.builtin.debug:
    var: "'string' | type_debug"

Tests

Return a boolean result.

# Compare semver version numbers.
- ansible.builtin.debug:
    var: "'2.0.0-rc.1+build.123' is version('2.1.0-rc.2+build.423', 'ge', version_type='semver')"

# Find specific values in JSON objects.
- ansible.builtin.command: ssm-cli get-diagnostics --output 'json'
  become: true
  register: diagnostics
  failed_when: diagnostics.stdout | to_json | community.general.json_query('DiagnosticsOutput[*].Status=="Failed"')

Loops

# Get the values of some special variables.
# See the 'Further readings' section for the full list.
- ansible.builtin.debug:
    var: "{{ item }}"
  with_items: ["ansible_local", "playbook_dir", "role_path"]

# Fail when any of the given variables is an empty string.
# Returns the ones which are empty.
- when: lookup('vars', item) == ''
  ansible.builtin.fail:
    msg: "The {{ item }} variable is an empty string"
  loop:
    - variable1
    - variableN

# Iterate through nested loops.
- vars:
    middles:
      - 'middle1'
      - 'middle2'
  ansible.builtin.debug:
    msg: "{{ item[0] }}, {{ item[1] }}, {{ item[2] }}"
  with_nested:
    - ['outer1', 'outer2']
    - "{{ middles }}"
    - ['inner1', 'inner2']

Validation

Assertions

- ansible.builtin.assert:
    that:
      - install_method in supported_install_methods
      - external_url is ansible.builtin.url
    fail_msg: What to say if any of the above conditions fail
    success_msg: What to say if all of the above conditions succeed

Error handling

Using blocks

Refer Blocks.

- name: Error handling in blocks
  block:
    - name: This executes normally
      ansible.builtin.debug:
        msg: I execute normally
    - name: This errors out
      ansible.builtin.command: "/bin/false"
    - name: This never executes
      ansible.builtin.debug:
        msg: I never execute due to the above task failing
  rescue:
    - name: This executes if any errors arose in the block
      ansible.builtin.debug:
        msg: I caught an error and can do stuff here to fix it
  always:
    - name: This always executes
      ansible.builtin.debug:
        msg: I always execute

Output formatting

Introduced in Ansible 2.5

Change Ansible's output setting the stdout callback to json or yaml:

ANSIBLE_STDOUT_CALLBACK='yaml'

# ansible.cfg
[defaults]
stdout_callback = json

yaml will set tasks output only to be in the defined format:

$ ANSIBLE_STDOUT_CALLBACK='yaml' ansible-playbook --inventory='localhost,' 'localhost.configure.yml' -vv --check
PLAY [Configure localhost] *******************************************************************

TASK [Upgrade system packages] ***************************************************************
task path: /home/user/localhost.configure.yml:7
ok: [localhost] => changed=false
  cmd:
  - /usr/bin/zypper
  - --quiet
  - --non-interactive
  …
  update_cache: false

The json output format will be a single, long JSON file:

$ ANSIBLE_STDOUT_CALLBACK='json' ansible-playbook --inventory='localhost,' 'localhost.configure.yml' -vv --check
{
  "custom_stats": {},
  "global_custom_stats": {},
  "plays": [
    {
      "play": {
        …
        "name": "Configure localhost"
      },
      "tasks": [
        {
          "hosts": {
            "localhost": {
              "action": "community.general.zypper",
              "changed": false,
              …
              "update_cache": false
            }
          }
        }
      ]
    }
  ]
}

Roles

Get roles

Roles can be either created:

ansible-galaxy init 'role-name'

or installed from Galaxy:

---
# requirements.yml
collections:
  - community.docker

ansible-galaxy install 'mcereda.boinc_client'
ansible-galaxy install --roles-path 'path/to/roles' 'namespace.role'
ansible-galaxy install 'namespace.role,v1.0.0'
ansible-galaxy install 'git+https://github.com/namespace/role.git,commit-hash'
ansible-galaxy install -r 'requirements.yml'

Assign roles

In playbooks:

---
- hosts: all
  roles:
    - web_server
    - geerlingguy.postgresql
    - role: /custom/path/to/role
      vars:
        var1: value1
      tags: example
      message: some message

Role dependencies

Set them up in role/meta/main.yml:

---
dependencies:
  - role: common
    vars:
      some_parameter: 3
  - role: postgres
    vars:
      dbname: blarg
      other_parameter: 12

and/or in role/meta/requirements.yml:

---
collections:
  - community.dns

Create custom filter plugins

See Creating your own Ansible filter plugins.

Execution environments

Container images that can be used as Ansible control nodes.

Ansible Builder aids in the creation of Ansible Execution Environments.
Refer Introduction to Ansible Builder for how to build one.

Builders' build command defaults to using:

execution-environment.yml or execution-environment.yaml as the definition file.
$PWD/context as the directory to use for the build context.

execution-environment.yml example

Refer Execution environment definition.

---
version: 3

build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre'

dependencies:
  ansible_core:
    package_pip: ansible-core==2.14.4
  ansible_runner:
    package_pip: ansible-runner
  galaxy: requirements.yml
  python:
    - six
    - psutil
  system: bindep.txt
  exclude:
    python:
      - docker
    system:
      - python3-Cython

images:
  base_image:
    name: docker.io/redhat/ubi9:latest
    # Other available base images:
    #   - quay.io/rockylinux/rockylinux:9
    #   - quay.io/centos/centos:stream9
    #   - registry.fedoraproject.org/fedora:38
    #   - registry.redhat.io/ansible-automation-platform-23/ee-minimal-rhel8:latest
    #     (needs an account)

# Custom package manager path for the RHEL based images
# options:
#   package_manager_path: /usr/bin/microdnf

additional_build_files:
  - src: files/ansible.cfg
    dest: configs

additional_build_steps:
  prepend_base:
    - RUN echo This is a prepend base command!
    # Enable Non-default stream before packages provided by it can be installed. (optional)
    # - RUN $PKGMGR module enable postgresql:15 -y
    # - RUN $PKGMGR install -y postgresql
  prepend_galaxy:
    - COPY _build/configs/ansible.cfg /etc/ansible/ansible.cfg

  prepend_final: |
    RUN whoami
    RUN cat /etc/os-release
  append_final:
    - RUN echo This is a post-install command!
    - RUN ls -la /etc

requirements.yml example

---
collections:
  - redhat.openshift

Commands example

pip install 'ansible-builder'
ansible-builder build --container-runtime 'docker' -t 'example-ee:latest' -f 'definition.yml'
ansible-runner -p 'test_play.yml' --container-image 'example-ee:latest'
ansible-navigator run 'test_play.yml' -i 'localhost,' --execution-environment-image 'example-ee:latest' \
  --mode 'stdout' --pull-policy 'missing' --container-options='--user=0'

Troubleshooting

Print all known variables

Print the special variable vars as a task:

- name: Debug all variables
  ansible.builtin.debug: var=vars

Force notified handlers to run at a specific point

Use the meta plugin with the flush_handlers option:

- name: Force all notified handlers to run at this point, not waiting for normal sync points
  ansible.builtin.meta: flush_handlers

Time tasks execution

Add profile_tasks the list of enable callbacks.

Choose one or more options:

Add it to callbacks_enabled in the [defaults] section of Ansible's configuration file:

[defaults]
callbacks_enabled = profile_tasks  # or ansible.posix.profile_tasks

Set the ANSIBLE_CALLBACKS_ENABLED environment variable:
```
export ANSIBLE_CALLBACKS_ENABLED='profile_tasks'
```

Run specific tasks even in check mode

Add the check_mode: false pair to the task:

- name: this task will make changes to the system even in check mode
  check_mode: false
  ansible.builtin.command: /something/to/run --even-in-check-mode

Dry-run only specific tasks

Add the check_mode: true pair to the task:

- name: This task will always run under check mode and not change the system
  check_mode: true
  ansible.builtin.lineinfile:
    line: "important file"
    dest: /path/to/file.conf
    state: present

Set up recursive permissions on a directory so that directories are set to 755 and files to 644

Use the special X mode setting in the file plugin:

- name: Fix files and directories' permissions
  ansible.builtin.file:
    dest: /path/to/some/dir
    mode: u=rwX,g=rX,o=rX
    recurse: yes

Only run a task when another has a specific result

When a task executes, it also stores the two special values changed and failed in its results.
One can use those as conditions to execute the next ones:

- name: Trigger task
  ansible.builtin.command: any
  register: trigger_task
  ignore_errors: true

- name: Run only on change
  when: trigger_task.changed
  ansible.builtin.debug: msg="The trigger task changed"

- name: Run only on failure
  when: trigger_task.failed
  ansible.builtin.debug: msg="The trigger task failed"

Alternatively, you can use special checks built for this:

- name: Run only on success
  when: trigger_task is succeeded
  ansible.builtin.debug: msg="The trigger task succeeded"

- name: Run only on change
  when: trigger_task is changed
  ansible.builtin.debug: msg="The trigger task changed"

- name: Run only on failure
  when: trigger_task is failed
  ansible.builtin.debug: msg="The trigger task failed"

- name: Run only on skip
  when: trigger_task is skipped
  ansible.builtin.debug: msg="The trigger task skipped"

Define when a task changed or failed

This lets you avoid using ignore_errors.

Use the changed_when and failed_when attributes to define your own conditions:

- name: Task with custom results
  ansible.builtin.command: any
  register: result
  changed_when:
    - result.rc == 2
    - result.stderr | regex_search('things changed')
  failed_when:
    - result.rc != 0
    - not (result.stderr | regex_search('all good'))

Set environment variables for a play, role or task

Environment variables can be set at a play, block, or task level using the environment keyword:

- name: Use environment variables for a task
  environment:
    HTTP_PROXY: http://example.proxy
  ansible.builtin.command: curl ifconfig.io

The environment keyword does not affect Ansible itself or its configuration settings, the environment for other users, or the execution of other plugins like lookups and filters.
Variables set with environment do not automatically become Ansible facts, even when set at the play level.

Set variables to the value of environment variables

Use the lookup() plugin with the env option:

- name: Use a local environment variable
  ansible.builtin.debug: msg="HOME={{ lookup('env', 'HOME') }}"

Check if a list contains an item and fail otherwise

- name: Check if a list contains an item and fail otherwise
  when: item not in list
  ansible.builtin.fail: msg="item not in list"

Define different values for `true`/`false`/`null`

Create a test and define two values: the first will be returned when the test returns true, the second will be returned when the test returns false (Ansible 1.9+):

{{ (ansible_pkg_mgr == 'zypper') | ternary('gnu_parallel', 'parallel') }}

Since Ansible 2.8 you can define a third value to be returned when the test returns null:

{{ autoscaling_enabled | ternary(true, false, omit) }}

Force a task or play to use a specific Python interpreter

Just set it in the Play's or Task's variables:

vars:
  ansible_python_interpreter: /usr/local/bin/python3.9

Provide a template file content inline

Use the ansible.builtin.copy instead of ansible.builtin.template:

- name: Configure knockd
  ansible.builtin.copy:
    dest: /etc/knockd.conf
    content: |
      [options]
        UseSyslog

Python breaks in OS X

Root Cause:

Mac OS High Sierra and later versions have restricted multithreading for improved security.
Apple has defined some rules on what is allowed and not is not after forking processes, and have also added async-signal-safety to a limited number of APIs.

Solution:

Disable fork initialization safety features as shown in Why Ansible and Python fork break on macOS High Sierra+ and how to solve:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Load files' content into variables

For local files, use lookups:

user_data: "{{ lookup('file', 'path/to/file') }}"

For remote files, use the slurp module:

- ansible.builtin.slurp:
    src: "{{ user_data_file }}"
  register: slurped_user_data
- ansible.builtin.set_fact:
    user_data: "{{ slurped_user_data.content | ansible.builtin.b64decode }}"

The contents are presented as base64 string. The decode is needed.

Only run a task when explicitly requested

Leverage the never tag to never execute the task unless requested by using the --tags 'never' option:

- tags: never
  ansible.builtin.debug:
    msg: …

Conversely, one can achieve the opposite by using the always tag and the --skip 'always' option:

- tags: always
  ansible.builtin.command: …

Using AWS' SSM with Ansible fails with error Failed to create temporary directory

Message example:

fatal: [i-4ccab452bb7743336]: UNREACHABLE! => {
  "changed": false,
  "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo \u001b]0;@ip-192-168-42-42:/usr/bin\u0007/home/centos/.ansible/tmp `\"&& mkdir \"` echo \u001b]0;@ip-192-168-42-42:/usr/bin\u0007/home/centos/.ansible/tmp/ansible-tmp-1708603630.2433128-49665-225488680421418 `\" && echo ansible-tmp-1708603630.2433128-49665-225488680421418=\"` echo \u001b]0;@ip-192-168-42-42:/usr/bin\u0007/home/centos/.ansible/tmp/ansible-tmp-1708603630.2433128-49665-225488680421418 `\" ), exited with result 1, stdout output: \u001b]0;@ip-192-168-42-42:/usr/bin\u0007bash: @ip-192-168-42-42:/usr/bin/home/centos/.ansible/tmp: No such file or directory\r\r\nmkdir: cannot create directory '0': Permission denied\r\r",
  "unreachable": true
}

Root cause:

By default, SSM starts sessions in the /usr/bin directory.