Ansible – Configuration management and infrastructure provisioning without containers

Even though I’m full steam ahead on the Docker train (it should really be a ship, to keep in line with the container metaphors), it was time to look at what everyone else is using for provisioning and configuration management.

Here’s my overview of getting Ansible up and running using my laptop as the control node and a few EC2 nodes, as well as a brief description of the components. And judging by the quality of the documentation, which is quite meaningful in my book, this is one excellent piece of software.

Local installation on macOS using Homebrew

Introduction — Ansible Documentation

  • Install locally via brew install ansible
  • Inventory stored in /etc/ansible/hosts
  • YAML format seems cooler and easier to manage
  • Hosts / group variables can be installed in subdirectories, e.g. /etc/ansible/host_vars

Sample Commands

Getting Started — Ansible Documentation

  • ping all hosts: ansible all -m ping
  • print environment variables for all hosts, specifying user and private key in script command: ansible all -a "printenv" -u ec2-user --private ~/.ssh/my-private-key.pem

Inventory

Inventory — Ansible Documentation

The Ansible “inventory” describes the available hosts. Default location for inventory file is /etc/ansible/hosts.

all:
  children:
    webservers:
      hosts:
        54.88.95.185:
          ansible_user: ec2-user
          ansible_ssh_private_key_file: ~/.ssh/my-private-key.pem
          ansible_become: true
        34.236.150.86:
          ansible_user: ec2-user
          ansible_ssh_private_key_file: ~/.ssh/my-private-key.pem
          ansible_become: true
      vars:
        ntp_server: ntp.atlanta.example.com
        proxy: proxy.atlanta.example.com
    dbservers:
      hosts:
        34.236.150.86:
          ansible_user: ec2-user
          ansible_ssh_private_key_file: ~/.ssh/my-private-key.pem
          ansible_become: true

Patterns

Patterns represent the targets for the script execution, i.e. which hosts to communicate with. The generic format is:

ansible <pattern_goes_here> -m <module_name> -a <arguments>

An example execution would be:

ansible webservers -m service -a "name=httpd state=restarted"

Pattern Options

Patterns — Ansible Documentation
The pattern can specify:

  • Individual Hosts
  • All hosts
  • Groups (=sets of hosts)
  • Groups of hosts that exclude other groups, e.g. hosts that are in group A but not in Group B
  • The intersection of two groups, i.e. hosts that have to be in two groups
  • Pretty much anything else that’s possible in Python

Ad-hoc commands

Ad-Hoc Commands — Ansible Documentation

Ad-hoc commands are commands that are not part of a playbook. They are executed using the -a flag in the ansible script, usually in combination with a specific Ansible module indicated by the -m flag.

Here’s an example to print out the PATH in a single host:

ansible dbserver -m shell -a "echo $PATH"

Copy files to servers and set correct permissions

One excellent use case listed in the tutorial for ad-hoc commands is copying a test file to all servers and set the correct permissions. This assumes that the user on the host has the permission to adjust file permissions.

File Transfer

This example creates a sample text file and sends it to all hosts using the copy module:

$ touch test.txt                                                                                    
$ echo 'This is just a test file' > test.txt
$ ansible all -m copy -a "src=/Users/danny/Desktop/test.txt dest=/tmp"

Change permissions for sample text file

This example uses the file module to change the permissions of a file on specific hosts:

$ ansible webservers -m file -a "dest=/tmp/test.txt mode=600"

Managing packages on hosts

Assuming your server is running a CentOS-based distro, you can use the yum module to install, remove, check, or update modules: Managing Packages

Managing services on hosts

You can start/stop or restart services easily on target hosts, e.g. make sure the cron daemon is running:

ansible dbservers -m service -a "name=crond state=started"

Deployments using Git

I have to admit that this is blowing my mind a bit, especially after trying a lot of different ways to deploy code. My last pre-Docker deployment method was AWS CodeDeploy, which requires a CodeDeploy agent to be installed on all targets. Granted, the Ansible git module requires Git to be installed, and most likely configured to pull files securely, but it seems a lot more straightforward to pull code and adjust the permissions accordingly for security.

Playbooks

Playbooks are described as the “instruction manuals” in the Ansible toolbox. Comparing them to a containerized environment on AWS, I would describe them as coming closest to a task definition, which contains the Docker images required for different services.

Playbooks are written in YAML and contains one more or “plays” in a list.

Plays

Plays is Ansible’s sports analogy, where each play maps a host or group of hosts to roles, which consists of tasks. Here’s an example with just one play:, which contains three tasks: Update Apache to the latest version, write the config file, and make sure apache is running.

---
- hosts: webservers
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root
  tasks:
    - name: ensure apache is at the latest version
      yum: name=httpd state=latest
    - name: write the apache config file
      template: src=/srv/httpd.j2 dest=/etc/httpd.conf
      notify:
      - restart apache
    - name: ensure apache is running (and enable it at boot)
      service: name=httpd state=started enabled=yes
  handlers:
    - name: restart apache
      service: name=httpd state=restarted

Task items can also be broken up into dictionaries for better readability:

---
- hosts: webservers
  vars:
    http_port: 80
    max_clients: 200
  remote_user: root
  tasks:
    - name: ensure apache is at the latest version
      yum:
        name: httpd
        state: latest
    - name: write the apache config file
      template:
        src: /srv/httpd.j2
        dest: /etc/httpd.conf
      notify:
      - restart apache
    - name: ensure apache is running
      service:
        name: httpd
        state: started
  handlers:
    - name: restart apache
      service:
        name: httpd
        state: restarted

Task Lists

Each play contains a list of tasks. Tasks are executed in order, one at a time, against all machines matched by the host pattern, before moving on to the next task. It is important to understand that, within a play, all hosts are going to get the same task directives. It is the purpose of a play to map a selection of hosts to tasks.

The goal of each task is to execute a module, with very specific arguments. Variables, as mentioned above, can be used in arguments to modules.

When running the playbook, which runs top to bottom, hosts with failed tasks are taken out of the rotation for the entire playbook. If things fail, simply correct the playbook file and rerun.

Task Composition

  • Every task should have a human-readable name, although not required
  • Most tasks take key=value arguments, or a yaml dictionary:
tasks:
  - name: make sure apache is running
    service: name=httpd state=started
tasks:
  - name: make sure apache is running
    service:
      name: httpd
      state: started
  • The command and shell module are the only tasks which take a list of arguments, and care about return codes, unless the ignore_errors flag is enabled.

Handlers: Basic event listener system for running operations on change

  • Task can contain notify sections, which define the handlers
  • Handlers are essentially lists of tasks that get triggered by the notifiers.
  • Handlers are referenced by a globally unique name
  • Handlers are notified by notifiers
  • Handlers will only run once per play, even if contained by multiple tasks

Here’s an example of a tasks that notifies two handlers.

- name: template configuration file
  template: src=template.j2 dest=/etc/foo.conf
  notify:
    - restart memcached
    - restart apache
handlers:
  - name: restart memcached
    service: name=memcached state=restarted
  - name: restart apache
    service: name=apache state=restarted

Listening handlers: Easy way to trigger multiple handlers

Instead of having the task specify all the handlers required to successfully restart the web server, handlers can listen to generic topics. This allows handlers to be combined as groups.

The following example has two handlers that listens to the restart web services topic:

handlers:
  - name: restart memcached
    service: name=memcached state=restarted
    listen: "restart web services"
  - name: restart apache
    service: name=apache state=restarted
    listen: "restart web services"

tasks:
  - name: restart everything
    command: echo "this task will restart the web services"
    notify: "restart web services"

Sample playbook: WordPress on Amazon AMI

There are a number of good starter playbooks on Github.. I’m giving the wordpress-nginx playbook a try on an Amazon AMI and found out that it needed some tweaking. The base AMI is Amazon Linux AMI 2017.09.1.20171103 x86_64 HVM.

Minor Tweaks

  • The Amazon AMI has renamed its MySQL-python package to MySQL-python27
  • The default_server in the NGINX configuration conflicts with the one setup in the playbook, so the default one needs to be removed

Playbook Download

Here’s the sample code that worked in combination with this specific version of the Amazon Linux AMI 2017.09.1.20171103 x86_64 HVM.

Ansible playbook for WordPress / NGINX