Ansible

Ansible provides a way of setting up servers in a controlled and repeatable way, utilising either third party or custom "playbooks"

Ansible does not care exactly how the server is provisioned. It could be a local LXC container, a local Virtualbox or libvirt VM, a remote cloud server or a remote full baremetal server. It does not require any software to be installed on the remote machine other than Python and SSH, which we can expect to be there as standard on any Linux host.

Edmonds Commerce have chosen Ansible as the tool of choice for managing server configuration and maintainence.

Ansible in a Nutshell

Ansible is configuration management software that we install locally to manage remote servers.

The machine on which we install Ansible is called the "Control Machine". The Control Machine is your system that does the setting up and managing of remote servers.

There is nothing that needs to be installed on the remote server, but your "control machine" does need setting up.

Once Ansible is installed then we need to make an Ansible project which includes a few files, generally YAML. These contain information on our inventory (the servers we are managing) and things we want to configure on there. The real work in Ansible is done with playbooks.

Once the files have been created, a playbook can be run on one or more servers to perform the desired tasks.

How Ansible Works

Best Practices

There are established best practices. Well worth reading through properly before you start your project.

Also, there is a github repo of ansible-examples which gives you something concrete to have a look at in terms of some Ansible projects.

Learning

Ansible has it's own comprehensive documentation

In our ebook library we have Ansible Beginner to Pro

Everything in Ansible is written in YAML I know :/, here is an Ansible page about YAML Syntax

Strongly suggest you read the full FAQ as well, for things like:

Quick Starts

Here are a few quick start articles that are worth having a look at

Official Quick Start Video

Here is the official quick start video as found here

Tip

you can put any old random email address in and it will start playing immediately for example: no@mail.com

Editing Ansible Files

Ansible files are YAML and so at a minimum you need something that will syntax highlight YAML files.

For more advanced options:

Ansible Vim

First you need to install vim

1
dnf -y install vim

Then you need to install vim-plug

1
2
curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

Then we will use vim-plug to install ansible-vim

To do this we need to do the following:

First - open your ~/.vimrc file in vim

1
vim ~/.vimrc

Then paste in the following at the bottom:

1
2
3
4
5
6
" =========== Vim Plug Plugins =============================
call plug#begin('~/.vim/vim-plug-plugins')

Plug 'pearofducks/ansible-vim'

call plug#end()

Then, in vim, type :w and hit return to write update the vimrc file

Then to install plugins, type :PlugInstall and that will open a vim pane with the installation. Once it is completed you can :q to close the pane.

Ansible PHPStorm

There is a plugin called YAML/Ansible support

However, it conflicts with the standard YAML plugin which is required by all kinds of other plugins like Symfony, so it is not usable.

The standard YAML plugin will provide basic syntax highlighting but nothing more

If you want to use the special plugin, it would be possible to create a separate PHPStorm profile and then use that. Perhaps using the EAP version for Ansible work?

Basic Setup

In this section we are going to provide opinionated instructions on how to set up Ansible for Edmonds Commerce workloads.

Installing Ansible on Control Machine

Assuming you are using Fedora on the desktop and you want to use this as your control machine, you can just run:

1
dnf -y install ansible

You can now use your local machine to manage your hosts and run playbooks.

Local Development and Testing

Warning

Don't run ansible commands on your host machine

The specifics of how to develop and test your Ansible configuration for a project depends on the production infrastructure

Single Server

If the project will host in production on a single server, then we should replicate that in development and staging by using a single container. As this is such a simple setup the it might be that Ansible is overkill or is already covered with our standard container creation systems.

Multiple Servers or Containers

If the project will host in production on multiple servers or containers then we can replicate that by having a local VM that in turn houses containers that emulate the production infrastructure.

This allows us to drop and recreate the entire infrastructure easily and cleanly.

The easiest way to test building containers inside a VM is to make the VM a control machine and then run the plays inside the machine to build and configure the containers.

Creating a local VM to Host LXC Containers

Because we use Fedora and the desktop and LXC containers here at Edmonds Commerce, we are going to use a libvirt based VM which will then host our test LXC containers.

To build our local VM, we are going to use Vagrant and libvirt - click here to read how to get those set up

Then in a folder of your choosing, we need to create a Vagrantfile such as:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#Set the machine name
$machineName="lxchost"

#Provide BASH commands to do basic provisioning
$script = <<BASH
echo "Running BASH provisioning..."
yum -y install ansible
BASH

Vagrant.require_version ">= 1.7.0"
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'
Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.hostname=$machineName
  config.vm.network "private_network", type: "dhcp"
  config.vm.define $machineName do |t|
    t.vm.host_name=$machineName
    config.vm.provider :libvirt do |v|
      v.memory = 8192
      v.cpus = 6
    end
  end
  config.vm.provision "shell",
    inline: $script
end

Then to build the machine, in the same directory as the Vagrant file simply run:

1
vagrant up

And you should see output along the lines of:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
joseph@joseph-thinkpad vagrant up
Bringing machine 'lxchost' up with 'libvirt' provider...
==> testbed: Box 'centos/7' could not be found. Attempting to find and install...
    testbed: Box Provider: libvirt
    testbed: Box Version: >= 0
==> testbed: Loading metadata for box 'centos/7'
    testbed: URL: https://atlas.hashicorp.com/centos/7
==> testbed: Creating shared folders metadata...
==> testbed: Starting domain.
==> testbed: Waiting for domain to get an IP address...
==> testbed: Waiting for SSH to become available...
==> testbed: Setting hostname...
==> testbed: Configuring and enabling network interfaces...
==> testbed: Rsyncing folder: /opt/Projects/ansible-scratch/ => /vagrant

Then you can vagrant ssh and sudo bash to gain root in the machine

Ansible and LXC

I have read lots of different articles and ideas about using Ansible with LXC

Here are some links:

Creating and Managing Containers Inside the Vagrant Box Using Ansible

To go to the next level, we are going to start a full blown ansible project and then build this inside the Vagrant machine.

This will build our LXC containers and then also configure them. This should be as close as possible to performing a build of LXC containers on a real live server.

So to do this, lets create a sub folder called ansible-project and in there we will initialise a basic Ansible project

1
2
3
4
5
6
#from the directory containing the Vagrantfile
mkdir ansible-project
mkdir ansible-project/inventory-local
mkdir ansible-project/roles
mkdir ansible-project/host_vars
mkdir ansible-project/group_vars

Inventory

The first thing for us to do is to create an inventory file, such as

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
cd ansible-project
# create an inventory file, containers named `cnt-{group}-id`
cat <<EOF > inventory-local/inventory.local.ini
[web]
cnt-web-1

[db]
cnt-db-1

[lxc]
cnt-web-1
cnt-db-1
EOF
This simple inventory lists 2 containers, one web and one database.

We have three groups: web, db, lxc

The lxc group contains all the containers we are going to build inside the Vagrant machine.

Roles

The next thing we need to add in there are the roles that we need to be in the plays. Roles are basically grouped tasks that achieve a specific set of configuration.

LXC Containers Creation

The first role we need is one to create the LXC containers. There are a few roles on Ansible Galaxy however I didn't find one that did exactly what I want to do, so I created one instead.

edmondscommerce.lxc

This role will handle installing LXC 2 on CentOS and then will create containers listed in the lxc group on the inventory.

To install this role, we need to run:

1
2
cd ansible-project
ansible-galaxy install --roles-path ./roles edmondscommerce.lxc

And we should see output along the lines of:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
joseph@joseph-thinkpad ansible-galaxy install --roles-path ./roles edmondscommerce.lxc
- downloading role 'lxc', owned by edmondscommerce
- downloading role from https://github.com/edmondscommerce/ansible-role-lxc/archive/master.tar.gz
- extracting edmondscommerce.lxc to /opt/Projects/ansible-scratch/ansible-project/roles/edmondscommerce.lxc
- edmondscommerce.lxc (master) was installed successfully
- adding dependency: edmondscommerce.copr-repository
- downloading role 'copr-repository', owned by edmondscommerce
- downloading role from https://github.com/edmondscommerce/role-copr-repository/archive/master.tar.gz
- extracting edmondscommerce.copr-repository to /opt/Projects/ansible-scratch/ansible-project/roles/edmondscommerce.copr-repository
- edmondscommerce.copr-repository (master) was installed successfully

Note

The LXC role has a dependency on another role - edmondscommerce.copr-repository See how ansible-galaxy has resolved this and also installed the dependency for us.

Main Playbook

So now we have our inventory and roles sorted out, we need a playbook to run

In the root of ansible-project directory, make a file called playbook-main.yml with these contents:

1
2
3
4
5
- name: install lxc and create lxc containers
  hosts: localhost
  become: true
  roles:
      - edmondscommerce.lxc
Sync this through to the VM
1
vagrant rsync
Enter the machine, become root and create the containers
1
2
3
4
5
6
7
vagrant ssh

sudo bash

cd /vagrant/ansible-project

ansible-playbook -i inventory.local playbook-main.yml

Tip

Don't edit the ansible-project files inside the VM Instead, edit them on your host machine, but have vagrant rsync-auto running in a separate terminal This will keep all changes synced