Ansible
Ansible provides a way of setting up servers in a controlled and repeatable way, utilising either third party or custom "playbooks"
Ansible does not care exactly how the server is provisioned. It could be a local LXC container, a local Virtualbox or libvirt VM, a remote cloud server or a remote full baremetal server. It does not require any software to be installed on the remote machine other than Python and SSH, which we can expect to be there as standard on any Linux host.
Edmonds Commerce have chosen Ansible as the tool of choice for managing server configuration and maintainence.
Ansible in a Nutshell¶
Ansible is configuration management software that we install locally to manage remote servers.
The machine on which we install Ansible is called the "Control Machine". The Control Machine is your system that does the setting up and managing of remote servers.
There is nothing that needs to be installed on the remote server, but your "control machine" does need setting up.
Once Ansible is installed then we need to make an Ansible project which includes a few files, generally YAML. These contain information on our inventory (the servers we are managing) and things we want to configure on there. The real work in Ansible is done with playbooks.
Once the files have been created, a playbook can be run on one or more servers to perform the desired tasks.
Best Practices¶
There are established best practices. Well worth reading through properly before you start your project.
Also, there is a github repo of ansible-examples which gives you something concrete to have a look at in terms of some Ansible projects.
Learning¶
Ansible has it's own comprehensive documentation
In our ebook library we have Ansible Beginner to Pro
Everything in Ansible is written in YAML I know :/
, here is an Ansible page about YAML Syntax
Strongly suggest you read the full FAQ as well, for things like:
- How do I see a list of all of the ansible variables
- How do I configure a jump host to access servers that I have no direct access to?
Quick Starts¶
Here are a few quick start articles that are worth having a look at
- https://wiredcraft.com/blog/getting-started-with-ansible-in-5-minutes/
- https://netapp.io/2017/01/30/getting-started-ansible-playbooks-can-addictive-heres-can-get-hooked/
- https://www.digitalocean.com/community/tutorials/configuration-management-101-writing-ansible-playbooks
Official Quick Start Video¶
Here is the official quick start video as found here
Tip
you can put any old random email address in and it will start playing immediately
for example: no@mail.com
Editing Ansible Files¶
Ansible files are YAML and so at a minimum you need something that will syntax highlight YAML files.
For more advanced options:
Ansible Vim¶
First you need to install vim
dnf -y install vim
Then you need to install vim-plug
curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim
Then we will use vim-plug to install ansible-vim
To do this we need to do the following:
First - open your ~/.vimrc
file in vim
vim ~/.vimrc
Then paste in the following at the bottom:
" =========== Vim Plug Plugins =============================
call plug#begin('~/.vim/vim-plug-plugins')
Plug 'pearofducks/ansible-vim'
call plug#end()
Then, in vim, type :w
and hit return to write update the vimrc file
Then to install plugins, type :PlugInstall
and that will open a vim pane with the installation. Once it is completed you can :q
to close the pane.
Ansible PHPStorm¶
There is a plugin called YAML/Ansible support
However, it conflicts with the standard YAML plugin which is required by all kinds of other plugins like Symfony, so it is not usable.
The standard YAML plugin will provide basic syntax highlighting but nothing more
If you want to use the special plugin, it would be possible to create a separate PHPStorm profile and then use that. Perhaps using the EAP version for Ansible work?
Note
You can install PyCharm and then install the Ansible plugin
Basic Setup¶
In this section we are going to provide opinionated instructions on how to set up Ansible for Edmonds Commerce workloads.
Installing Ansible on Control Machine¶
Assuming you are using Fedora on the desktop and you want to use this as your control machine, you can just run:
sudo dnf install python3 python3-pip -y
pip3 install --user ansible
ansible -v
Warning
Do not install ansible with anything but pip3
You can now use your local machine to manage your hosts and run playbooks.
Desktop Level Tweaks¶
Here are some other tweaks:
Alias for Ansible Playbook¶
alias ansible-playbook='ANSIBLE_KEEP_REMOTE_FILES=1 ansible-playbook -vvv '
Local Development and Testing¶
Warning
Don't run ansible commands on your host machine
The specifics of how to develop and test your Ansible configuration for a project depends on the production infrastructure
Single Server¶
If the project will host in production on a single server, then we should replicate that in development and staging by using a single container. As this is such a simple setup the it might be that Ansible is overkill or is already covered with our standard container creation systems.
Multiple Servers or Containers¶
If the project will host in production on multiple servers or containers then we can replicate that by having a local VM that in turn houses containers that emulate the production infrastructure.
This allows us to drop and recreate the entire infrastructure easily and cleanly.
The easiest way to test building containers inside a VM is to make the VM a control machine and then run the plays inside the machine to build and configure the containers.
Creating a local VM to Host LXC Containers¶
Because we use Fedora and the desktop and LXC containers here at Edmonds Commerce, we are going to use a libvirt based VM which will then host our test LXC containers.
To build our local VM, we are going to use Vagrant and libvirt - click here to read how to get those set up
Then in a folder of your choosing, we need to create a Vagrantfile
such as:
#Set the machine name
$machineName="lxchost"
#Provide BASH commands to do basic provisioning
$script = <<BASH
echo "Running BASH provisioning..."
yum -y install ansible
BASH
Vagrant.require_version ">= 1.7.0"
ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'
Vagrant.configure("2") do |config|
config.vm.box = "centos/7"
config.vm.hostname=$machineName
config.vm.network "private_network", type: "dhcp"
config.vm.define $machineName do |t|
t.vm.host_name=$machineName
config.vm.provider :libvirt do |v|
v.memory = 8192
v.cpus = 6
end
end
config.vm.provision "shell",
inline: $script
end
Then to build the machine, in the same directory as the Vagrant file simply run:
vagrant up
And you should see output along the lines of:
joseph@joseph-thinkpad vagrant up
Bringing machine 'lxchost' up with 'libvirt' provider...
==> testbed: Box 'centos/7' could not be found. Attempting to find and install...
testbed: Box Provider: libvirt
testbed: Box Version: >= 0
==> testbed: Loading metadata for box 'centos/7'
testbed: URL: https://atlas.hashicorp.com/centos/7
==> testbed: Creating shared folders metadata...
==> testbed: Starting domain.
==> testbed: Waiting for domain to get an IP address...
==> testbed: Waiting for SSH to become available...
==> testbed: Setting hostname...
==> testbed: Configuring and enabling network interfaces...
==> testbed: Rsyncing folder: /opt/Projects/ansible-scratch/ => /vagrant
Then you can vagrant ssh
and sudo bash
to gain root in the machine
Ansible and LXC¶
I have read lots of different articles and ideas about using Ansible with LXC
Here are some links:
- https://kerunix.com/Managing%20LXC%20hosts%20with%20ansible%20and%20a%20jump%20host.html
- https://github.com/gitinsky/ansible-role-lxc
- http://docs.ansible.com/ansible/latest/lxc_container_module.html
- https://docs.ansible.com/ansible/devel/plugins/connection/lxc.html
- https://andreas.scherbaum.la/blog/archives/910-Ansible-and-LXC-Containers.html
- https://www.hastexo.com/blogs/florian/2016/02/21/containers-just-because-everyone-else/
Creating and Managing Containers Inside the Vagrant Box Using Ansible¶
To go to the next level, we are going to start a full blown ansible project and then build this inside the Vagrant machine.
This will build our LXC containers and then also configure them. This should be as close as possible to performing a build of LXC containers on a real live server.
So to do this, lets create a sub folder called ansible-project
and in there we will initialise a basic Ansible project
#from the directory containing the Vagrantfile
mkdir ansible-project
mkdir ansible-project/inventory-local
mkdir ansible-project/roles
mkdir ansible-project/host_vars
mkdir ansible-project/group_vars
Inventory¶
The first thing for us to do is to create an inventory file, such as
cd ansible-project
# create an inventory file, containers named `cnt-{group}-id`
cat <<EOF > inventory-local/inventory.local.ini
[web]
cnt-web-1
[db]
cnt-db-1
[lxc]
cnt-web-1
cnt-db-1
EOF
We have three groups: web
, db
, lxc
The lxc group contains all the containers we are going to build inside the Vagrant machine.
Roles¶
The next thing we need to add in there are the roles that we need to be in the plays. Roles are basically grouped tasks that achieve a specific set of configuration.
LXC Containers Creation¶
The first role we need is one to create the LXC containers. There are a few roles on Ansible Galaxy however I didn't find one that did exactly what I want to do, so I created one instead.
This role will handle installing LXC 2 on CentOS and then will create containers listed in the lxc
group on the inventory.
To install this role, we need to run:
cd ansible-project
ansible-galaxy install --roles-path ./roles edmondscommerce.lxc
And we should see output along the lines of:
joseph@joseph-thinkpad ansible-galaxy install --roles-path ./roles edmondscommerce.lxc
- downloading role 'lxc', owned by edmondscommerce
- downloading role from https://github.com/edmondscommerce/ansible-role-lxc/archive/master.tar.gz
- extracting edmondscommerce.lxc to /opt/Projects/ansible-scratch/ansible-project/roles/edmondscommerce.lxc
- edmondscommerce.lxc (master) was installed successfully
- adding dependency: edmondscommerce.copr-repository
- downloading role 'copr-repository', owned by edmondscommerce
- downloading role from https://github.com/edmondscommerce/role-copr-repository/archive/master.tar.gz
- extracting edmondscommerce.copr-repository to /opt/Projects/ansible-scratch/ansible-project/roles/edmondscommerce.copr-repository
- edmondscommerce.copr-repository (master) was installed successfully
Note
The LXC role has a dependency on another role - edmondscommerce.copr-repository See how ansible-galaxy has resolved this and also installed the dependency for us.
Main Playbook¶
So now we have our inventory and roles sorted out, we need a playbook to run
In the root of ansible-project
directory, make a file called playbook-main.yml
with these contents:
- name: install lxc and create lxc containers
hosts: localhost
become: true
roles:
- edmondscommerce.lxc
Sync this through to the VM¶
vagrant rsync
Enter the machine, become root and create the containers¶
vagrant ssh
sudo bash
cd /vagrant/ansible-project
ansible-playbook -i inventory.local playbook-main.yml
Tip
Don't edit the ansible-project
files inside the VM
Instead, edit them on your host machine, but have vagrant rsync-auto
running in a separate terminal
This will keep all changes synced
Destroy all Containers¶
If you want oto properly test things, you should be totally destroying and rebuilding your conainers
This one liner will destroy all containers
for c in `\lxc-ls`; do lxc-destroy -fn $c; done
Yaml Syntax¶
Ansible uses Yaml, so you need to have an understanding of how Yaml works
- https://learnxinyminutes.com/docs/yaml/
- https://kapeli.com/cheat_sheets/YAML.docset/Contents/Resources/Documents/index
Multi LIne values:¶
Ansible uses YAML syntax in its playbooks. YAML has a number of block operators:
The > is a folding block operator. That is, it joins multiple lines together by spaces. The following syntax:
key: >
This text
has multiple
lines
The | character is a literal block operator. This is probably what you want for multi-line shell scripts. The following syntax:
key: |
This text
has multiple
lines
Would assign the value This text\nhas multiple\nlines\n to key.