Containers

Container Useful Commands

Copying / Push file on the cluster container

  • scpContainerPush [containerID - OR - containerName] [localPath] [remotePath]
  • Example scpContainerPush container-staging /path/to/local/ /path/to/remote/

Copying / Pull file from the cluster container

  • scpContainerPull [containerID - OR - containerName] [remotePath] [localPath]
  • Example scpContainerPush container-staging /path/to/remote/ /path/to/local/

Ssh into Cluster Container

  • sshContainer [containerName]
  • Example sshContianer container-staging

Creating Containers

Creating website containers

  1. ssh cluster1
  2. Choose a project type
    • Generic:
      1
      2
      cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/cluster/setupPublicFacingContainers/setupStagingContainer/
      bash ./createGenericContainer.bash [clientname] {optional container suffix - defaults to staging} {optional PHP version - defaults to 55 - options include 54|55|56|70}
      
    • Magento 2:
      1
      2
      cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/cluster/setupPublicFacingContainers/setupStagingContainer/
      bash ./createMagento2Container.bash [clientname] [pubKey] [privKey] {optional container suffix - defaults to staging}
      
  3. You may be asked the below during the container set up, if this is the case just use (E)xpand
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    You have an existing certificate that contains a portion of the domains you
    requested (ref: /etc/letsencrypt/renewal/"Client-Name".developmagento.co.uk.conf)
    
    It contains these names: "Client-Name".developmagento.co.uk,
    www."Client"."Name".developmagento.co.uk
    
    You requested these names for the new certificate: "Client-Name".developmagento.co.uk,
    www."Client"."Name".developmagento.co.uk,
    www.staging."Client-Name".developmagento.co.uk.
    
    Do you want to expand and replace this existing certificate with the new
    certificate?
    -------------------------------------------------------------------------------
    (E)xpand/(C)ancel:
    

Optional extra container setup

  • ssh-copy-id gitBare will prevent the need to type in passwords when git push/pulling
  • Nginx etc should be installed using Container Assets

Creating a Jira Container

Danger

DON'T CLOSE THE TERMINAL AT ANY POINT!

Not until you are 100% sure Jira is ready.

  1. Open up a terminal
  2. First we need to ssh into cluster 2. In the terminal input the following command.
    1
    ssh cluster2
    
  3. Then change the directory.
    1
    cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/cluster/setupPublicFacingContainers/setupClientJira
    
  4. Pick a client name for the Jira that is short but recognisable

    • It'll be a good idea to input the command containerList` so that you have a list of names already in use.
    • Normally we try aim for 4 or less characters.
  5. You then need to input the following command, changing the "clientname" to the name you have chosen. For example: bash .run.bash ec is what we might use for Edmonds Commerce.

    1
    bash ./run.bash clientname
    

  6. You will see the terminal going through the script and eventually reach a point where you need to go to proxmox. The terminal will ask you to type "done". Do not do this yet.
  7. Log into Proxmox and make sure you go through the set up first in the following steps:

    1. Now you have logged in, look for the container with the name you picked.
      • The name should be "clientname-jira".
    2. Double click it to open it.
    3. Go to the network tab.
    4. Click add
    5. Fill in the details that you can get from the terminal. If you can't see them on the terminal then you don't need it. But make sure you fill out everything you can.
    6. Click save
  8. Now type done in the terminal and hit enter

  9. Note down the output, you'll need this when setting up Jira

Accessing Containers

Accessing Cluster Containers

1
sshContainer container-name

will log you in as ec on the container. Container names can be found in containerList

Password Prompts when accessing containers

If you are being asked for a password then you might need to add the ssh key to the ssh-agent

1
2
ssh-add ~/.ssh/id_rsa
eval `ssh-agent -s`

Configuring Containers

MySQL/phpMyAdmin

phpMyAdmin is installed on all containers and is accessed at http://ip.add.re.ss/pma

The root password is contained in ~/.my.cnf

Allowing Paypal to send IPN to a developmagento website

If you need to test if Paypal orders are working on the developmagento website. You need to make some modifications to the public container to allow such things.

  • ssh to cluster by typing ssh cluster1
  • type pct enter 102 which will enter public container
  • change directory to cd /etc/nginx/conf.d
  • open the specific configuration file for your container, it should be [clientname].conf
  • Find the specific vhost you want to allow paypal to send requests and responses

Look for something similar to the below for specific vhost implementation:

Redacted

  • To disable the client certificate requirment, using a # symbol comment out the following lines:
1
2
    ssl_verify_client on;
    ssl_verify_depth 2;
  • Allow IP's from Paypal and deny from anything else by putting this code, after the server opening curly bracket:
1
2
    allow 194.12.9.18;
    allow any.any.any.any;
  • The final result should look something similar to this:

Redacted

Be sure to check Paypal's list of public IPs

Make sure to undo everything after you placed a test order using paypal...

// TODO: This Paypal should be done differently. It should still use client certificate, except skip it for Paypal IP's... However it takes time to find and test it how it should be done. Therefore needs to be postponed.

Debugging Containers that Won't Start

If you find a container that won't start, the best way to get detailed info is to run it in foreground mode with debug logging

for example:

1
lxc-start -n 230 -F -lDEBUG

Which will then give you some output along the lines of:

1
2
3
4
5
6
7
close (rename) atomic file '/fastboot' failed: Disk quota exceeded
error in setup task PVE::LXC::Setup::pre_start_hook
lxc-start: conf.c: run_buffer: 405 Script exited with status 1.
lxc-start: start.c: lxc_init: 450 Failed to run lxc.hook.pre-start for container "230".
lxc-start: start.c: __lxc_start: 1321 Failed to initialize container "230".
lxc-start: tools/lxc_start.c: main: 366 The container failed to start.
lxc-start: tools/lxc_start.c: main: 370 Additional information can be obtained by setting the --logfile and --logpriority options.

In this instance, it was hard drive space that had run out

Setting up new vhost on a public container

To setup some new vhost on a public container proceed with the steps below. An example of new vhost for www.something.someclient.developmagento.co.uk and someclient contaienr id is 333

1
2
3
4
ssh cluster1
pct enter 102
cd ~/createVhost
bash setup_staging_vhost.bash someclient www.something 333

Local Container SSL Certificate Copying

If you're working with a container that doesn't have local SSL keys for desktop.com in place under /etc/ssl/nginx then you need to proceed with the steps below

Note

The example below is for clientname-staging container. Please make sure you change commands to the container you're working with.

Redacted

Once you have the SSL certificates you need to set them up with your Nginx configuration. If you already have SSL certificates configured then it should be an easy drop in replacement, but if not, be sure the following is present.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
server {
    listen 80;
    listen 443 ssl http2;

    server_name www.mylocalsite.dev;

    ssl_certificate /path/to/your/file.crt;
    ssl_certificate_key /path/to/your/file.key;

    # ...

}

Fixing Nginx PID file issues

If you are having issues with nginx and client docs, this guide will cover how to fix the issue.

First thing you will need to do is ssh into the container you would like to apply the fix, see example below: sshContainer container-name

Once you are in the container be sure to run the following command: sudo bash (to avoid any permission issues)

Next run systemctl status nginx this will print 2 possible outcomes

outcome 1: Green circle which will indicate nginx is running as it should and nothing to worry about.

outcome 2: Red circle which means nginx is not running and you need to correct it. To solve that issue follow the steps below.

1
2
3
4
5
6
sudo bash
mkdir /etc/systemd/system/nginx.service.d/
echo "[Service]
PIDFile=/run/nginx.pid" > /etc/systemd/system/nginx.service.d/override.conf
systemctl daemon-reload
systemctl start nginx

When you have done the above step run systemctl status nginx one more time to make sure the output is green and nginx is running.

Creating certificates on key master

1
2
3
4
5
6
7
8
ssh cluster1

# Start the container by ID
pct start 154

# The commands below will generate SSL certificate keys and push into staging container on the cluster
cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/cluster/setupPublicFacingContainers
bash _nginxSSLCert.bash staging clientname

Pulling down the keys to local container

1
2
3
4
5
6
sudo bash
cd /var/lib/lxc/clientname-staging/rootfs/etc/ssl
mkdir nginx
cd nginx
# scp will download the keys from the cluster machine
scp cluster1:/etc/ssl/nginx/clientname/certs/www.staging* .

Upgrading PHP

Option 1 - Container Asset

Probably the safest way is to use the container asset.

  1. Make sure that you have copied the latest version of hte container asset into your container
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#Copy container asset into container
ssh cluster 1
cd /opt/Projects/cluster/shellscripts/cluster
./pushContainerAssetsToContainer.bash PHP-FPM_Nginx {CONTAINER_ID}

#Enter the container
pct enter {CONTAINER_ID}

#Become root, remove old PHP and install new
sudo bash
yum remove 'php*'
cd /home/ec/PHP-FPM_Nginx_Install
bash run.bash 72

#Check installed version
php -v

Option 2 - Upgrade Script

To easily upgrade your PHP version, try this script:

To use this script to upgrade your PHP version, you might do something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
#Become root
sudo bash

#Go to /root
cd ~

#Grab the latest version of the script and save as `phpupgrade.bash`
wget https://gist.githubusercontent.com/edmondscommerce/d3a450d77bc31573dca9bb77cdcf33a3/raw/ -O phpupgrade.bash

#Edit the script, change the from/to PHP versions as required
vim phpupgrade.bash

#Now run the script
bash phpupgrade.bash

#Now check the PHP version
php -v

Don't forget to update your PHPStorm PHP version](/Development-Tools/PHPStorm.md#setting-the-php-version).

Cloning Containers

Warning

Containers on the Cluster are designed to be the One True Container. These are the only containers that are backed up, and other developers will clone the Cluster container.

Always make sure valuable changes to local containers are also applied to the Cluster container.

Cloning containers from cluster to desktop

There is a bash function with tab completion that makes this process a bit easier. There is a separate function for each cluster machine.

This command is a wrapper around /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/desktop/lxc/cloneFromCluster.bash.

1
cluster[X]_clone [container-name] (skip_dump=true)
  • X represnts the cluster number
  • container-name can be found in containerList
  • skip_dump dictates whether or not a new dump of the container is made. This defaults to true, meaning last night's dump will be used.

Examples

1
2
3
cluster1_clone ec-internal-jira # Clone ec-internal-jira from cluster1
cluster2_clone clientname-jira  # Clone clientname-jira from cluster2
cluster1_clone ec-internal-jira false # Creating a new dump

Local container hosts file entry

  1. On the container, establish the "desktop.com" domain in /etc/nginx/conf.d/{clientname}.conf
  2. Find its IP address with ip addr (or lxc-ls from host)

Redacted

You might then choose to add a symlink in /opt/Projects:

1
2
cd /opt/Projects
ln -s /var/lib/lxc/clientname-staging/rootfs/var/www/vhosts/{clientname}/

Cloning containers from other desktops

Execute commands below to clone the container to your local machine

1
2
cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/desktop/lxc
sudo ./cloneFromOtherDesktopMachine.bash container-name desktop-hostname-or-ip [desktop-username]

Cloning The Same Container Multiple Times

If you try to clone a container that you already have, then you need to manually fix some things to allow them both to exist locally.

  1. Rename your current container

    1
    2
    3
    su
    cd /var/lib/lxc
    mv container-name container-name-orig
    

  2. Clone as normal Either from Cluster or other desktop, or if local simply copy the /var/lib/lxc folder

  3. Update Config

1
2
3
su
cd /var/lib/lxc/container-name
vim config

You need to update:

  • lxc.rootfs
  • lxc.utsname

  • Update Network Config

1
2
3
su
cd /var/lib/lxc/container-name/rootfs/etc/sysconfig/network-scripts
vim iffg-eth0
And you need to set the IP address to a new local one

1

Deleting a Container and Cleaning Up

Archiving and Removing a Container

You have to be on the cluster machine to run this. Do the following

1
2
cd /opt/Projects/snippets-edmondscommerce/Cluster/shellscripts/cluster/
bash archiveAndRemoveContainer.bash container-name

This will create an archive, place that into Dropbox, and then delete the container.

Removing public nginx config

1
2
3
4
ssh cluster1
pct enter 102 # for the public container
rm /etc/nginx/conf.d/{container-name}.conf
rm -rf /var/www/vhosts/{container-name}.developmagento.co.uk

Removing SSL Certificates and Keys

1
2
3
4
ssh cluster1
rm -rf /etc/ssl/nginx/{container-name}
pct enter 109 # for the keyMaster container
rm -rf /etc/ssl/intermediate/{container-name}

Container Creation Method

  • Script that we are running is Cluster/shellscripts/cluster/setupPublicFacingContainers/setupStagingContainer/createGenericContainer.bash
  • Naming should be clientName/containerSuffix (e.g edmondscommerce/handbook)
  • bits of the container can not have space or dashes
  • if we only expect to have a one container for the client then staging is fine (which is the default) otherwise use something else
  • php version defaults to 55 which is magento 1 compatible but should use highest
  • for magento 2 use 70, magento 2 requires 70 at least ( for brand new magento 2 use 7.1 for legacy magento 2 use 7.0)
  • project Name defaults to staging
  • SO the command would be Cluster/shellscripts/cluster/setupPublicFacingContainers/setupStagingContainer/
  • bash createGenericContainer.bash {clientName (edmondscommerce)} {containerName (handbook)} {php version (71)}
  • it creates a new centos container,
  • it installs mysql first and secures it
  • it installs php-fpm and modules
  • it installs composer
  • it installs ngnix
  • it installs the phpmyadmin in default folder
  • it creates default vhost
  • clone the container down
  • whilst that running
  • go to git bare: ssh gitBare which is where our clients repos are
  • cd repos/
  • create a client folder if its not there already and go into the folder
  • e.g cd edmondscommerce/
  • e.g git init --bare edmondscommerce-handbook to initialize a repo
  • after cluster is cloned down
  • follow the instruction given at the end by the clone script
  • lxc-attach into new container
  • sudo bash to become root
  • cd /home/ec/PHP-FPM_Nginx_Install/
  • bash runOptional_setupVhost.bash {repoName (e.g edmondscommerce-handbook)} standard
  • this creates a vhost file in /etc/nginx/conf.d/
  • now in your main terminal, update the hosts file to add a entry e.g (192.168.122.26 www.handbook.edmondscommerce.desktop.com)