If you have a bash script that you want to make sure there is only ever one instance of, for example something triggered by cron that might not have finished the next time cron tries to trigger it then you might like this little snippet:

This is built for running Magento shell scripts (if you don’t know about these, check them out) that are run on cron.

Also note the logging that keeps log files

Note the use of a character class in grep means it wont match itself – nice eh

#!/bin/bash
HOUR=<code>date +'%H:%M'</code>
RUNNING=<code>ps waux | grep &quot;longrunner[.]php&quot;</code>
if [ &quot;&quot; == &quot;$RUNNING&quot; ]
then
    echo &quot;Its not running, we can now run it&quot;
    php -f /home/my/public_html/shell/longrunner.php -- import &gt; /home/my/public_html/var/log/mylog.txt 2&gt;&amp;1
    cp -f  /home/my/public_html/var/log/my.txt /home/my/public_html/var/log/${HOUR}.my.txt
    echo &quot;COMPLETED&quot;
else
    echo &quot;It is running, aborting running this time&quot;
fi

OK so you love Magento’s shell scripts! They are great for quickly putting together bulk processes that have full access to the Magento ORM.

These kinds of shell scripts will often require above normal amounts of RAM for the kind of heavy lifting they lend themselves well to. If you are finding your script is banging against some mysterious memory limit that really shouldn’t be there then perhaps this is your answer:

You might see an error message such as this:

Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20 bytes)

The reason this might be happening is that one of the things the abstract shell class does is parse the .htaccess file and apply any PHP configuration it finds. This is pretty slick and I can definitely see the value of it however if you are not aware its happening and you have a memory limit defined in a .htaccess file then it will be applied to your shell environment.

The solution is up to you, edit the htaccess file or override this method in your shell class so that it doesn’t do things you do not want:

    /**
     * Parse .htaccess file and apply php settings to shell script
     *
     */
    protected function _applyPhpVariables()
    {
        $htaccess = $this->_getRootPath() . '.htaccess';
        if (file_exists($htaccess)) {
            // parse htaccess file
            $data = file_get_contents($htaccess);
            $matches = array();
            preg_match_all('#^\s+?php_value\s+([a-z_]+)\s+(.+)$#siUm', $data, $matches, PREG_SET_ORDER);
            if ($matches) {
                foreach ($matches as $match) {
                    @ini_set($match[1], $match[2]);
                }
            }
            preg_match_all('#^\s+?php_flag\s+([a-z_]+)\s+(.+)$#siUm', $data, $matches, PREG_SET_ORDER);
            if ($matches) {
                foreach ($matches as $match) {
                    @ini_set($match[1], $match[2]);
                }
            }
        }
    }

I had a requirement to use Xdebug to step through a PHP script that is being run on the command line.

Oh no I thought, this is bound to be really complicated to figure out because I normally use the web browser and I have never done this before.

Pleasant surprise, its actually really easy to do this, all you need to do on the command line is run the following command:

export XDEBUG_CONFIG="idekey=netbeans-xdebug"

Then get your IDE (PHPStorm for me) listening for Xdebug and then run your PHP file on the command line.

php ./myfile.php

That’s it, it just works.

One caveat (which I actually quite like) is that it will now ALWAYS get picked up by xdebug which could get in your way. To stop this, simply unset the variable like this:

unset XDEBUG_CONFIG

When you are monitoring a log file you may want to narrow it down, and format the results. This simple one line command will break up the output from a log file to make it easier to quickly read


tail -f /path/to/log | grep "search term" | sed 's/\(.*\)/----------\n\1\n----------/'


In a follow up to our popular blog post describing the process of getting AMTU 1 set up on a Centos server, this post describes how to get the latest version 2 of AMTU running.

Things have moved on and thankfully it is a lot easier than it was.

Firstly get Sun Java installed. You will need to get the URL for downloading the RPM from here:

http://www.oracle.com/technetwork/java/javase/downloads/jre7-downloads-1637588.html

Accept the agreement, download the correct RPM version and then pause it. Copy the download URL and paste it into this command:

cd /tmp
wget [paste-url]-here]
rpm - jre [hit tab to complete]

Now check the correct java version is being used:

java -version

You should get something looking like:

[root@94 tmp]# java -version
java version "1.7.0_05"
Java(TM) SE Runtime Environment (build 1.7.0_05-b06)
Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)

Now you can install AMTU

wget https://d28hcfptedr5ia.cloudfront.net/install/AMTU_unix.sh
chmod +x AMTU_unix.sh
./AMTU_unix.sh

Keep hitting enter to get to the bottom of the terms and then agree, agree to all the other options. That is AMTU installed.

The final step is to configure this using the CLI configure utitlity.

You need to create an xml file for the AMTU configuration. You can see a sample file in /opt/amtu/xml/sample.xml.

Lets copy that file and then edit it accordingly.

cd /opt/amtu/xml
cp sample.xml configuration.xml
vim configuration.xml

You do need to be signed up for MWS to get some of the keys etc that are required. If not already, go to this page and hit the sign up button.

https://developer.amazonservices.co.uk/index.html

Once you have updated the XML configuration you run this command to apply it:

/opt/amtu/Utilities/configure SETUP /opt/amtu/xml/configuration.xml

If you get some error messages at this point, edit the configuration and try again. Once you have the configuration set you can start the daemon.

/opt/AMTU/Utilities/amtu_daemon start

If you want to make git-diff use colours by default, try running this little command:

echo '[color]
 branch = auto
 diff = auto
 status = auto
[color "branch"]
 current = yellow reverse
 local = yellow
 remote = green
[color "diff"]
 meta = yellow bold
 frag = magenta bold
 old = red
 new = cyan
[color "status"]
 added = yellow
 changed = green
 untracked = cyan' >> ~/.gitconfig

Simply copy and paste the above to your command line and hit return.

Then if you run:

git diff master..BRANCH

You will get a nice coloured display

Found this great article on writing bash scripts defensively. Glad to say most of the advice we are already following with our bash scripts however there are a couple of new things I intend to roll in on Monday.

Recommended reading for anyone scripting in Bash.

http://www.davidpashley.com/articles/writing-robust-shell-scripts.html

If you need to transfer a large amount of data from one server to another you might normally create a large tar archive, send it across and then untar it on the other end.

However a slightly slicker and easier repeated way of doing this is to pipe tar directly from one server to the other doing all three steps in one go.

For this you would create a script on each server. You could of course type the commands directly but in the interests of easy repeatability and removing the scope for human error, scripts are my preferred way of doing this kind of task.

Old server script:

#!/bin/bash

cd /var/www/vhosts

# where new server IP is 123.123.123.123 and 9999 is the port the new server is listening on
tar -xvj . | nc 123.123.123.123 9999

New Server Script:

#!/bin/bash

cd /var/www/vhosts

nc -l 9999 | tar -xvf --same-owner

Now save these scripts and make them executable (chmod +x)

Now run the new server script on the new server and then run the old server script. You should quickly see the list of files being transferred appearing in both terminals.

That’s it.

Running PHP on the command line, via cron etc there are a few common problems you can hit and from experience this my recommendations to avoid them.

It seems that (on CentOS at least) PHP does not display errors by default on the command line. You will want it to I assume so you can understand what has gone wrong with the cron task etc.

You can use php.ini files but its not always easy to know exactly which ini file is going to to be used. If certain settings are required, use the -d flag to set specific ini directives directly when you run PHP.

I have just had an issue where a script was running out of memory but no error message was displayed, despite the fact that display errors was turned on. However digging around showed that memory usage was hitting the limit and the script died silently. Increasing the memory limit fixed it.

Here is my standard PHP CLI runner:

/usr/bin/php -d display_errors -d memory_limit=512M /path/to/php/file.php

The bash shell is awesome and scripting using a combination of PHP and Bash is a favourite technique of mine.

I want to be able to do it all in Netbeans but if you have the PHP flavour of Netbeans then its missing shell scripting support as standard. To get it to work easily, simply install the C/C++ plugin by going to Tools-Plugins-Available Plugins and ticking the C/C++ box.

It would be great if Netbeans would include bash scripting in with the PHP module so we can keep Netbeans as trim as possible, however this solution gives the desired result so I’m happy enough