What is Rsync in general ?

Rsync works like the rcp command in linux. It is used to copy files across machines. It uses efficient checksum algorithm that looks for modified files by time and size. It is used for two purposes:

Backup

Rsync is used to transfer file from the local to a remote machine. By default Rsync will use rsh, however you can instead use ssh for high security. Compressed files are transferred to save file transfer time and reduce the load on the network. It helps to transfer bulk files in minutes.

Mirroring

Rsync can synchronize file between two servers location(i.e) local and remote. By using the checksum algorithm it updates local files only if it is updated in remote server.

How to use Rsync

rsync -avze ssh -p2020 root@server.com:/sitepath/files   /localdata

This command transfer files from remote server to local machine.

a Archive. It is denotes all related files and documents.
v Verbose.It will display status message as it is processing.
z Compression.It achives better compression ratio.
e It allows to chose alternative remote shell program for communication.

rsync -avze ssh -p2020 --exclude home/var/cache root@server.com:/sitepath/files /localdata

This command will transfer files from remote server to local machine by excluding cache folder.

rsync -avz /sitepath/files root@server.com:/localdata 

This command will transfer all file from local to remote server. It is uses default remote shell protocol.

What is an SSH access?

SSH stands for Secure Socket Shell. SSH is a protocol, like HTTP, it gives us a way to gain a secure access to remote computers.

The principle is to connect two networked computers via a secure encrypted channel within an insecure network environment such as the Internet.

With SSH access we can easily move files around and run programs on your server. We can also transmit files to and from our own network onto your server, for example to take a copy of your site for local development or to push updated files onto your server.

SSH needs a server application installing on your server (most web servers will have this by default) and an SSH client on your local machine to allow you to connect to your server.

Where do you get your SSH access from?

If you need SSH access details you should ask your hosting company, they should be able to install it for you if it is not already in place. If you are on shared hosting then the chances are that you will not be able to get SSH access details, if you need SSH access then you should consider upgrading your hosting to something that will support it.

SSH access details will consist of at least:

domain or IP address:

domain.com

123.123.123.123

username:

mywebserverlogin

password:

ihy324579ySDWf

it may also include a port if SSH is not running on the standard port.

2020

Why do we need the SSH access details?

We will always ask our customers to provide us with SSH access details. This is for two main reasons:

1. Power and Speed

SSH access allows us to run commands directly on the server, compress files, run scripts etc. For example we will generally run some static analysis scripts for our Magento customers using a bash script that we can run with SSH access. This gives us some instant feedback on the state of your Magento installation and allows us to spot any potential problems.

If we need to get a copy of your site files, using SSH we can prepare a compressed archive of your site files allowing us to download them much faster. Likewise we can easily get a reliable database dump, especially if you have a large database that might cause problems for browser based tools such as phpMyAdmin.

2. Security

SSH is a much more secure protocol than something like FTP. In SSH everything is encrypted. It has been built from the ground up to be a very secure way of networking and inter communicating between servers on an insecure network such as the web.

If you have a long running script that is designed to run as cron but may be run manually from the terminal then you may want to enforce screen so that the process can’t be aborted by the SSH terminal being closed for whatever reason.

We already blogged about how to force running as screen here.

This is an extension of this that also checks for running as Cron and if so, does not force screen.

# First off, ensure running from Cron and if not, Screen
if [ -t 1 ]
then
    echo "Not running as Cron Task"
    if [ -n "$STY" ];
    then
        echo 'We are inside screen, good';
    else
        echo "Not in a screen, please run with screen"
        echo "Exiting"
        exit 1
    fi
else
    echo "Running as Cron Task"
fi

Simply paste this somewhere close to the top of your script before it does anything and this will work nicely.

If you like the admin functionality to manage plugins, themes and other updates that wordpress features and would like to have this working when you are developing locally then you might find that you need to set up an FTP server locally just for this purpose.

Perhaps a better solution, especially if you use a *nix based desktop such as Linux is to try this extension:

http://wordpress.org/extend/plugins/ssh-sftp-updater-support/

This will extend the FTP functionality to also use SFTP or SSH for the updates. If you are already running an SSH server on your desktop then there is very little left to do. If you are not then a quick

sudo apt-get install openssh-server

And you are ready to go.

From this point you can then use the admin functionality to manage plugins etc which is a nice slick way of doing things.

Useful tip – if you’ve ever been ssh’d into a remote machine and don’t want to break the connection or open a new session just to add a new tunnel, there is a way

Press [return] then ~ then C to get to an ssh prompt, then you can add tunnels as you would at the command line e.g.
-L 80:localhost:8080

Then press [return] to return to the session you were running. Cool trick.

If you are handling a server migration and would like to have a scripted way to copy the crontab from one machine to the other then you might like this little snippet.

ssh -p2020 root@123.123.123.123 'crontab -l' | crontab -

This will get the contents of the root crontab from one server and apply it to the current server, replacing any current cron tab settings.

This is nice if you want to have a repeatable server resynch process whilst you are migrating and the crontab on the old server may change

So there is often a requirement for sharing files but what happens if those files are files that absolutely must be secure, like private ssh keys?

Well Dropbox whilst a good product is third-party and not encrypted so if there’s an exploit, the contents of the files could well be exposed to people you don’t want.

Enter Wuala, a service by Lacie. Similar to Dropbox in many ways, but differing in that the files are encrypted locally, so if you loose your password, the files will be inaccessible, permanently. Believe it or not, that’s a really good thing!

Oh, and there’s a client for Linux, Mac, Windows and Mobile (Android and iPhone).

If you manage a few servers, you might often find yourself with a few terminal windows open and it can get tricky to remember which terminal window is which.

To make this a lot easier you can create a custom terminal launcher for your panel and make it use a custom terminal colour scheme and profile and also make it automatically log into the server in question via SSH so it really is either that server or nothing (unless you exit but keep the terminal window open for some reason).

For your production server for example you might do the following:

Open up a terminal and create a new profile by selecting Edit-Profiles then hit the [new] button and create a profile called “Production”. Customise it to suit.

To create a new launcher, right click your panel and select “Add to Panel”

Then click “Custom Application Launcher”

For the name, call it something obvious like “Production”

The command should be like this:

gnome-terminal --window-with-profile=Production --command="ssh username@server.com"

Now choose a nice icon by clicking the springy icon that is default and navigate around till you find one you like.

Save the launcher and try clicking it, it should automatically use the special colour scheme you set and should also either log you straight into SSH (if you are using SSH Keys) or will give you the password prompt.

Never be confused again!

If you are finding it takes ages for a password prompt to come up when trying to connect to a CentOS server (and possibly others) this is a solution.

The cause is that your IP address does not have the correct reverse DNS etc settings.

You can of course try to fix all that but if you need a quick fix, you can edit the sshd configuration and disable

GSSAPIAuthentication yes

to: GSSAPIAuthentication no

and

UseDNS yes

to: UseDNS no

If you are frustrated with Samba and would like something a bit more solid and speedier whilst being very easy to set up, check out SSHFS.

To install it you simply need to do

sudo apt-get install sshfs

Once that’s done you can mount a remote filesystem in a very similar way to accessing a remote machine via SSH

eg

#Mount
sshfs  -p2020 -o idmap=user $USER@laptop.com:/home/joseph/Projects ~/Mounts/Laptop/Projects

#Unmount
fusermount -u ~/Mounts/Laptop/Projects

You need to have previously created the local folder you are going to mount to (in this case /home/joseph/Mounts/Laptop/Projects)

Note also I am using a non standard SSH port with the -p2020 bit (using port 2020). If you are using standard SSH ports you can remove this, or of course change the port number to whatever you prefer.

https://help.ubuntu.com/community/SSHFS for more info