As I’ve mentioned my server provides Ruby On Rails service for companies also, so I’d like to write in short about Rails hosting. Rails is a framework written in Ruby. There’s a ruby application repository called rubygems like apt-get. So when you want to upgrade your Rails, as in apt-get, you can do it by the gem management application. The current rails version is over the 2.0 release and rubygems is over the 1.3. When we were working on the sites (those are being hosted currently), we used Rails 1.2.3 and rubygems 0.7. Today these versions are unsupported, and the upgrade is not so trivial.

Rails application opens ports by its own server engine (you should only call script/server in the project to start a process on the 3000 port), or you can use other software, for example Mongrel (there is another solution with fastcgi, but the restart is complicated). You can specify the number of the listening ports and the number of the parallel threads to response to the requests. You need to bunch these ports to port 80, so you need a server that proxies the requests coming on the port 80 to the Rails processes. So briefly, you need a web server to proxy, and Mongrel processes which run the Rails applications.

Currently the Passenger module to Nginx and Apache skips the application server part of the previous process (the Mongrel processes), and creates Rails instances directly from the web server, so you should specify only your application’s public directory in the web server’s configuration file, and it works out of the box. So, I have an old active system with the 3-step-process, and there is a better solution that I would like to support and would be comfortable in the future.

I was sitting on a fence how I should solve this problem, how I could separate similar environments to different scopes. Building a virtual server park could be an alternative, but I don’t want to run a lot of apache web servers and ssh daemons concurrently. My other problem was ssh protocol doesn’t not support virtual hosts, and since I haven’t got different IP addresses to each virtual server, I should support ssh access on different ports, which I don’t want.

I tried to make a solution for this hosting problem, and finally the result was different chrooted environments for the old and the new version of Rails. I decided to use that kind of separation for all bigger components, like database services, name server, mail server, developer part, so I would call these bigger environments as racks. Backuping files is very important on servers, therefore I needed to found a solution for that problem also, so I’d like to show some tricks which is very comfortable later.

Receipt:

You will need squashfs and aufs first. Aufs is an union filesystem in kernel space (using postfix caused some aufs ooops in kernel, so I recommend to use fuse-unionfs in this case, the problem it’s running in user space, so the context switches cause performance decrease)

First create a minimal chroot environment that will be extended to service oriented rack. Using mksquashfs, we pack it to an image file that will be the hibernated service. Using aufs, make a merge of the read only image file’s content with a writable directory on the file system. All changes in this rack appear in this writable directory, so when you want to make a backup, you have to save only the writable partition. So the goal is each read only image (created by mksquashfs) contains (only) the whole software environment for a service (like mail-server, database-server), and each more file like configurations, created and modified files will be on the writable partition.

My directory structure is the following:

/racks - all the data of the racks
/racks/iso - iso files which contain the services
/racks/readonly - readonly directories mounted by loopback
/racks/readwrite - modified files
/racks/active - merged readonly and a readwrite directories
/var/share - shared files like unix sockets, common log files...
/racks/share/dev - dev directory into chroot environments

Building steps:

Let webserver-1.0.iso is the web server environment. Building racks will be in the next part of this thread, till then you can create chroot environment with debootstrap in Debian or Ubuntu (of course it won’t contain webserver components), or download a Gentoo stage 3 archive.

debootstrap --variant=minbase lenny /tmp/chroot_base
mksquashfs /tmp/chroot_base /racks/iso/webserver-1.0.iso

webserver-1.0.iso is in the /racks/iso directory, and you can burn it to a disc for backup. According to the previous directory roles, you need to create the corresponding directories in the rack structure, so you need

/racks/readonly/webserver
/racks/readwrite/webserver
/racks/active/webserver

Mount the image to the readonly directory:

mount -o loop,ro /racks/iso/webserver-1.0.iso /racks/readonly/webserver

Merge it with the readwrite part to the active directory

mount -t aufs -o br=/racks/readwrite/webserver=rw:/racks/readonly/webserver=ro none /racks/active/webserver

After merge is ready, we have to mount the default directories:

Mount /proc

mount -t proc none /racks/active/webserver/proc

Mount /dev

mount -o bind /racks/share/dev /racks/active/webserver/dev

Mount /var/share

mount -o bind /var/share /racks/active/webserver/dev

Mount /home

mount -o bind /home /racks/active/webserver/home

And finally, jump to the rack with

chroot /racks/active/webserver

I have a script that runs these steps, starts daemon applications after the mount process, when you want to stop the rack it stops all daemons and processes and unmount the directories in the right order.When you have a new upgrade from a software (for example when a new Nginx or Apache is released), just extract the current image to somewhere, upgrade the software in the new one, test it by chroot into it, when everything seems to be good, make a new iso with squashfs and edit your script to use webserver-2.0.iso instead of 1.0. The writable layer is the same, and if something goes wrong, you just back up to 1.0.

In the next part I’m going to show how you can create environments from scratch, which contain only that software you need.