SIDUS—the Solution for Extreme Deduplication of an Operating System

SIDUS (Single-Instance Distributing Universal System) was developed at Centre Blaise Pascal (Ecole normale supérieure de Lyon, Lyon, France), where one administrator alone is in charge of 180 stations. Emmanuel Quemener started SIDUS in February 2010, and he significantly cut his workload for administering this park of stations. SIDUS is now in use at the supercomputing centre PSMN (Pôle Scientifique de Modélisation Numérique) of the Ecole normale supérieure de Lyon.

With SIDUS, you can provide a new user with a complete functional environment in just a few seconds. You can probe corrupted computers without disassembling anything. You can test new equipment without installing an OS on them. You can make your life so much easier when managing hundreds of cluster nodes, of workstations or of self-service stations. You drastically can reduce the amount of storage needed for the OS on these machines.

Disclaimer

SIDUS is not LTSP. LTSP is a solution for the simplified management of thin terminals through X11 or RDP access to a server. Thus, all the processing load is on the latter server. On the contrary, SIDUS makes full use (or partial use, as the user wishes) of the station's resources. Only the OS is stored remotely.

SIDUS is not FAI. FAI or Kickstart offer full simplified installs so that administration can be reduced or dismissed altogether. On the contrary, SIDUS offers a single system in a tree that integrates the base system as well as all manually installed applications.

SIDUS is flexible. When organizing IT-training sessions, you might want to give participants a specific virtual environment. But once they download it, you cannot modify it for them. SIDUS offers users a single given environment that is easily configurable at any moment.

SIDUS is not exotic. SIDUS makes use of services available with any distribution (DHCP, PXE, TFTP, NFSroot, DebootStrap and AUFS). You can install SIDUS knowing only these few keywords. Besides, SIDUS makes use of distribution tricks from live CDs. SIDUS works on Debian, all the way from version Etch.

How Good Is SIDUS?

SIDUS is:

  • Universal: platform-independent, x86 or x86_64 architectures.

  • Efficient: installing takes a few minutes, and booting takes a few seconds.

  • Energy-saving: it takes only one core, 1GB of RAM, 40GB in disk space and an Ethernet (Gbit) network.

  • Scalable: tested successfully on a hundred nodes.

  • Multipurpose: we chose to use Debian as it comes with broad integration of open-source scientific software.

Installing SIDUS on Your System

It takes a little preparation for your system to host SIDUS. We have several services at our disposal in order to deploy our clients: DHCP, TFTP and NFS servers. Now, either you are on great terms with your own IT staff, or you are able to access freely the well-defined LDAP and DNS servers:

  • DHCP service provides the client with one IP address but propagates two complementary pieces of information: IP address of the TFTP server (variable "next-server") and the name of the PXE binary, often called pxelinux.0.

  • TFTP service then comes into play. Booting the system is enabled by TFTP, through the binary pxelinux.0, the kernel and startup of the client's system. If you need to give a client some parameters, you just build a dedicated file whose name stems from the client's MAC address (prefixing with 01 and replacing : with -).

  • NFS service now enters the loop: it gives the system's root via its protocol (NFSroot). Accordingly, you will install your client system in this root—for example, /src/nfsroot/sidus.

In our configuration, we have used isc-dhcp-server, tftpd-hpa and nfs-kernel-server for the servers DHCP, TFTP and NFS, respectively. Let's look into this configuration.

For DHCP, the configuration file (/etc/dhcp/dhcpd.conf) reads:


next-server 172.16.20.251;
filename "pxelinux.0";
allow booting;

For TFTP, there are three files and one directory (pxelinux.cfg) in /srv/tftp:


./pxelinux.0 
./vmlinuz-Sidus
./initrd.img-Sidus
./pxelinux.cfg

The pxelinux.0 file comes from the syslinux-common package. In pxelinux.cfg, there is the file called default.

To boot, you need the following: the kernel vmlinuz-Sidus, the system initrd.img-Sidus and the server NFSroot 10.13.20.13 with the mountpoint /srv/nfsroot/sidus.

Below is an example of a boot file. It takes two inputs: tmpfs and iscsi (we'll come back to the iscsi input later on):


DEFAULT tmpfs

LABEL tmpfs
KERNEL vmlinuz-Sidus
APPEND console=tty1 root=/dev/nfs
    initrd=initrd.img-Sidus
    nfsroot=10.13.20.13:/srv/nfsroot/sidus,
    rsize=8192,wsize=8192,tcp ip=dhcp aufs=tmpfs

LABEL iscsi
KERNEL vmlinuz-Sidus
APPEND console=tty1 root=/dev/nfs
    initrd=initrd.img-Sidus
    nfsroot=10.13.20.13:/srv/nfsroot/wheezy64,
    rsize=8192,wsize=8192,tcp ip=dhcp aufs=iscsi
    ISCSI_TARGET_IP=10.13.20.14
    ISCSI_INITIATOR=iqn.2013-04.zone.sidus.target:
    default root=LABEL=ISCSI

Regarding the NFS server, it takes one line in the file /etc/exports to configure it:


/srv/nfsroot/sidus
10.13.20.0/255.255.255.0(ro,no_subtree_check,async,no_root_squash)

Here, we open a read-only access to stations with IP between 10.13.20.1 and 10.13.20.254.

Once you have configured these three services (DHCP, TFTP and NFS), you can install a full SIDUS. Note that you also will need a root for user accounts (via NFSv4) and a process enabling their identification/authentication (via LDAP or Kerberos). We have deployed SIDUS on environments where these services are provided by third-party servers but also on standalone environments. Installing an OpenLDAP server with SSL or a Kerberos server is off-topic, so we simply show the client configuration files for our infrastructure (again, LDAP for identification/authentication and NFSv4 for user folders).

______________________

Emmanuel Quemener defines his job as an "IT test pilot". His work at the HPC "Centre Blaise Pascal" (Lyon, France) involves software integration, storage, scientific computing with GPUs and technology transfer in science.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Interesting system. Certainly

Ann Honni Mousse's picture

Interesting system.
Certainly lighter to install and maintain than MIT's Athena (but less powerful too).

Interesting

Anonymous's picture

Respect to the sysadmins.

Reply to comment | Linux Journal

Internet Marketing's picture

Great post. I was checking constantly this blog and I am impressed!
Very useful information specifically the final part :) I
deal with such info a lot. I used to be looking for this
particular info for a very lengthy time.

Thank you and good luck.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix