Setting Up Virtual Security Zones in a Linux Cluster

The Distributed Security Infrastructure lets you create disjoint virtual security zones on a physical Linux cluster.
The Problem

In this section, we walk through a simple scenario, which presents a problem and explains how DSI can help solve it. Say we want to share a cluster of two nodes (we begin small), among two telecommunication operators, PhoneMania and RingBell, each running their own applications on the cluster's nodes. Both offer a phone quotation service: end users call entry point servers (using TelecomClient) and request quotes for given companies. The entry point servers (PhoneManiaEP and RingBellEP) forward the requests to their back-end servers (PhoneManiaBE and RingBellBE), which retrieve the quotes and send them back to the end user.

From a cluster operational point of view, the problem is the following: how can we prevent a PhoneMania application from forwarding its requests to RingBell's back-end servers? Without any specific security infrastructure, PhoneMania could do so when its back-end server is overloaded or simply when it does not have the requested information—not to mention more aggravated scenarios of subscriber's data theft or intentional harm meant for competitors and so on. To illustrate such a scenario, we implemented all actors as simple UDP client and server applications (Figure 4).

Figure 4. Simple Telecom Scenario

Here is the fraud scenario step by step:

  • PhoneMania and RingBell launch their back-end servers on a node called munster:

    [munster demo]$ ./RingBellBE -h -p 9001
    RINGBELL: bind on
    [munster demo]$ ./PhoneManiaBE -h
    -p 8801
    PHONEMANIA: bind on

  • Then, as PhoneMania is overloaded, he decides to use RingBell's resources. So, on node colby, the entry point server of PhoneMania (port 8800) forwards all requests from his customers to RingBell's back-end servers (port 9001):

    [colby demo]$ ./PhoneManiaEP -h
    -p 8800 -b -r 9001
    PHONEMANIA: bind on
    PHONEMANIA: connect on

  • When a client requests a quotation at PhoneMania's entry point (port 8800), PhoneMania actually uses RingBell's back-end server to answer (port 9001). Simply put, PhoneMania gets paid by using RingBell's resources:

    [colby demo]$ ./TelecomClient -h
    -p 8800
    Connecting to :
    Requesting quotation for Ericsson
    Quote Ericsson
    [munster demo]$
    RINGBELL backend : processing quotation request for
    RINGBELL backend : quotation for Ericsson is 83
    Quote Ericsson

To prevent this, we propose to subdivide the shared cluster securely into disjoint zones with DSI. Next, we show step by step how to use DSI to do this.

Installing and Configuring DSI

First, we need to install DSI on all nodes of the cluster. After downloading the latest DSI tarball from SourceForge (see Resources), DSI should compile on your machine, as it uses the standard configure and make strategy. We detail how to build and install DSI in the DSI documentation found on the SourceForge site.

You should run the Security Manager on each node. For our two-node cluster, this means it runs on colby and munster:

[colby]$ cd ~/dsi
[colby]$ source
[colby]$ ~/dsi/bin/dsiSecManager

To simplify, colby also acts as a security server:

[colby]$ cd ~/dsi
[colby]$ source
[colby]$ ~/dsi/bin/dsiSecServer

The SS and SMs communicate with each other using CORBA event channels.

We load the DSI kernel module DSM on each node to enforce security at the kernel level:

$ cd ~/dsi/lsm
$ su root
# ./load
# /sbin/lsmod
Module                  Size  Used by    Not tainted
dsm                    36332   0 (unused)

Then, we configure DSI by defining different IP addresses used on each node for secure and nonsecure communications. To do so, we wrote a tool called DciInit; see the DSI documentation at the SourceForge site for more details on the format of the dci_policy.conf file and how to use DciInit:

$ cd ~/dsi/user/tools
$ ./DciInit ~/dsi/etc/dci_policy.conf

The Solution: Setting Up the Virtual Subclusters

Basically, to create disjoint virtual subclusters, you need to assign different ScIDs to PhoneMania's resources (in our example, ScID=10) and RingBell's resources (ScID=20). Then, add new rules to DSP to restrict any connection from the zone defined by ScID 10 to the zone defined by ScID 20 and vice versa. By organizing the resources of each operator in separate groups, without any possible connection between them, we actually achieve a virtual subdivision of the cluster. Additionally, the administrator can create another zone defined by ScID 30 with privileges to access both subclusters for administrative purposes.

First, let's assign the ScIDs of each binary on each node (using the DSI SetSID tool):

$ ~/dsi/user/tools/SetSID PhoneManiaEP 10
Changing from SID 0 to SID 10
$ ~/dsi/user/tools/SetSID PhoneManiaBE 10
Changing from SID 0 to SID 10
$ ~/dsi/user/tools/SetSID RingBellEP 20
Changing from SID 0 to SID 20
$ ~/dsi/user/tools/SetSID RingBellBE 20
Changing from SID 0 to SID 20
$ ~/dsi/user/tools/ls_dsi .
-rwxr-xr-x      lmcaxpr install 10    PhoneManiaBE
-rwxr-xr-x      lmcaxpr install 20    RingBellBE
-rwxr-xr-x      lmcaxpr install 10    PhoneManiaEP
-rwxr-xr-x      lmcaxpr install 20    RingBellEP