Fibre Channel for Linux

Data communication has traditionally been divided into two quite distinct worlds, the network world and the channel world. The channel world usually deals with well-structured, closed and fixed environments. Usually there is one host system and several attached devices, like a master-slave setup. Communication is usually restricted to occur only between the master and one of the attached devices.

In the network world, the picture is quite different. Here the environment is unstructured, open and behaves much more unpredictably. All devices can talk to any other device at any time. One consequence of this is that more control is needed to correctly handle connections, access permissions, route information and other aspects of correct behavior. The result is that channels are often characterized by high throughput and low overhead, while networks tend to have low throughput and high overhead. However, networks are much more flexible, more scalable and run for longer distances than channels. Fibre Channel (FC) attempts to combine the best of these two worlds. The work on the Fibre Channel standard started in 1988 as an extension of work on the Intelligent Peripheral Interface--Level 3 (IPI-3). Fibre Channel is today an open standard as defined by ANSI (American National Standards Institute) and OSI (Open Systems Interconnect).

Some of the goals of the Fibre Channels standard are to provide support for existing protocols, flexible topologies, high bandwidth and long distances.

One of the features of Fibre Channel is the use of existing, well-established protocols for data transmission. These are called upper layer protocols in Fibre Channel. FC does not impose any new format for data, hence all existing applications can continue to use standard protocols. Fibre Channel just becomes the transport layer for these protocols. Hence, there is no need to rewrite any applications in order to take advantage of Fibre Channel. Both SCSI and IP are among the protocols Fibre Channel supports. Of course, vendors can choose what protocols to actually implement in a product; usually only a few upper layer protocols are implemented in a product.

Fibre Channel offers three different topologies that can be combined to fit any needed configuration. The simplest topology is a point-to-point connection between two nodes. A node in FC is any device attached to a FC network; both computers and storage devices are nodes. A more advanced topology is available by using an Arbitrated Loop. This configuration allows up to 126 devices to share one loop. Finally, Fibre Channel switches are available that can form fabrics with thousands of nodes. In addition, Fibre Channel is hot-swappable, meaning new nodes can be added to or removed from a topology without powering all other nodes down.

Concerning bandwidth, the Fibre Channel equipment currently available supports 1 Gbps (100 MBps data transfer). Two Gbps equipment is under development, and announcements have been made that some equipment will be available in 2001.

With the right equipment, a single Fibre Channel link can run up to 10 km without the need for repeaters. Single-mode optical fibre and 1,300 nm lasers are needed for this. With multi-mode fibre, distances up to 500 meters are supported, and electrical cabling is available up to 13 meters. It is important to make sure that all cables meet the specifications for FC-cables.

Fibre Channel Topologies

Four topologies are available in Fibre Channel: Point-to-Point, Fabrics, Arbitrated Loops and Fabrics with Arbitrated Loops.

Point-to-point represents the simplest Fibre Channel topology. It provides a dedicated 100MB connection between two nodes, be it two host machines, one host machine and one storage device, or one host and a switch. This kind of topology may be useful for connecting two machines that are physically separated by some distance and are in need of a dedicated high-bandwidth connection.

The topology created by using one or more switches is called a fabric. Switches were listed together with Point-To-Point part of the original FC specifications. The switch would provide point-to-point connections between itself and the connected nodes, thus allowing communication between all the nodes. Large topologies can then be created.

Arbitrated Loops were actually added on to the original FC-specifications. A topology that would combine some of the features of Point-To-Point with some of the features from Fabrics was needed. One of the main arguments against pure fabric is the price per fabric-port, which gets very high when each fabric-port is connected to just one node. Although this offers each device full connection to the fabric, most nodes do not constantly need the full bandwidth of Fibre Channel. The Arbitrated Loop offers (as the name suggests) a shared loop topology. The loop can contain up to 126 nodes and one fabric-port with all the nodes arbitrating for usage of the loop.

A FC loop physically connects the outgoing connectors to the incoming connector of the next node. At any given time, only one port can be sending frames. Before a port can transmit any data, it has to win an arbitration for the loop. Since several or all of the nodes might be arbitrating for usage of the loop at the same time, performance will degenerate as more nodes are added. A loop with 127 active ports is likely to not operate with the performance one would like.

By combining the Fabric and the Arbitrated Loop, very large and flexible networks can be created. A port on the switch can be connected to an Arbitrated Loop. Every device on this loop now has access to the other devices on the loop as well as every device that is connected to the switch. A loop connected to a fabric is known as a Public Loop.



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Re: Fibre Channel for Linux

Anonymous's picture

Just a warning the the URL for the FC Loop Community now points to Bechy's Webcam. While I enjoyed the view, I'd rather keep my job ;)