Streaming Media

Frank describes the hardware and software technology used on the server side of the streaming process.
Data Encoding

What is data encoding and why is this necessary? The most important process of encoding for computer delivery requires that the data be put into digital form. The real world is based on continuous or analog data. Most computers in use today are based on digital technology. Everything in the system must be represented by some combination of 1s and 0s because all any computer can do is turn individual circuits on (1) or off (0).

Two other factors exist that require encoding to be more than just a simple conversion of analog to digital data. Because typical data is owned and current majority opinion is that it must be protected from illegal copying and distribution, the digitized data stream is most often encrypted during or after the encoding process. Technological limitations on the speed of data flow over various communications links also require that the data be reduced in size, or compressed, so that existing hardware can smoothly execute the streaming process in real time.

The method you use to encode your content depends on encoding software and hardware, CPU processing capabilities and speed, network technology, and the data storage and retrieval requirements. If you intend to stream the data to an audience under your direct control, your questions will focus more on the technology required. If your business is to stream arbitrary data to arbitrary clients, in addition to the technology, you have to consider standards, de facto standards and specific customer requirements.

The term codec refers to the coder/decoder pair that is used to encode and decode your content. Most codecs are proprietary, and because of the huge volume of data that has been encoded in these hidden formats, it is likely to be some time before the industry completely moves to open standards, if it ever does. The most popular proprietary codecs used today come from RealNetworks, Microsoft and Apple.

However there is a strong movement toward the Moving Picture Experts Group (MPEG) formats. MPEG formats have the advantage of being recognized as international standards, and they offer the highest compression ratios with the smallest loss of data quality. There are two versions in common use, MPEG-1 and MPEG-2, but MPEG-4, a new format designed specifically for interactive multimedia applications, is likely to become the standard codec until MPEG-7 is released (possibly in 2001). The most popular flavor, MPEG-2, contains patented IP that can be used by executing a single MPEG-2 patent portfolio license (in the US,

Although the licensing is not free, the encoding and decoding algorithms are open, and open-source implementations can be created. MPEG LA announced in September of this year, a plan “aimed at providing fair, reasonable, nondiscriminatory worldwide access under one license to patents that are essential for implementing the international MPEG-4 Systems standard”. Use of open standards, such as MPEG, solve at least one main concern of companies encoding their multimedia technology. They will not be forced to return to any specific vendor for service in the future. There are so many patents that cover MPEG technology that it will take a significant engineering breakthrough to produce a competitive freely licensable alternative.

Do you need to understand the technology behind each codec thoroughly before making a decision on which one(s) to use? I would say that a complete understanding is not necessary, but certainly you should understand to the degree necessary to estimate resulting file sizes and to take into account any encoding and decoding overhead. You should also understand the needs of your clients before you decide on the codecs you will use. Proprietary codecs often have no client support for any platform other than the initial encoding platform. It is also very important to consider any technology licensing issues, and be especially conscious of how you will deal with financial liabilities that you may accumulate because of the use of specific technologies.

Data Storage, Retrieval and Transmission

Once you have required hardware and software to create encoded content, consider how you will store your content for transmission at a rate that satisfies the demands of your clients. If your data comes from a real-time, direct-media feed, such as from a video camera, current encoding hardware usually allows only one live feed per encoding card, and your storage requirements are simply the system RAM needed to hold encoded data before it passes to the Network Interface Card (NIC). For One to One, or One to Many data flow, this type of content delivery is straightforward. Because each live feed you supply to your clients requires an individual encoding card, and usually a dedicated computer as the server for that stream, use simple addition to figure out the increased costs.

If your data is preprocessed and streamed either at some later scheduled streaming time, or video on demand (VOD) by the client, then the hardware and software you use for these functions could become a critical bottleneck to the data flow through your system. Here is an example to demonstate the type of information you will need in order to estimate your system resources.

Suppose your business expects to supply a maximum of 10,000 clients each with continuous stream for one hour, but with a peak time of two hours per day where 80% of the clients could be expected to be on-line. Suppose that the average client demands 300 KBps streams to display flicker free, uninterrupted multimedia content. Suppose also that on average there are 100 different content files being accessed at any time from a pool of 5,000 titles.

Looking at the peak demand sets the upper-limit requirements of our data storage and retrieval subsystem. We decide not to offer VOD to our clients, but allow them only to tune in to some scheduled broadcast. That strategic decision allows us to weigh the number of different pieces of simultaneous content much more heavily than the maximum number of clients. Assuming that your network router can handle multicasting, i.e., sending out the same information to as many IP addresses as may register to receive the data (multicasting is just beginning to become available through some local ISPs), we have enough information to estimate the storage and retrieval subsystem requirements. Our total storage requirement is the product of the number of titles, the average run length of each file and the bit rate at which the data will be streamed. In this case, it is 5,000 files x 3,600 seconds run length per file x 300,000 bps/8 bits per byte of data = 675GBs of storage.

How fast must our data travel from the disk drive to the NIC in order to keep up with the expected client demand? To calculate this answer, we compute the product of the number of different files being read simultaniously from the disks and the average data rate: 100 x 300,000 / 8 = 3.75MBs per second. Had we decided to offer VOD, that number could jump by a factor of 8,000, the peak number of users who would be asking to start viewing the stream at totally random times, to give us a requirement of 30GBs per second of data needed to be read from our disk drives! We would most likely want to reduce that bandwidth requirement by some intelligent management method. For example, we could choose to allow a new video to start only at the start of each minute. That would be fairly transparent to the end user but would help by allowing us to take advantage of multicast capabilities in our system. We now have to consider the size and number of disk drives, the maximum average data-transfer rate from the drives, and the maximum data-handling capacity of the PCI bus where the data must pass twice before it gets streamed (once from the disk drive to the host, then across the bus again to the NIC).

We cannot find one disk drive that stores that much data, so we have to come up with a scheme to divide the storage across at least several drives. We also know that our clients will be really upset if their show dies midstream, so we have to create for some type of backup plan to account for possible disk failures. Solutions to these problems are often handled with RAID systems and with sophisticated load-balancing software.

If a RAID 0 solution is used, where two identical drives contain mirror images of the same data, how do we access the data in the most efficient manner? Each disk controller typically contains proprietary control software that attempts to optimize the data flow from multiple disk drives. This is a very complex problem, and there is no universal “best” solution. If only two files are being accessed, we can read one file from disk one while we seek the correct track to read the next file from disk two. Now we add a request for file three. Is it more efficient to get it from disk one or from disk two? Maybe it would be best to interweave sectors from each of the two disks, but remember that the slowest operation on a disk drive is the track seek time.

We can help tune our system by some intelligent management of file placement. We might load popular files on all of the disks in our system, but only load the less actively viewed content on two different disk drives. That approach assists in load balancing and reduces the total storage required for necessary redundancy.

The more we know about how our disks and disk controllers operate, the better we can tune our system for optimum performance. Here again, we do not need to know exactly how our storage and retrieval subsystems works, but the more we know, the better off we are.

The final link between your system and the Internet is provided by your network cards. Try to engineer your system so that multimedia content has the shortest possible path to the Internet. Be conscious of the total bandwidth limitations in your network and in each NIC, and remember that adding multiple NICs to a single computer may not significantly improve your throughput due to bus speed limitations.

Theoretical maximum throughputs listed in specifications are rarely achieved in the real world. When trying to estimate the amount of hardware you need in your system, good engineers who know how to find and benchmark bottlenecks are worth their weight in gold. Before you purchase a huge system, build a smaller prototype and spend a little time finding out where the bottlenecks lie. The better you profile your system, the more effective you will be in tuning it to handle your cost requirements.