Streaming MPEG-4 with Linux

Editors' Note: This article has been
updated since its original posting.Seven years ago, when I was still a clueless kid, I played my
first video clip on the Internet using a 56k connection. It was in
Real Media format and contained a video track in 11Kbps and a mono
audio track. All the buffering drove me crazy. Today, with all
sorts of high-speed lines, you seldom come across network
congestion during playback of a streamed video clip. This new speed
has enabled rapid development of multimedia on the Internet. One of
them is video streaming.What is video streaming? Traditionally, we must download the
whole video file in order to play it. With streaming, video data
sent to the streaming client is decoded and played immediately, as
it is received. Implementations such as video conferencing,
surveillance systems and video on demand (VOD) all are based on
video streaming technology.When people talk about video streaming, the first word that
comes to mind is "expensive". Even if you are only converting
existing media content, the software required for conversion might
not be affordable for normal home users. One of the major reasons
to use Linux for video streaming is it is inexpensive. Most Linux
video encoding and streaming tools are free, and they are easy to
set up. In less than an hour, one could have a streaming server
serving a dozen of MPEG-4 clips. MPEG-4 allows encoding to happen
at different bit rates and resolutions, while these are limited in
some other codecs. The utmost advantage of using MPEG-4, of course,
is it offers a standard format that is becoming more popular.
Unlike proprietary formats, MPEG-4 is an open standard, so adding
MPEG-4 support is easy. A few MPEG-4 audio and video players
already are on the market, and more of them will be released in the
future.About the SetupIf you are streaming MPEG-4 clips, the setup is simple. You
need only an MPEG-4 streaming server with a fast connection. A DSL
line would be enough if you are serving only a few buddies. If you
want to implement video conferencing or surveillance systems,
though, you need a live MPEG-4 encoder and a compatible video
capturing device under Linux. Of course, boards are on the market
that encode video to MPEG-4 on-fly, but they are generally quite
expensive for home users. Under Linux, the MPEG4IP suite, the
FFmpeg multimedia system and Apple's Darwin Streaming Server offer
an inexpensive way to stream and create MPEG-4 content.Compiling and Installing MPEG4IPThe MPEG4IP project began in summer 2000 by David Mackie, who
works on multimedia streaming for Cisco Systems. He is no longer
active on the project and Bill May is now the main developer. The
project is licensed under Mozilla Public License 1.1, but the
codecs used in MPEG4IP are subject to patent royalties depending on
how they are used. You should read the COPYING file in the package
before using them. Licensing fees also are associated with MPEG-4,
including a per-stream charge. People who plan to go commercial
should check it out first.In the MPEG4IP box are handy tools that can help you encode
and play MPEG-4 clips. You may get your copy from
mpeg4ip.sourceforge.net/
or from its CVS on SourceForge. While I was writing this article,
0.9.7 was the latest stable release. If you would like to try out
new features, get release 0.9.7.7 from the CVS repository. You also
need the LAME
package if you are compiling mp4live (enabled by default),
the live MPEG-4 encoder for live broadcasting. Extract the package
and begin compiling:

tar zxvf mpeg4ip-0.9.7.tar.gz
./bootstrap
make
make install

To compile without mp4live (and LAME) add
--disable-mp4live after
./bootstrap. MPEG4IP uses a modified version of
the SDL library, so you do not need to bother installing it. You do
need the GTK+ library, though, if you want to compile the GUI
player.Compiling and Installing FFmpegThe FFmpeg Multimedia System itself is a complete solution
for creating and streaming MPEG-4. When this article was written,
the stable release was 0.4.6. The streaming server supports only
HTTP streaming at this stage, so we take advantage of FFmpeg, the
encoder, at the moment. Get the source from SourceForge and begin
compilation:

tar zxvf ffmpeg-0.4.6.tar.gz
./configure
make
make install

This configuration is sufficient for encoding MPEG-4 video.
If you want MP3 and OggVorbis encoding support, add
--enable-mp3lame and
--enable-vorbis and install their respective
libraries beforehand.Extracting Content from the SourceHere at my school video clips are stored in different
formats. This is likely to be the case in most situations--you do
not need to convert only one format to MPEG-4 but many formats.
Some of my fellow video team members store the clips in MPEG-1 and
2, some of them in DivX or XviD AVI and some use DV AVI if they do
not have time to encode. Here, we discuss how we encode these
formats to MPEG-4, as they are likely to be the most common formats
we might come across.No single tool with a single button can do all the encoding
work well for you. You need to encode the video and audio
separately, so the first thing to do is split up the video and
audio. Under Linux, FFmpeg is a Swiss knife for video processing.
It encodes and decodes various video formats and supports some
basic video processing, such as de-interlacing. To extract the
audio track from the source, type:

ffmpeg -i <input_file> -vn <output_file>

The command can be applied to both DV/DivX/XviD AVI and MPEG
files. The vn switch disables video in the output file. The
resulting file would be a raw PCM audio file.Encoding the AudioTo generate an MPEG-4 complaint clip, you need MPEG-4 AAC
audio. The MPEG4IP suite includes a redistributed AAC encoder
called FAAC. At the of this writing, FAAD2 is available on
SourceForge and uses the version 1.1 library for encoding, which is
a bit faster. If you are interested, you can grab the source from
faac.sourceforge.net.
A copy of the FAAC binary should have been installed properly when
you were installing the MPEG4IP suite.To encode the raw PCM audio track split by FFmpeg,
type:

faac -m4 -b64 -r48000 -pLC <input_file> <output_file>

The -m flag specifies the AAC MPEG version. We are creating
MPEG-4 AAC so we use -m4. The -b flag specifies the bit rate for
the encoded output. A value of 32 or 64 produces satisfactory
quality and is an economical size for streaming. The -r flag
specifies the sampling rate of the input file; otherwise, FAAC
would not read the file. Instead, it would produce an error
notifying the user that the file format is not supported. The last
-p flag, which specifies the profile, is a bit more complicated.
The Main profile includes all the coding tools (an MPEG term
understood as "features"). The LC profile, which stands for low
complexity, reduces decoding complexity on the client side (for
slow devices or machines). The LTP profile, which stands for long
term prediction, provides significant improvement in quality but
also increases decoding complexity. It is best used for audio
tracks with clear pitch. I use LC most of the time for my school's
MPEG-4 clips.Encoding the VideoMPEG4IP suite comes with two encoders, mp4encode and xvidenc.
mp4encode encodes an AVI file to an MPEG-4 video clip with its
built-in OpenDivX encoder or ISO MPEG-4 encoder, if specified.
xvidenc, on the other hand, encodes raw video file with the XviD
encoder. If you encode with these utilities, the input AVI file
must contain a video track in raw YV12 (YUV12) format, and the
audio track must be in raw PCM format. In this article, instead, I
show how we can use FFmpeg to simplify the video encoding process.
Because FFmpeg decodes DV, MPEG-1 and 2, DivX and XviD, we can skip
the step of preparing YV12 raw for mp4encode or xvidenc. The
command for encoding from these formats to ISO MPEG-4 video is the
same:

ffmpeg -i <input_file> -an -b 300 -vcodec mpeg4 <output_file>

FFmpeg then encodes the input file without audio (the -an
flag) at 300 kilobits/second, using the ISO MPEG-4 encoder to an
AVI video file containing a single MPEG-4 video track. If you want
to perform inverse telecine (IVTC) for NTSC source under Linux, you
would need transcode, another nice program suite that provides
useful video processing functionalities. It also is possible to
encode the video in 2-pass mode to get the best result.1st pass:

ffmpeg -i <input_file> -an -pass 1 -passlogfile <log_file>
-qscale 2 -vcodec mpeg4 <output_file>

2nd pass:

ffmpeg -i <input_file> -an -pass 2 -passlogfile <log_file>
-b 300 -vcodec mpeg4 <output_file>

The input file, the output file and the log file in both
passes must be the same. Also, you should specify your desired bit
rate during the second pass instead of the first.Multiplexing: Combining the Video and
AudioAfter encoding the audio and video, it is time to combine
them to create a real MPEG-4 clip. The MPEG4IP suite provides a
nice tool called mp4creator for this purpose. It can add or remove
tracks in an MPEG-4 file. But, let's first create the MPEG-4 clip
by adding a video track:

mp4creator -c myvideo.avi -hint mytest.mp4

The -hint flag tells mp4creator to add an additional hint
track for the video track. Hinting is required if you are streaming
a clip with Darwin Streaming Server. If the mytest.mp4 file does
not exist already, mp4creator creates it for you. To add an audio
track, enter:

mp4creator -c myaudio.aac -hint -interleave mytest.mp4

Both commands perform a similar action. The second command
introduced the -interleave flag, which creates an interleaved RTP
payload format for the audio hint track. If you are using MP3 for
your audio track (possibly not), ignore this flag as it supports
only AAC. Finally, optimize the clip:

mp4creator -optimize mytest.mp4

This command optimizes the file layout, hence improving its
speed during streaming. The rearranged MPEG-4 file contains control
information at the beginning of the file, thereby enabling HTTP
streaming of the file. To see a list of tracks in an MPEG-4 file,
use:

mp4creator -list mytest.mp4

If you have a successful MPEG-4 clip, the list should look
something like this (a clip currently being streamed at my
school):TrackTypeInfo1videoMPEG-4 Simple @ L3, 30.030 secs, 299 kbps, 352x240 @
29.97 fps2hintPayload MP4V-ES for track 15audioMPEG-4, 29.973 secs, 65 kbps, 48000 Hz6hintPayload mpeg4-generic for track 57odObject Descriptors8sceneBIFSBefore putting the clip up to the streaming server, you
should test it. The MPEG4IP suite comes with mp4player and
gmp4player for playing locally stored or streamed MPEG-4 files. You
should use gmp4player, as the console player provides little
control of video playback. If everything looks fine, upload the
MPEG-4 file to your streaming server's media directory. Hurray!
Your video is ready to be streamed over the net!Setting Up the Darwin Streaming ServerStarting from its fourth version, the Darwin Streaming Server
does not stream only QuickTime movies but also MPEG-4 ones. The
benefit is Darwin Streaming Server is completely free for
non-commercial use, and it runs under Linux smoothly with a small
load.To get a copy of the streaming server, go to the
Darwin
project page. You are required to have a valid APSL (Apple
Public Source License) registration in order to download the binary
or the source. Follow the instructions on the web site for details
on how to obtain an APSL registration. Get the server binary for
Linux (the web site shows Red Hat Linux 7.x), and proxy if you need
to serve clients behind firewall. Uncompress the package to a local
directory and begin the installation:

tar zxvf DarwinStreamingSrvr4.1.2-Linux.tar.gz
cd DarwinStreamingSrvr4.1.2-Linux
./Install

Make sure you have Perl installed for the web-based
administration interface. Install also Net::SSLeay and OpenSSL with
a valid certificate if you want SSL support. The installation
script automatically copies the files to suitable places and
displays where they are copied to. It then asks you for an
administrative user name and password, which is used later to log
in to the web-based administration interface, where you can access
most of the server's features.Next, launch the administration interface using the Perl
script streamingadminserver.pl. It launches Darwin Streaming Server
if it is not already running. Point your favourite browser to the
video server's port 1220, 1240 if you enabled SSL support. Log in
with the account you set during the installation. As this is your
first time logging in, you are presented with the Setup Assistant.
It guides you through the basic settings with detailed descriptions
for each item. You need a valid SSL certificate to enable SSL
support, unless you modify the script. After completing the Setup
Assistant, you will have a running Darwin Streaming Server ready to
flood the line with MPEG-4 videos.Streaming the ClipNow you have an MPEG-4 clip on hand and ready to stream. One
option is to stream it over HTTP; simply put the clip in a publicly
accessible directory on your web server and point your player to
the clip's URL. If you want to stream over RTSP, direct your player
to rtsp://<server_name>/<clip_name>.
(For example, rtsp://video.wyk.edu.hk/sample_100kbit.mp4 is a
sample clip provided by Darwin Streaming Server.) Streaming over
RTSP allows random seeking without waiting for the clip to be
downloaded progressively.If you want to create a playlist or make a live broadcast,
you would need to create an SDP (session description protocol)
file. To create one, log into your video server's administration
interface and then go to the Playlists section. Create a new movie
playlist and drag the accompanying media to the playlist. Give the
playlist a name and a mount point, which is an SDP file. Save the
list, and do not forget to start the list afterwards. Instead of
pointing your player to the MPEG-4 file, supply the new URL,
rtsp://<server_name>/<sdp_name> to
receive the broadcast. Congratulate yourself! You have successfully
created a streaming MPEG-4 system and your first MPEG-4 video
clip.If you want to stream live video from your capturing device,
use mp4live which comes with MPEG4IP. You need a
Video4Linux-compatible capturing device, Video4Linux and I2C
support in the kernel for live broadcasting. The user interface of
mp4live is quite self-explanatory. You can record the video to an
MPEG-4 file or stream it to clients with unicasting or
multicasting. Click the Generate button to generate an SDP file,
and copy it to your video server's media directory.If you are within a local LAN or enterprise, multicasting
would be the best method. In this method (on both QuickTime, Real
and gmp4player), you pass the SDP file directly to the program. Set
up a web server on the broadcasting machine, then point the player
to http://<server machine>/capture.sdp,
where capture.sdp is the aforementioned generated SDP file. With
unicasting, you would use rtsp://<server
machine>/capture.sdp instead. The server machine must
be running Darwin Streaming Server, and the result is the streams
are unicasted and repeated. This should be used only where
multicast is not available.A Few Notes on CompatibilityPeople who try encoding the video may complain about the
video quality. The MPEG-4 codecs used in Windows and Linux are the
same, so what is the difference? The key component that determines
the ultimate quality of the video is the rate controller. The fact
is rate controllers in Linux still need some work. I recommend
using the famous VirtualDub (free and licensed under the GPL) if
you can, because it is more powerful in video encoding and
processing and significantly enhances video quality.Most people prefer using QuickTime, RealPlayer or Windows
Media Player to play streaming video. Starting with version 6,
QuickTime has built-in support for MPEG-4, so what you need to do
is upgrade your QuickTime Player. For RealPlayer and Windows Media
Player, you must install the EnvivioTV plug-in, which can be freely
downloaded and used. You may get your copy at
www.envivio.com.Many discussions of QuickTime 6's MPEG-4 support can be found
on the Net. In short, a bunch of users cannot play streamed MPEG-4
with QuickTime 6, and sometimes they receive strange error
messages. The problem lies in the implementation of MPEG-4 in
QuickTime 6. It is known that MPEG-4 video encoded with GMC (global
motion compensation), quarterpixels (qpel) and chroma motion cannot
be played with QuickTime 6. QuickTime 6 also uses an MPEG
quantizer, which improves cripsness and is better than H.263 for
preserving quality. Extensive tests in my school show that MPEG-4
AAC audio with bit rates under 64 kilobits/second generate strange
error messages in QuickTime 6. The players provided by MPEG4IP,
however, do not have the same problems with QuickTime 6.QuickTime 6:

• Friendly user interface.
• Popular.
• Integrates with browsers.
• Plays only some MPEG-4 files.

MPEG4IP mp4player:

• The Windows version player lacks a good user
  interface.
• Not very well known to the public.
• Manual input of URL is required.
• Plays most MPEG-4 files.

ConclusionBefore encoding an MPEG-4 streaming video clip, you must
consider a few things. Do you want to keep the maximum quality or
compatibility? Who is the intended audience for the clip? If you
are showing the clip to a large group of people or the general
public, it is best to encode the clip so it may be played by common
video players. If you want to preserve quality, you can use an MPEG
quantizer during encoding or the MPEG-4 encoding tools discussed
above. You should provide download of the MPEG4IP player so users
are able to play them properly. The MPEG4IP suite includes Visual
Studio workspace files, which let you compile Windows version of
the player, wmp4player. Provide it as well, as most of your clients
are probably running Windows.MPEG-4 will be the trend in video streaming in the coming
years. The MPEG-4 implementations we have today are still not the
full MPEG-4 toolbox. For example, integration of natural and
synthetic content and broadcast-grade synchronization need to be
advanced. With the technology becoming more mature, MPEG-4 will
soon become the standard for internet video.AcknowledgementsI would like to thank Bill May, the main developer for the
MPEG4IP project, for his comments and suggestions on this
article.ResourcesMPEG4IP
ProjectFFmpeg Multimedia
SystemApple
Darwin Streaming Servertranscode

email: dszeto@wyk.edu.hk