Version 1.0
First Published: May 7, 2004
Last Updated: May 11, 2004
Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.
The field of digital video has been maturing, but in some arenas it is still a zoo filled with incompatible, competing standards.
The arenas exist because of technological limitation in delivery mechanism particularly in the bandwidth requirement. The bandwidth determines the maximum rate of data delivery, and in turn is limited by the transport media. The major arenas are: low, medium, and high bandwidth. There is no commonly-agreed boundary between arenas, although many would agree on the following:
+--------------+-----------------+ | Bandwidth | Rate (Mbps) | +--------------+-----------------+ | Low | 0 - 0.5 | +--------------+-----------------+ | Medium | 0.5 - 1.5 | +--------------+-----------------+ | High | > 1.5 | +--------------+-----------------+
By the above classifications, some media or delivery mechanisms easily fall into a certain arena, while some other falls into multiple arenas. For example, the satellite broadcasting usually falls under the medium arena, while the video playing (or streaming) over the LAN may falls under either medium or high, depending on the bandwidth capacity of the LAN.
The industry seems to have settled on some standards for the medium and high bandwidth arenas. For example, the MPEG-2 standard has been adopted in many circles: satellite broadcasting, SVCD, and DVD.
However, in the low bandwidth arenas, specifically for delivery through the Internet, there is still no clear winner. This article describes the popular alternatives for the low bandwidth arenas.
In the low bandwidth arenas, deciding on the "best" format is far from obvious. Common solution to this situation has been to publish the same video clip in multiple formats.
Never the less, a video publisher needs to select a primary format. The trade-offs to consider in selecting the format involve the following key parameters:
Video format determines the layout of the video and audio streams in the file/transmission. The format determines what kind of operations can be performed to the streams for the purpose of playing it. Some formats allow seeking (jumping to a random time index), some allow streaming (playing of the video without having the complete video), and some even allow embedding of subtitles.
Each format usually accept only certain codecs. For example, the MPG format can only be used to stored MPEG1-encoded video stream and MP2/MP1-encoded audio.
Following is a list of popular formats:
MPEG formats is a family of formats specified by the MPEG (Moving Picture Experts Group) organisation.
A common characteristics shared by members of this format is that the video is stored in independent groups giving it resistance to damage. If a particular group is damaged/not readable/missing, the player can simply skip to the next good group. This property also makes this format good for streaming.
However, MPEG formats can accept only certain codecs, and many of these codecs do not perform well in low-bandwidth situation. Thus, while formats of this family can be used in streaming application, streaming this format across the Internet may prove to be a challenge. An exception to this is the MP4 format, which is designed to be streamed (more on this below).
The MPG format is the most basic format. It accepts MPEG-1 video, and MP2 (MPEG-1 layer 2) or MP1 (MPEG-1 layer 1) audio. It is the first popular cross-platform format. Almost all video players in any platform are able to play the MPG file.
The DAT format is used in VCD and SVCD. In VCD, MPEG-1 video and MP2/MP1 audio is used. In SVCD, MPEG-2 video and MP2 audio.
The VOB format is used in DVD. It accepts MPEG-2 video, and AC3/DTS/MP2/PCM audio.
The MP4 format is the newest format and has a good future outlook. It allows multiple video and audio streams to be included. It also allows subtitles and pictures.
A content publisher can publish the same video under different resolutions by putting them as different video streams. As a result, end-user can choose how much bandwidth (lower resolution == lower bandwidth) to use in viewing the video.
The format can contain only MPEG-encoded streams (some exception for the audio stream). It supports MPEG-4, MPEG-2 and MPEG-1 video codecs, and MPEG-1 (MP1, MP2, MP3) audio codecs.
However, this format is relatively new and popular only within the OSX and Linux community. It has yet to enjoy popularity nor widespread support in the Windows community.
Microsoft AVI (Audio Video Interleaved) is a multi-purpose format. It accepts various codecs (however the audio codec must be a CBR (constant bit-rate, detailed in the Codec section) codec). Currently, it is most commonly used as the container of DIVX, XVID video, and MP3 (MPEG-1 layer 3) audio.
AVI is popular in Windows. Although Microsoft is urging content developers to move to the new ASF format. Nevertheless, new codec developers still favours it because AVI can reach a wide audience (Windows user).
This format is not suitable for streaming because it requires the information at the headers to interpret the following video and audio streams. If the header is missing, then the rest of the stream is useless. If a part of the stream are missing, then the remaining parts are useless because they would no longer match the information described in the header.
Another disadvantage is the multitudes of codecs that it accepts. A user who wish to play an AVI file has to have both an AVI player and also the appropriate codec to interpret the video and audio streams within it. AVI players are pretty much widely available for most of the popular platforms: Windows, Linux, and OSX. However, some codecs are released only for the Windows platform, rendering videos encoded with those codecs unplayable in other platforms. This differs from the MPEG players which is almost always guaranteed to be able to interpret the video and audio streams within the MPG file.
AVI supports two different interleaving model:
ASF stands for Active Streaming Format. As its name implies, it is targeted towards streaming usage. It represents Microsoft's effort to upgrade their old AVI format. There are two non-compatible versions of ASF: ASF v1.0 and ASF v2.0. v2.0 is almost never seen in the Internet.
The content of an ASF file can be encoded using any codec. The most common encoding used in ASF format is the WMV (Window Media Video) and WMA (Windows Media Audio) codecs. Both codecs are Micorsoft's properietary codec, unavailable to non-Microsoft platform.
Apple's QuickTime is a format that has been around for a long time. It can contain various video/audio codecs. Nevertheless, its popularity is still pretty much restricted to within the Apple (MacOS, OSX) community since the usual media players in Windows do not understand this format.
QT is suitable for streaming usage and can contain subtitles.
Real's RealMedia format is a format for streaming usage. It contains only the proprietary RealVideo and RealAudio encoded streams. It supports subtitles.
A new format that is freely available and freely re-implementable (the equivalent of open source in data format). It is not patent-free (it is pracically impossible for any technology in this field to be patent-free), but it is patent-clear.
This format is similar in capability to the QT format. It is not popular yet, and mentioned here only because it is the only 'free' format available.
Central to digital video technology is the codec. Codec describes the method to encode (for transmission, storage purposes) and decode (for playback, editing, etc.) information, which in this review means video and audio.
There have been many codecs, both for video and audio. Some of them are no longer used in new media, for example the Cinepak codec. New codecs are still being created to achieve a higher quality/size ratio.
Most codecs perform lossy compression which means it removes certain information from the source. What and how much to remove depends on the codec developers' judgement. For example, in MP3, lower and high frequency sound are de-emphasised (partial removal). The quality of a given codec, beside a factor of the algorithm used, is also influenced by the bit-rate setting. The higher the bit-rate, the less information needs to be removed, but the more storage it consumes.
Newer codecs tend to be a VBR codec. A VBR (variable bit-rate) codec uses less storage when recording a relatively static scene such as found in "talking head" video where there is a person talking with little body movement and relatively unchanging background (interview video).
Unless specified, the following codecs can operate both in VBR and CBR (constant bit-rate) mode.
The main players in the codecs field is the MPEG codecs. There are three MPEG codec specifications, each one includes video and audio codecs specifications: MPEG-1, MPEG-2, and MPEG-4. However, except for MPEG-1 layer 2 and 3 (MP2 and MP3), the audio codecs of these specifications are relatively unpopular. So, when people are speaking about MPEG-4, unless the context dictates otherwise, they are referring to the video codec specification of MPEG-4.
MPEG-1 is the oldest, most widely supported codec. MPEG-1 is a CBR codec. MPEG-2 is mostly found in SVCD and DVD. MPEG-4 is a new, popular codec considered by many to be the 'in' codec right now.
Many newer codecs are MPEG-4 compliant codecs. Compliant codecs means that the encoded information produced by the codecs can be decoded by any compliant decoder.
The most popular MPEG-4 compliant codec is DivX, followed by XviD. XviD started when the development of DivX became closed-source. These difference, though, affect only the content producer as both these codecs produce MPEG-4 compliant files. There are DVD players that can play DivX files. Although it is not often advertised, since DivX is MPEG-4 and XviD is also MPEG-4, these players can also play XviD files.
Non-MPEG-4 codecs:
WMV, RealVideo, and Sorenson. WMV was, at one point in the past, a MPEG-4 compliant codec. Sorenson was a popular codec in Apple, but its position was eroded by MPEG-4.
The default player in XP, Windows Media Player (WMP), is able to play AVI, ASF, and MPG formats. It does not come with popular codecs, like DivX or XviD. However, people who watch videos (such as trailers) downloaded from the Internet, usually have installed both codecs. Actually, even though it is not necessary to install both codecs because a DivX file can be decoded with XviD decoder (MPEG-4 compliance) and vice versa, there is no easy way to force WMP to do so.
RealPlayer (RP) is the second most popular player on this platform. Despite being second, it really lags behind WMP. I think this is mainly because of the past fiasco that Real did. It used to be that RP installation process also installed a bunch of annoying features, thus alienating its potential user base. Only recently, with RP 10, RP installation becomes well-behaved.
QT (QT) is a distant third. MP4 format is supported only in recent QT (QT 6 or newer).
The main player is QT, which understands QT and MP4 formats. It has a built-in MPEG-4 codecs. As a result, it can play many DivX and/or XviD files out-of-the-box. It can't play all because MPEG-4 are not DivX or XviD (although the reverse is true).
There is WMP for OSX available. The main player for playing ASF format. RP is also available.
This platform boasts the most flexible player ever, MPlayer, which is capable of playing almost any video file in existence. Although MPlayer has been ported to both OSX and Windows, the ports are still relatively unknown/unpopular in those platforms.
MP4, RealMedia, ASF and QT formats are suitable for streaming and playable in all platforms. Thus, I limit my choices to these formats.
These formats require purchase of the producer (encoder) and the streaming server, in addition to any licensing cost.
Although, at least, Real provide free version of their producers and streamer, some of their functionalities are either missing or limited. This is especially true with regards to their free streamer which has been limited to serve only 5 concurrent user.
This format is being phased out by Apple in favour of MP4.
There are free, open-source MP4 producer and streamer. These products are not crippled and do not have licensing issue.
Additionally, Apple has made their Darwin Streaming Server, which is also capable of streaming MP4 format, available for free for non-commercial use.
Apple Quicktime (Media Player)
http://www.apple.com/quicktime/
Apple Darwin (Streaming Server)
http://developer.apple.com/darwin/
FFMPEG (Content Producer, Streaming Server)
http://ffmpeg.sourceforge.net/
MPlayer Documentation (Media Player, Content Producer)
http://www.mplayerhq.hu/DOCS/HTML/en/index.html
MPlayer for OSX
http://mplayerosx.sourceforge.net/
MPEG4IP (Content Producer (specialising in MPEG4 creation)
http://mpeg4ip.sourceforge.net/
Streaming MPEG-4 with Linux
Donald Szeto
2003 March 13
http://www.linuxjournal.com/article.php?sid=6720&mode=thread&order=0
transcode (Content Producer)
http://www.theorie.physik.uni-goettingen.de/%7Eostreich/transcode/
RealNetworks, Inc. (Content Producer, Streaming Server)
http://www.realnetworks.com/audience/index.html
Wikipedia
http://en.wikipedia.org/wiki/{AVI,MPEG-1,MPEG-2,MPEG-4,ASF,QT}
Windows Media Encoder (Content Producer)
http://www.microsoft.com/windows/windowsmedia/9series/encoder/default.aspx
Windows Media Server (Streaming Server)
http://www.microsoft.com/windows/windowsmedia/9series/server.aspx
Windows Multimedia Formats
W3 Schools
http://www.w3schools.com/media/media_windowsformats.asp
Xiph.Org
http://www.xiph.org/