Category Archives: Knowledge Base

Read our in-depth articles explaining the business and technology aspects of streaming videos.

A Business & Entertainment History of Streaming Media

With video increasingly becoming the medium of communication over the internet, it is a sobering thought that many of the technologies now being used seamlessly were tried out, at enormous usage costs, in the early-to-mid 90s. Some of the first uses of internet streaming media in this period were:

  • The first major live sports webcast on the internet on September 5, 1995 – a Major League Baseball game between the Seattle Mariners and the New York Yankees.
  • A Rolling Stone Live Concert was broadcast live from Cotton Bowl in Dallas on 18 November 1994. Knowing the broadcast channel to be live and open to anyone, Severe Tire Damage, a band created by employees at Xerox PARC at Palo Alto, CA broadcast an impromptu performance live before and after the Rolling Stone Broadcast. The broadcast was still not accessible to a wide audience – the workstation required for receiving the broadcast and the telephone connection (which would have then cost $20,000) meant that only about 200 computers were logged in for the broadcast.
  • On November 10, 1995, Seattle Symphony [1] performed the first symphonic concert live with guest musicians Slash (Guns ‘N’ Roses, Velvet Revolver), Matt Cameron (Soundgarden, Pearl Jam) and Barrett Martin (Screaming Trees). To access this online event, users would have required a 28.8 modem and a “Cyberian Rhapsody” CyberTix. The visual element for the stream was created by four multimedia teams – EPG Media, XSI Media, the Center for MultiMedia, and a research group at the University of Washington. The imagery was provided by Corbis Media, privately held by Microsoft Chairman Bill Gates.
Streaming Media firsts

Article from Billboard, November 95

What follows is the story of the evolution of Streaming Media, and the struggle for dominance between RealNetworks and Microsoft in the Dot Com era, and post that the emergence of Flash and Apple in the streaming media space.

RealNetworks and Microsoft – Streaming Media in the Dot Com Bubble

Microsoft’s involvement in streaming media was initiated in 1993, when they sent Windows development kit videos to developers.

Progressive Networks launched RealAudio 1.0 in April 1995. With this they launched the first streaming media application. VDOnet, Xing and Microsoft were three other companies that were developing platforms for streaming technologies. Xing’s platform – StreamWorks – supported both audio and video and was launched shortly after the launch of RealAudio 1.0.

RealPlayer Streaming Media

In 1997, RealNetworks (rebranded from Progressive Networks) agreed a broadcast network deal with MCI Communications (who were one of the most important telecommunications company in the 90s).[2] The goal for this deal was for MCI’s internet infrastructure to provide support for upto 50,000 simultaneous streams. This was the first major effort towards creating a content delivery network (CDN) for reaching out to mass media consumers.

Soon, the increasing significance of streaming media and the related support services of encoding, delivery and hardware led to specialized technologies in each field. Both Sun and Cisco were early entrants into the hardware market for streaming media. Also MP3 and MPEG file format licenses were very expensive, and the competition between Microsoft and RealNetworks was largely around cornering the standards for codecs – if the RealNetwork codec became standard then they would have had control over licensing, thereby being de facto leaders of the streaming media industry.

The Dot Com bubble burst in 2000-01, and the expected demand for streaming media was not achieved. RealNetworks started focusing on content, creating a content subscription service called RealOne. They also announced a three-year deal in 2001 with Major League Baseball to create subscription service for live audio webcasts. By 2003 RealNetworks had initiated its exit from the streaming media business, announcing a partnership with Sony, following that up with acquiring music and gaming companies over the years.

iPod and iPhone – Apple takes over

Microsoft, the incumbent leader in the streaming media space, largely disregarded the emergence of Apple in the field of streaming media, leaving it to the entertainment division to compete with the emerging iPod. Apple had released iTunes for Windows. Flash emerged as a streaming + encoding solution, and Apple followed up their entry with their HTTP Live Streaming (HLS) protocol. HLS is now being used by Apple Devices for adaptively streaming both live and on-demand files. Flash remains the incumbent video streaming technology, with the fragmentation in HTML5 making the shift to the newer technology not as popular as was previously expected.

References:

  1. “Seattle Symphony Goes Live”, Billboard, November 1995
  2. New York Times, 22 November 1994
  3. Dan Rayburn, StreamingMedia Magazine

What is Transcoding?

Digital storage of any media requires conversion from analog to a digital format. The initial digital format after media production is still a raw file, and for it to be stored and accessed across different devices it needs to be compressed in the particular digital format compatible across the device. This form of video compression for files to make them compatible with a target device is called encoding.

The terms encoding and transcoding are sometimes used interchangeably, inspite of the use-case difference between the two.

Generally speaking, encoding refers to the process of converting uncompressed data to a desired format. This is understood to be a lossy process. On the other hand, transcoding is the process of decoding a video file from one format to an uncompressed format, and then encoding the uncompressed data to the desired format. Video transcoding is commonly used when the video file is being moved from a source to a different destination, and when the two support different file formats.

One of the most important uses of video transcoding is in uploading video from one source – a desktop, to an online video hosting site, so that the format is supported by video hosting site.

Some other terminology in use regarding video transcoding and encoding are:

  • Transmuxing – Conversion to a different container format without changing the file itself
  • Transrating – Conversion to a different bitrate using the same file format

What is a Codec?

A video codec is any device/software that compresses a video file. A device/software that only compresses an analog file is called an encoder, whereas a device/software that only decompresses a compressed digital file to analog is a decoder. The term ‘codec’ comes out of the concatenation of the two terms encoding and decoding.

How do codecs work?

For any codec to work it needs to compress the frames. There are two types of frame compressions – inter-frame and intra-frame compression. In intra-frame compression, each frame is compressed independently of the adjacent frames. It is therefore essentially image-compression applied to video.

Inter-frame compression on the other hand identifies redundancies across frames to compress videos. This is includes any elements of the moving image that may be static – say a static background in a talking head video. Inter-frame compression is much more efficient than intra-frame compression, and so most codecs are optimized to identify redundancies across frames.

What are some of the most prominent codecs?

MPEG (Motion Picture Experts Group) is the most common family of video codecs. International Standards Organization (ISO), whose standards impact the computer and consumer electronics markets has had at the following list of codecs as its standards:

  • MPEG-1 in 1993
  • MPEG-2 in 1994
  • MPEG-4 in 1999
  • AVC/H.264 in 2002

H.264 is the most prevalent compression standard in use currently. For a period in the late 90s and early 2000s, the rivalry between RealNetworks and Microsoft was around creating their own proprietary formats as the standards for codecs. RealNetwork’s RealVideo, Microsoft’s Windows Media Video, ON2’s VP6 and Sorenson Video 3 were the dominant proprietary codecs. The H.264 codec was added to Apple’s QuickTime 7 in 2005, and in 2007 added H.264 support for Flash.

VP9 is the proprietary video compression format and codec developed by ON2 Technologies, which were acquired by Google in 2010. VP9 is available under new BSD License by Google with source code available as libvpx VP9 codec SDK. 

Theora is a free lossy video compression format, distributed without licensing fees. Theora is derived from VP3 codec, which has been released into the public domain. Ogg Video Container format uses Theora as its compression format.

How is a media file format distinct from a codec?

A file format is a container, inside which is the data that has been compressed by a video codec. A single file format may support multiple video codecs. For example the Audio Video Interleaved (AVI) file format contains data that compressed from any of a range of codecs.

What are some of the most prominent containers?

QuickTime File Format is a multimedia container format used by Apple’s multimedia framework.

MP4 is the most popular container format used for storing digital audio and video. Both QuickTime and MP4 file formats use the same MPEG-4 format.

FLV is the file container format used for video content using Adobe Flash Player. Flash video content can be embedded within SWF files.

WebM is the royalty-free container format, sponsored by Google. WebM uses the VP8 and VP9 codecs as compression formats.

Ogg container format is maintained by Xiph.Org foundation, and is not restricted by software patents used in H.264. Ogg is supported by the Wikipedia community.

Advanced Systems Format (ASF) is Microsoft’s proprietary video container format designed for streaming media.

 

Reference

“What is a Codec?” Jan Ozer, StreamingMedia.com

Watermarking to enforce DRM for Protected Streaming

Watermarking is the process of integrating a message within streaming media such as online video, mainly for copyright enforcement. The watermarking message may be made either visible or invisible. The message used in visual watermarking may be the name of the viewer, their IP address, or any details regarding the distribution channel through which the video may have reached the user. When the watermark is clearly visible, the viewer is aware of this copyright protection mechanism, and the watermark therefore acts as a deterrent to online video piracy. On the other hand an invisible watermark is much more difficult to embed, and does not act as an effective deterrent.

Essential Properties of Watermarks

The essential features on the basis of which a watermark can be evaluated are:

  • Effectiveness – That the watermark will be correctly detected, to ensure that source of video leakage is identified (which is the primary use of Watermarking)
  • Image fidelity – Ensuring that the quality of the image is not diminished as a result of the use of the watermark
  • Payload size – Size of the watermark
  • Robustness – To ensure that the watermark is able to withstand white noise, compression, printing and scanning, scaling and cropping

Implementation

Dynamic watermarking for streaming media is a means of copyright protection, and complements DRM mechanisms such as secure encryption and key-based authentication. Watermarking is done by overlaying a text or image over the video content.

Text

Any overlay text may be dynamic or static – the position of watermark on the screen may be moving throughout the length of the video, or it may remain static. The parameters given for an effective watermark implementation are:

  • Text
  • Text position (for static text)
  • Opacity of text – Increasing the transparency of text would improve the quality of the underlying video, and also make it more difficult to remove the watermark.
  • Color
  • Size and
  • For moving text, the interval of time after which position is changed

In a standard implementation, these parameters can be input using a JSON query. For more on adding watermark features check this blog.

Different Kind of Texts used

The different texts that may be used in the watermark can be:

  • Copyright owner
  • Recording the distributor
  • Recording the distribution chain
  • Identifying the content user

For watermarking for deterrence purposes, identifying the user (through a session ID for example) is the most effective method. Timestamp, user details like name, E-mail, phone number and user ID can be used as text, for effective tracking.

Image

Watermarks can also be made as images moving across the screen. The motion of the watermark can be used to identify the specific person/ groups through which the content has been leaked. This could be done by comparing the specific motion in the leaked copy with the motion on the server-side copies of users. The watermark motion can be matched to the specific user, or (in case of large number of users) to a narrower group of users.

Effective Deterrent to Screen Recording

Digital watermarking is a very effective deterrent to video recording via screen recording software. Digital watermarking can be used both covertly and explicitly. The difference is that whereas covertly embedding watermarks means that customers are unaware of the added security feature, for overlaid watermarking customers are aware that the feature is there. In this scenario watermarking would act as a more effective deterrent, whereas in the case of invisible watermark the primary purpose is enforcing copyright enforcement, which necessitates the arrangements for litigation. For this reason, the process of overlaid watermarking is actually much more transparent to the end viewer, and does its job of being a deterrent.

Vital for Independent Content Creators

The key idea behind DRM is to protect intellectual property rights of content creators. For small to medium scale content providers, losses due to copyright infringement can be significant enough to make the process of content creation wholly unprofitable. Lack of profit means that content producers have little incentive to create their original content. Watermarking complement DRM to ensure that these digital rights are not violated. Because the sharer can be traced, watermarking acts as an effective deterrent against illegal sharing.

The robustness quality of watermark ensures that it can only be removed by deteriorating quality too much, and that therefore the watermark will be present for any effective sharing of video.

Steganography

Steganogaphy is the process of hiding a message covertly inside another message. As mentioned earlier, steganography is effective only for purposes of copyright enforcement.

References

Digital Watermarking

Watermarking

For more in our Knowledge Base article category, check out our article on What is Transcoding?