HEVC video standard finished; high-end improvements coming

An array of companies have finished work on video compression technology called HEVC or H.265 that promises better video to start with and that paves the way for higher-end extensions next year, they announced today.

The High Efficiency Video Codec supports 4K "UltraHD" video -- and perhaps 8K as well if the video industry can convince buyers that so many pixels are worthwhile. Perhaps more important, given how many people watch video online these days, it doubles video quality for a given network data capacity.

HEVC has the potential to spread very widely indeed. It's the successor to the Advanced Video Codec (AVC), aka H.264, used in videocameras, TV, Blu-ray discs, online video streaming, and more. Codecs are technology used to encode and decode video or audio for more efficient transmission and storage.

To begin with, HEVC comes in two flavors, the "main profile" for conventional video, which supports 8-bit color data, and the "main 10 profile" with 10-bit video for higher-end uses. On top of that, the standard supports a still-image format.

HEVC specification drafts have described its use up to a resolution of 7,680x4,320 pixels. That 33-megapixel image is hardly a near-term need, but the video and cinema industries are edging toward 4K video resolutions with images of more than 8 megapixels. HEVC's better use of bandwidth is designed to enable higher resolutions without crippling networks and overstuffing storage systems.

More HEVC work is afoot at the Moving Picture Experts Group (MPEG), which represents two standards groups, the International Organization for Standardization (ISO) and the International Telecommunications Union (ITU), that produced HEVC/H.265 and AVC/H.264.

New video uses
First are HEVC extensions that let the standard store more color information by using in the full-color 4:4:4 format and the horizontally subsampled 4:2:2 format, which preserves more color information than the 4:2:0 format used in the first incarnation. These extensions will also support color data "up to 12 bits and beyond," the ISO said.

Second, another range of HEVC extensions will enable 3D and multiview video.

The color, 3D, and multiview extensions will be done in January 2014, the ISO said.

One potentially unpleasant aspect of the technology, though, is that like H.264, it's encumbered by patents that potentially mean royalty payments for those using the technology. Such payments are common for devices such as cameras and computers, but Web standards groups shun patent payment tollbooths on basic technology such as putting video on the Web. That's why Google released its VP8 codec royalty-free as part of the WebM project and why, presumably, it's working on VP9.

In contrast, the ISO patent policy, which is shared with the ITU, doesn't object to patents. Instead, it requires organizations with patents that bear on standards to release them for royalty-free use or license them "on a non-discriminatory basis on reasonable terms and conditions." That policy also is called RAND, short for reasonable and nondiscriminatory, which means for example that one company can't refuse to license the technology to a rival.

According to the ISO's list of patent declarations, companies that have said they have patents that bear on HEVC (known formally as ISO/IEC 23008-2 at the ISO) include Microsoft, Intel, Qualcomm, Research In Motion, Huawei, Thomson Licensing, and NTT DoCoMo.

It's not clear that VP8 and VP9 will be able to completely sidestep patent constraints. The group that licenses pools of patents for MPEG standards, called MPEG LA, has said 12 organizations reported they own patents essential to VP8 use, but so far MPEG LA hasn't offered a license to the VP8 patent pool despite looking into the matter for more than two and a half years.

MPEG LA plans to license a pool of HEVC-related patents. It issued a call for patents essential to HEVC in June 2012. "MPEG LA is pleased to assist in facilitating a convenient, independently administered, one-stop patent licensing alternative to assist users with implementation of their technology choices and invites all patent holders to participate," it said at the time.

More than two dozen patent owners have responded so far, and they're planning to hash out details next month. "In February, we will be holding a third meeting with essential HEVC patent owners to discuss a patent pool license for HEVC. There are currently 25 essential patent owners participating," MPEG LA said in a statement today.

MPEG LA declined to comment on any VP8 patent developments.

Google's VP9
Google started looking into VP9 seriously in the third quarter of 2011, according to a Google presentation about "VP-Next" (PDF), aka Next-Generation Open Video (PDF), or NGOV. Google is acutely aware of its competition. One VP9 goal is to "further reduce bitrates by at least 10 to 20 percent to move ahead of HEVC," Adrian Grange of the Chrome Media Group at Google, said in a presentation.

And as with HEVC, support for higher resolution is central. "HD is the new sweet spot," Grange said.

To get there, Google isn't starting from scratch. "It is an evolutionary development from VP8, not a completely new paradigm," Google said in another presentation.

According to another presentation about NGOV, a top priority is halving the network bandwidth requirement at a given image quality. Another priority is doing so with a video decoder that's only 40 percent more complex. Higher complexity for decoding translates heavier processing demands and therefore power usage, major constraints in the era of smartphones and tablets.

Although Google is leading the VP9 work, it's not alone. One element of the technology, called asymmetric discrete sine transform (PDF), is being developed as part of a collaboration with researchers at the University of California at Santa Barbara.

Updated at 12:53 p.m. PT with comment from MPEG LA.