Automatically upgrade your video content to a new and improved codec

Easy & Safe Codec Modernization with Beamr using Nvidia GPUs 

Following a decade where AVC/H.264 was the clear ruler of the video encoding world, the last years have seen many video coding options battling to conquer the video arena. For some insights on the race between modern coding standards you can check out our corresponding blog post.

Today we want to share how easy it can be to upgrade your content to a new and improved codec in a fast, fully automatic process which guarantees the visual quality of the content will not be harmed. This makes the switchover to newer encoders a smooth, easy and low cost process which can help accelerate the adoption of new standards such as HEVC and AV1. When this transformation is done using a combination of Beamr’s technology with the Nvidia NVENC encoder, using their recently released APIs, it becomes a particularly cutting-edge solution, enjoying the benefits of the leading solution in hardware AV1 encoding.

The benefit of switching to more modern codecs lies of course in the higher compression efficiency that they offer. While the extent of improvement is very dependent on the actual content, bitrates and encoders used, HEVC is considered to offer gains of 30%-50% over AVC, meaning that for the same quality you can spend up to 50% fewer bits. For AV1 this increase is generally a bit higher.. As more and more on-device support is added for these newer codecs, the advantage of utilizing them to reduce both storage and bandwidth is clear. 

Generally speaking, performing such codec modernization involves some non-trivial steps. 

First, you need to get access to the modern encoder you want to use, and know enough about it in order to configure the encoder correctly for your needs. Then you can proceed to encoding using one of the following approaches.

The first approach is to perform bit-rate driven encoding. One possibility is to use conservative bitrates, in which case the potential reduction in size will not be achieved. Another possibility is to set target bitrates that reflect the expected savings, in which case there is a risk of losing quality. For example, In an experimental test of files which were converted from their AVC source to HEVC, we found that on average, a bitrate reduction of 50% could be obtained when using the Beamr CABR codec modernization approach. However, when the same files were all brute-force encoded  to HEVC at 50% reduced bitrate, using the same encoder and configuration, the quality took a hit for some of the files.

 

This example shows the full AVC source frame on top, with the transcodes to HEVC below it. Note the distortion in the blind HEVC encode, shown on the left, compared to the true-to-source video transformed with CABR on the right.

The second approach is to perform the transcode using a quality driven encode, for instance using the constant QP (Quantization Parameter) or CRF (Constant Rate Factor) encoding modes with conservative values, which will in all likelihood preserve the quality. However, in this case you are likely to unnecessarily “blow up” some of your files to much higher bitrates. For example, for the UGC content shown below, transcoding to HEVC using a software encoder and CRF set to 21 almost doubled the file size.

Yet another approach is to use a trial and error encode process for each file or even each scene, manually verifying that a good target encoding setup was selected which minimizes the bitrate while preserving the quality. This is of course an expensive and cumbersome process, and entirely unscalable.

By using Beamr CABR this is all done for you under the hood, in a fully automatic process, which makes optimized choices for each and every frame in your video, selecting the lowest bitrate that will still perfectly preserve the source visual quality. When performed using the Nvidia NVENC SDK with interfaces to Beamr’s CABR technology, this transformation is significantly accelerated and becomes even more cost effective. 

The codec modernization flow is demonstrated for AVC to HEVC conversion in the above high-level block diagram. As shown here, the CABR controller interacts with NVENC, Nvidia’s hardware video encoder, using the new APIs Nvidia has created for this purpose. At the heart of the CABR controller lies Beamr’s Quality Measure, BQM, a unique, patented, Emmy award winning perceptual video quality measure. BQM has now been adapted and ported to the Nvidia GPU platform, resulting in significant acceleration of the optimization process .  

The Beamr optimization technology can be used not only for codec modernization, but also to reduce bitrate of an input video, or of a target encode, while guaranteeing the perceptual quality is preserved, thus creating encodes with the same perceptual quality at lower bitrates or file sizes. In any and every usage of the Beamr CABR solution, size or bitrate are reduced as much as possible while each frame of the optimized encode is guaranteed to be perceptually identical to the reference. The codec modernization use case is particularly exciting as it puts the ability to migrate to more efficient and sophisticated codecs, previously used primarily by video experts, into the hands of any user with video content.

For more information please contact us at info@beamr.com 

How to deal with the tension on the mobile network – part 2 (VIDEO Interview)

In late July, I reported on the “news” that Verizon was throttling video traffic for some users. As usual, the facts around this seemingly punitive act were not fully understood, which triggered this blog post.

At IBC last month (September 2017), I was interviewed by RapidTV where much of the conversation was around the Apple news of their support for HEVC across the device ecosystem running iOS 11 and High Sierra. As I was reviewing this interview, it seemed natural to publish it as a follow up to the original post.

There is no doubt that mobile operators are under pressure as a result of the network crushing video traffic they are being forced to deliver. But the good news is that for those operators who adopt HEVC, they are going to enjoy significant bitrate efficiencies, possibly as high as 50%. And for many services, though they will chose to take some savings, this means they’ll be able to upgrade their resolutions to full 1080p while simultaneously improving the video quality they are delivering.

I hope you find this video insightful. Our team has a very simple evaluation offer to discuss with all qualified video services and video distributors. Just send an email to sales@beamr.com and we’ll get in touch with the details.

We Celebrate with Cake!

At Beamr, when we celebrate, we do it with cake!

Today’s very special, and oh so yummy cake celebration, was a recognition of the amazing milestone that we reached on May 31, 2017 as the result of Beamr acquiring Vanguard Video on April 1st, 2016. Our vision for buying Vanguard as a firmly entrenched leader in HEVC video encoding was to combine Beamr’s world class content-adaptive optimization technology with the world’s best HEVC encoder. The results as we demonstrated at NAB 2017, are nothing short of breathtaking.

Can you imagine second screen HD at 1.5Mbps and 4K UHD with HDR at just 10Mbps? With Beamr 5x, and now that WWDC2017 saw Apple enabling HEVC across their devices, the time is now to move to HEVC so your users can enjoy enhanced UX and improved video quality.

Beamr 5x is available for private beta testing, contact us for more information.

Keep an eye out for all our news, because we’ve only just begun. The technology that we have introduced to the video encoding industry has set a new standard for performance and savings, and what the future holds is nothing short of earth shattering.

And yes, precisely 23 seconds after this picture was taken, this cake was unrecognizable!

2016 Paves the Way for a Next-Gen Video Encoding Technology Explosion in 2017

2016 has been a significant year for video compression as 4K, HDR, VR and 360 video picked up steam, paving the road for an EXPLOSION of HEVC adoption in 2017. With HEVC’s ability to reduce bitrate and file sizes up to 50% over H.264, it is no surprise that HEVC has transitioned to be the essential enabler of high-quality and reliable streaming video powering all the new and exciting entertainment experiences being launched.

Couple this with the latest announcement from HEVC Advance removing royalty uncertainties that plagued the market in 2016 and we have a perfect marriage of technology and capability with HEVC.

In this post we’ll discuss 2016 from the lenses of Beamr’s own product and company news, combined with notable trends that will shape 2017 in the advanced video encoding space.  

>> The Market Speaks: Setting the Groundwork for an Explosion of HEVC

The State of 4K

With 4K content creation growing and the average selling price of UHD 4K TVs dropping (and being adopted faster than HDTVs), 4K is here and the critical mass of demand will follow closely. We recently did a little investigative research on the state of 4K and four of the most significant trends pushing its adoption by consumers:

  • The upgrade in picture quality is significant and will drive an increase in value to the consumer – and, most importantly, additional revenue opportunities for services as consumers are preconditioned to pay more for a premium experience. It only takes a few minutes viewing time to see that 4K offers premium video quality and enhances the entertainment experience.
  • Competitive forces are operating at scale – Service Providers and OTT distributors will drive the adoption of 4K. MSO are upping their game and in 2017 you will see several deliver highly formidable services to take on pure play OTT distributors. Who’s going to win, who’s going to lose? We think it’s going to be a win-win as services are able to increase ARPUs and reduce churn, while consumers will be able to actually experience the full quality and resolution that their new TV can deliver.
  • Commercially available 4K UHD services will be scaling rapidly –  SNL Kagan forecasts the number of global UHD Linear channels at 237 globally by 2020, which is great news for consumers. The UltraHD Forum recently published a list of UHD services that are “live” today numbering 18 VOD and 37 Live services with 8 in the US and 47 outside the US. Clearly, content will not be the weak link in UHD 4K market acceptance for much longer.
  • Geographic deployments — 4K is more widely deployed in Asia Pacific and Western Europe than in the U.S. today. But we see this as a massive opportunity since many people are traveling abroad and thus will be exposed to the incredible quality. They will then return home to question their service provider, why they had to travel outside the country to see 4K. Which means as soon as the planned services in the U.S. are launched, they will likely attract customer more quickly than we’ve seen in the past.

HDR adds WOW factor to 4K

High Dynamic Range (HDR) improves video quality by going beyond more pixels to increase the amount of data delivered by each pixel. HDR video is capable of capturing a larger range of brightness and luminosity to produce an image closer to what can be seen in real life. Show anyone HDR content encoded in 4K resolution, and it’s no surprise that content providers and TV manufacturers are quickly jumping on board to deliver content with HDR. Yes, it’s “that good.” There is no disputing that HDR delivers the “wow” factor that the market and consumers are looking for. But what’s even more promising is the industry’s overwhelmingly positive reaction to it. Read more here.

Beamr has been working with Dolby to enable Dolby Vision HDR support for several years now, even jointly presenting a white paper at SMPTE in 2014. The V.265 codec is optimized for Dolby Vision and HDR10 and takes into account all requirements for both standards including full support for VUI signaling, SEI messaging, SMPTE ST 2084:2014 and ITU-R BT.2020. For more information visit http://beamr.com/vanguard-by-beamr-content-adaptive-hevc-codec-sdk

Beamr is honored to have customers who are best in class and span OTT delivery, Broadcast, Service Providers and other entertainment video applications. From what we see and hear, studios are uber excited about HDR, cable companies are prepping for HDR delivery, Satellite distributors are building the capability to distribute HDR, and of course OTT services like Netflix, FandangoNow (formerly M-GO), VUDU, and Amazon are already distributing content using either Dolby Vision or HDR10 (or both). If your current video encoding workflow cannot fully support or adequately encode content with HDR, it’s time to update. Our V.265 video encoder SDK is a perfect place to start.

VR & 360 Video at Streamable Bitrates

360-degree video made a lot of noise in 2016.  YouTube, Facebook and Twitter added support for 360-degree videos, including live streaming in 360 degrees, to their platforms. 360-degree video content and computer-generated VR content is being delivered to web browsers, mobile devices, and a range of Virtual Reality headsets.  The Oculus Rift, HTC Vive, Gear VR and Daydream View have all shipped this year, creating a new market for immersive content experiences.

But, there is an inherent problem with delivering VR and 360 video on today’s platforms.  In order to enable HD video viewing in your “viewport” (the part of the 360-degree space that you actually look at), the resolution of the full 360 video delivered to you should be 4K or more.  On the other hand, the devices on the market today which are used to view this content, including desktops, mobile devices and VR headsets only support H.264 video decoding. So delivering the high-resolution video content requires very high bitrates – twice as much as using the more modern HEVC standard.

The current solution to this issue is lowered video quality in order to fit the H.264 video stream into a reasonable bandwidth. This creates an experience for users which is not the best possible, a factor that can discourage them from consuming this newly-available VR and 360 video content.  But there’s one thing we know for sure – next generation compression including HEVC and content adaptive encoding – and perceptual optimization – will be a critical part of the final solution. Read more about VR and 360 here.

Patent Pool HEVC Advance Announces “Royalty Free” HEVC software

As 4K, HDR, VR and 360 video gathers steam, Beamr has seen the adoption rate moving faster than expected, but with the unanswered questions around royalties, and concerns of who would shoulder the cost burden, distributors have been tentative. The latest move by HEVC Advance to offer a royalty free option is meant to encourage and accelerate the adoption (implementation) of HEVC, by removing royalty uncertainties.

Internet streaming distributors and software application providers can be at ease knowing they can offer applications with HEVC software decoders without incurring onerous royalties or licensing fees. This is important as streaming app content consumption continues to increase, with more and more companies investing in its future.

By initiating a software-only royalty solution, HEVC Advance expects this move to push the rest of the market i.e. device manufacturers and browser providers to implement HEVC capability in their hardware and offer their customers the best and most efficient video experience possible.

 

>> 2017 Predictions

Mobile Video Services will Drive the Need for Content-adaptive Optimization

Given the trend toward better quality and higher resolution (4K), it’s more important than ever for video content distributors to pursue more efficient methods of encoding their video so they can adapt to the rapidly changing market, and this is where content-adaptive optimization provides a massive benefit.

The boundaries between OTT services and traditional MSO (cable and satellite) are being blurred now that all major MSOs include TVE (TV Everywhere streaming services with both VOD and Linear channels) in their subscription packages (some even break these services out separately as is the case with SlingTV). And in October, AT&T CEO Randall Stephenson vowed that DirecTV Now would disrupt the pay-TV business with revolutionary pricing for an  Internet-streaming service at a mere $35 per month for a package with more than 100 channels.

And get this – AT&T wireless is adopting the practice of “zero rating” for their customers, that is, they will not count the OTT service streaming video usage toward the subscriber’s monthly data plan. This represents a great value for customers, but there is no doubt that it puts pricing pressure on the operational side of all zero rated services.

2017 is the year that consumers will finally be able to enjoy linear as well as VOD content anywhere they wish even outside the home.

Beamr’s Contribution to MSOs, Service Providers, and OTT Distributors is More Critical Than Ever

When reaching to consumers across multiple platforms, with different constraints and delivery cost models, Beamr’s content adaptive optimizer perfects the encoding process to the most efficient quality and bitrate combination.

Whether you pay by the bit delivered to a traditional CDN provider, or operate your own infrastructure, the benefits of delivering less traffic are realized with improved UX such as faster stream start times and reduced re-buffering events, in addition to the cost savings. One popular streaming service reported to us that after implementing our content-adaptive optimization solution their rebuffering events as measured on the player were reduced by up to 50%, while their stream start times improved 20%.

Recently popularized by Netflix and Google, content-adaptive encoding is the idea that not all videos are created equal in terms of their encoding requirements. Content-adaptive optimization complements the encoding process by driving the encoder to the lowest bitrate possible based on the needs of the content, and not a fixed target bitrate (as seen in traditional encoding processes and products).

A content-adaptive solution can optimize more efficiently by analyzing already-encoded video on a frame-by-frame and scene-by-scene level, detecting areas of the video that can be further compressed without losing perceptual quality (e.g. slow motion scenes, smooth surfaces).

Provided the perceptual quality calculation is performed at the frame level with an optimizer that contains a closed loop perceptual quality measure, the output can be guaranteed to be the highest quality at the lowest bitrate possible. Click the following link to learn how Beamr’s patented content adaptive optimization technology achieves exactly this result.

Encoding and Optimization Working Together to Build the Future

Since the content-adaptive optimization process is applied to files that have already been encoded, by combining an industry leading H.264 and HEVC encoder with the best optimization solution (Beamr Video), the market will be sure to benefit by receiving the highest quality video at the lowest possible bitrate and file size. As a result, this will allow content providers to improve the end-user experience with high quality video, while meeting the growing network constraints due to increased mobile consumption and general Internet congestion.

Beamr made a bold step towards delivering on this stated market requirement by disrupting the video encoding space when in April 2016 we acquired Vanguard Video – a premier video encoding and technology company. This move will benefit the industry starting in 2017 when we introduce a new class of video encoder that we call a Content Adaptive Encoder.

As content adaptive encoding techniques are being adopted by major streaming services and video platforms like YouTube and Netflix, the market is gearing up for more advanced rate control and optimization methods, something that fits our perceptual quality measure technology perfectly. This fact when combined with Beamr having the best in class HEVC software encoder in the industry, will yield exciting benefits for the market. Read the Beamr Encoder Superguide that details the most popular methods for performing content adaptive encoding and how you can integrate them into your video workflow.

One Year from Now…

In one year from now when you read our post summarizing 2017 and heralding 2018, what you will likely hear is that 2017 was the year that advanced codecs like HEVC combined with efficient perceptually based quality measures, such as Beamr’s, provide an additional 20% or greater bitrate reduction.

The ripple effect of this technology leap will be that services struggling to compete today on quality or bitrate, may fall so far behind that they lose their ability to grow the market. We know of many multi-service operator platforms who are gearing up to increase the quality of their video beyond the current best of class for OTT services. That is correct, they’ve watched the consumer response to new entrants in the market offering superior video quality, and they are not sitting still. In fact, many are planning to leapfrog the competition with their aggressive adoption of content adaptive perceptual quality driven solutions.  

If any one service assumes they have the leadership position based on bitrate or quality, 2017 may prove to be a reshuffling of the deck.

For Beamr, the industry can expect to see an expansion of our software encoder line with the integration of our perceptual quality measure which has been developed over the last 7 years, and is covered by more than 50 patents granted and pending. We are proud of the fact that this solution has been shipping for more than 3 years in our stand-alone video and photo optimizer solutions.

It’s going to be an exciting year for Beamr and the industry and we welcome you to join us. If you are intrigued and would like to learn more about our products or are interested in evaluating any of our solutions, check us out at beamr.com.

We Need a Revolution of 4K!

Don’t panic or stop reading, we used the word ‘revolution’ in the title and though admittedly it’s provocative being less than a week from the US Presidential elections, we are talking about entertainment and TV, not politics. Cue the massive sigh of relief here…

Our story starts with a recent article published in PC Magazine titled “Meet Two Companies That Want to Revolutionize 4K Video”, where the author Troy Dreier examines the state of 4K and some of the issues surrounding the rate of 4K adoption, specifically a chicken-and-egg problem. As Dreier points out, 4K UHD TVs are being bought in considerable numbers “over 8 million 4K TVs to date, 1.4 million in the US.”

But what about content?

Although, 4K is already far more widely deployed in Asia Pacific and Western Europe, in the US cable and satellite customers are seeing limited content choices, with almost no options in broadcast, leaving consumers turning to online distribution services to satisfy their needs.

But with this comes another problem facing streaming providers, the commodity of the internet: bits.

Though the internet is getting much faster and infrastructure is improving, overall average speeds are still just 15.3 Mbps per household, making it difficult to deliver 4K UHD video sustainably. Or at least with the quality promise that the TV vendors are making. This ultimately, puts the pressure on network operators and over-the-top content suppliers to do everything they can to lower the number of bits they transport without damaging the picture quality of the video.

To this point, Dreier suggests that video optimization solutions are needed to “condense 4K video.” Dreier goes on to point out two solutions that are solving this problem, and one of them he highlights is Beamr’s content adaptive optimization solution, Beamr Video.

At the heart of our video encoding and processing technology solutions is the Beamr content adaptive quality measure that is backed up by more than 20 granted patents with another 30 still pending.  

The Beamr Video optimization technology is based on a proprietary, low complexity, reliable, perceptual quality measure. Or put simply, we have the most advanced commercially available content adaptive quality measure available. The existence of this measure enables controlling a video encoder, to obtain an output clip with maximal compression of the video input, while still maintaining the input video resolution, format and visual quality. This is performed by controlling the compression level frame by frame, in such a way that the maximum number of bits are squeezed out of the file, while still resulting in a perceptually identical visual output.

An important characteristic of our quality measures is that it operates as a full-reference to the source which insures that artifacts are never introduced as a result of the bitrate reduction process. Many “alternative” solutions struggle with inconsistent quality as they operate in an open loop, which means at times quality may be degraded while at other times they leave “bits on the table.”

With so much at stake for next generation entertainment formats, it is critical that every new encoding and video processing technology be evaluated for quality and useability. This is why we are proud of the customers we have which include major Hollywood studios, premium OTT content distributors, MSOs and large video platforms.

Beamr Video in the real world with 720p VBR input, reduced 21%:

beamr_video_live

For more information on the why and how behind content adaptive solutions, download the free Beamr Content Adaptive Tech Guide.

Immersive VR and 360 video at streamable bitrates: Are you crazy?

There have been many high-profile experiments with VR and 360 video in the past year. Immersive video is compelling, but large and unwieldy to deliver. This area will require huge advancements in video processing – including shortcuts and tricks that border on ‘magical’.

Most of us have experienced breathtaking demonstrations that provide a window into the powerful capacity of VR and 360 video – and into the future of premium immersive video experiences.

However, if you search the web for an understanding of how much bandwidth is required to create these video environments, you’re likely to get lost in a tangled thicket of theories and calculations.

Can the industry support the bitrates these formats require?

One such post on Forbes in February 2016 says No.

It provides a detailed mathematical account of why fully immersive VR will require each eye to receive 720 million pixels at 36 bits per pixel and 60 frames per second – or a total of 3.1 trillion bits per second.1

We’ve taken a poll at Beamr, and no one in the office has access to those kinds of download speeds. And some of these folks pay the equivalent of a part-time salary to their ISP!

Thankfully the Forbes article goes on to explain that it’s not quite that bad.

Existing video compression standards will be able to improve this number by 300, according to the author, and HEVC will compress that by 600 down to what might be 5.2 Gbps.

The truth is, the calculations put forth in the Forbes piece are very ambitious indeed. As the author states:

“The ultimate display would need a region of 720 million pixels for full coverage because even though your foveal vision has a more narrow field of view, your eyes can saccade across that full space within an instant. Now add head and body rotation for 360 horizontal and 180 vertical degrees for a total of more than 2.5 billion (giga) pixels.”

A more realistic view of the way VR will rollout was presented by Charles Cheevers of network equipment vendor ARRIS at INTX in May of this year.2

Great VR experiences including a full 360 degree stereoscopic video environment at 4K resolutions could easily require a streaming bandwidth of 500 Mbps or more.

That’s still way too high, so what’s a VR producer to do?

Magical illusion, of course. 

In fact, just like your average Vegas magician, the current state of the art in VR delivery relies on tricks and shortcuts that leverage the imperfect way we humans see.

For example, Foveated Rendering can be used to aggressively compress the areas of a VR video where your eyes are not focused.

This technique alone, and variations on this theme – can take the bandwidth required by companies like NextVR dramatically lower, with some reports that an 8 Mbps stream can provide a compelling immersive experience. The fact is, there are endless ways to configure the end-to-end workflow for VR and much will depend on the hardware and software and networking environments in which it is deployed.

Compression innovations utilizing perceptual frame by frame rate control methodologies, and some involving the mapping of spherical images to cubes and pyramids, in an attempt to transpose images into 5 or 6 viewing planes, and ensure the highest resolution is always on the plane where the eyes are most intensely focused, are being tried.3

At the end of the day, it’s going to be hard to pin down your nearest VR dealer on the amount of bandwidth that’s required for a compelling VR experience. But there’s one thing we know for sure – next generation compression including HEVC and content adaptive encoding – and perceptual optimization – will be a critical part of the final solution.

References:

(1) Found on August 10, 2016 at the following URL: http://www.forbes.com/sites/valleyvoices/2016/02/09/why-the-internet-pipes-will-burst-if-virtual-reality-takes-off/#ca7563d64e8c

(2) Start at 56 minutes. https://www.intxshow.com/session/1041/  — Information and a chart is also available online here: http://www.onlinereporter.com/2016/06/17/arris-gives-us-hint-bandwidth-requirements-vr/ 

(3) Facebook’s developer site gives a fascinating look at these approaches, which they call dynamic streaming techniques. Found on August 10, 2016 at the following URL:  https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/

Translating Opinions into Fact When it Comes to Video Quality

This post was originally featured at https://www.linkedin.com/pulse/translating-opinions-fact-when-comes-video-quality-mark-donnigan 

In this post, we attempt to de-mystify the topic of perceptual video quality, which is the foundation of Beamr’s content adaptive encoding and content adaptive optimization solutions. 

National Geographic has a hit TV franchise on its hands. It’s called Brain Games starring Jason Silva, a talent described as “a Timothy Leary of the viral video age” by the Atlantic. Brain Games is accessible, fun and accurate. It’s a dive into brain science that relies on well-produced demonstrations of illusions and puzzles to showcase the power — and limitation — of the human brain. It’s compelling TV that illuminates how we perceive the world.(Intrigued? Watch the first minute of this clip featuring Charlie Rose, Silva, and excerpts from the show: https://youtu.be/8pkQM_BQVSo )

At Beamr, we’re passionate about the topic of perceptual quality. In fact, we are so passionate, that we built an entire company based on it. Our technology leverages science’s knowledge about the human vision system to significantly reduce video delivery costs, reduce buffering & speed-up video starts without any change in the quality perceived by viewers. We’re also inspired by the show’s ability to turn complex things into compelling and accessible, without distorting the truth. No easy feat. But let’s see if we can pull it off with a discussion about video quality measurement which is also a dense topic.

Basics of Perceptual Video Quality

Our brains are amazing, especially in the way we process rich visual information. If a picture’s worth 1,000 words. What’s 60 frames per second in 4k HDR worth?

The answer varies based on what part of the ecosystem or business you come from, but we can all agree that it’s really impactful. And data intensive, too. But our eyeballs aren’t perfect and our brains aren’t either – as Brain Games points out. As such, it’s odd that established metrics for video compression quality in the TV business have been built on the idea that human vision is mechanically perfect.

See, video engineers have historically relied heavily on two key measures to evaluate the quality of a video encode: Peak Signal to Noise Ratio, or PSNR, and Structured Similarity, or SSIM. Both metrics are ‘objective’ metrics. That is, we use tools to directly measure the physics of the video signal and construct mathematical algorithms from that data to create metrics. But is it possible to really quantify a beautiful landscape with a number? Let’s see about that.

PSNR and SSIM look at different physics properties of a video, but the underlying mechanics for both metrics are similar. You compress a source video where the properties of the “original” and derivative are then analyzed using specific inputs, and metrics calculated for both. The more similar the two metrics are, the more we can say that the properties of each video are similar, and the closer we can define our manipulation of the video, i.e. our encode, as having a high or acceptable quality.

Objective Quality vs. Subjective Quality


However, it turns out that these objectively calculated metrics do not correlate well to the human visual experience. In other words, in many cases, humans cannot perceive variations that objective metrics can highlight while at the same time, objective metrics can miss artifacts a human easily perceives.

The concept that human visual processing might be less than perfect is intuitive. It’s also widely understood in the encoding community. This fact opens a path to saving money, reducing buffering and speeding-up time-to-first-frame. After all, why would you knowingly send bits that can’t be seen?

But given the complexity of the human brain, can we reliably measure opinions about picture quality to know what bits can be removed and which cannot? This is the holy grail for anyone working in the area of video encoding.

Measuring Perceptual Quality

Actually, a rigorous, scientific and peer-reviewed discipline has developed over the years to accurately measure human opinions about the picture quality on a TV. The math and science behind these methods are memorialized in an important ITU standard on the topic originally published in 2008 and updated in 2012. ITU BT.500 (International Telecommunications Union is the largest standards committee in global telecom.) I’ll provide a quick rundown.

First, a set of clips is selected for testing. A good test has a variety of clips with diverse characteristics: talking heads, sports, news, animation, UGC – the goal is to get a wide range of videos in front of human subjects.

Then, a subject pool of sufficient size is created and screened for 20/20 vision. They are placed in a light-controlled environment with a screen or two, depending on the set-up and testing method.

Instructions for one method is below, as a tangible example.

In this experiment, you will see short video sequences on the screen that is in front of you. Each sequence will be presented twice in rapid succession: within each pair, only the second sequence is processed. At the end of each paired presentation, you should evaluate the impairment of the second sequence with respect to the first one.

You will express your judgment by using the following scale:

5 Imperceptible

4 Perceptible but not annoying

3 Slightly annoying

2 Annoying

1 Very annoying

Observe carefully the entire pair of video sequences before making your judgment.

As you can imagine, testing like this is an expensive proposition indeed. It requires specialized facilities, trained researchers, vast amounts of time, and a budget to recruit subjects.

Thankfully, the rewards were worth the effort for teams like Beamr that have been doing this for years.

It turns out, if you run these types of subjective tests, you’ll find that there are numerous ways to remove 20 – 50% of the bits from a video signal without losing the ‘eyeball’ video quality – even when the objective metrics like PSNR and SSIM produce failing grades.

But most of the methods that have been tried are still stuck in academic institutions or research labs. This is because the complexities of upgrading or integrating the solution into the playback and distribution chain make them unusable. Have you ever had to update 20 million set-top boxes? Well if you have, you know exactly what I’m talking about.

We know the broadcast and large scale OTT industry, which is why when we developed our approach to measuring perceptual quality and applied it to reducing bitrates, we were insistent on staying 100% inside the standard of AVC H.264 and HEVC H.265.

By pioneering the use of perceptual video quality metrics, Beamr is enabling media and entertainment companies of all stripes to reduce the bits they send by up to 50%. This reduces re-buffering events by up to 50%, improves video start time by 20% or more, and reduces storage and delivery costs.

Fortunately, you now understand the basics of perceptual video quality. You also see why most of the video engineering community believes content adaptive sits at the heart of next-generation encoding technologies.

Unfortunately, when we stated above that there were “all kinds of ways” to reduce bits up to 50% without sacrificing ‘eyeball video quality’, we skipped over some very important details. Such as, how we can utilize subjective testing techniques on an entire catalog of videos at scale, and cost efficiently.

Next time: Part 2 and the Opinionated Robot

Looking for better tools to assess subjective video quality?

You definitely want to check out Beamr’s VCT which is the best software player available on the market to judge HEVC, AVC, and YUV sequences in modes that are highly useful for a video engineer or compressionist.

VCT is available for Mac and PC. And best of all, we offer a FREE evaluation to qualified users.

Learn more about VCT: http://beamr.com/h264-hevc-video-comparison-player/