Uncategorized

Uncategorized

For Video Streaming Services, The Internet Is A Shared Resource To Preserve

Posted on by Dror Gill

TL;DR 

Post COVID-19, the work from home trend will continue, and this will extend the pressure on the Internet from video traffic. Even with the EU Commissioner’s call for video services to reduce their traffic by 25%, as Internet traffic patterns shift from corporate networks to mobile, fixed wireless, and broadband networks, the need to reduce video bandwidths will continue beyond COVID-19. Consumers will still demand the highest quality, and those streaming services meeting their expectations while delivering video in as small a footprint as possible will dominate the market. Now is the time for the streaming video industry to play an active role in adopting more efficient codecs and content-adaptive bitrate technology so that streaming video services can ensure a great user experience without disrupting the Internet. 

https://youtu.be/ltYNDQkiyl8

The Internet is a shared resource to preserve.

For the video streaming industry, Thursday, March 19th, marked the day of reckoning for runaway bitrates and seemingly never-ending network capacity. On March 19th, Thierry Breton, the European Commissioner for the Internal Market tweeted, “let’s #SwitchToStandard definition when HD is not necessary.” The result is that most of the best known US video services, including Facebook and Instagram, agreed to a 25% reduction in bandwidth used for video delivered in Europe, UK, and Israel. With other countries rumored to follow suit. 

We can blame COVID-19 for the strain as a result of closed schools and businesses, leading to increased use of video conferencing, streaming video services, and cloud gaming. Verizon reported that total web traffic is up 22% between March 12th and March 19th, while week-over-week usage patterns for streaming video services increased by 12%. However, it’s easily predictable that these numbers are trending even higher as the quarantine and shelter in place orders expanded, as evidenced by Cloudflare reporting Internet traffic is 40% higher than pre COVID-19 levels.

The purpose of this article is to provide a framework for how video streaming services may want to think about the Internet post COVID-19 where video streaming services and video-centric applications will need to consider their utilization of the Internet as a shared and not an unlimited resource.

Content-Adaptive Encoding is no longer a nice to have for streaming services. 

There are multiple technical and technology options available for reducing video bitrate. The fastest to implement, however, is to drop the resolution of the video. By manipulating the video playlist (called a manifest) that organizes the various resolutions and bit rates that enable the video player to adapt to the speed of the network, a video service can achieve immediate savings by merely serving a lighter weight version of the video. Standard-definition (SD) instead of high-definition (HD). This approach is what most of the complying services have taken, but, it is not a sustainable answer since dropping resolution impacts video experience negatively.

A more advanced technique known as Content-Adaptive Encoding works by guiding the encoder to adapt the bitrate allocation to the needs of the video content. 

Reducing resolution is not what consumers want, and this will make content-adaptive encoding essential for many video encoding workflows. Because content-adaptive encoding solutions require integration, for some services, it was relegated to the “nice to have” list. But now, with the sweeping changes to video consumption that is driving network saturation, those services that must compete with high visual quality, are shifting the priority to “must-have.” 

Effective tools and methods to be a good citizen of the Internet.

If we are going to be a good citizen of the Internet, we should understand what tools and methods are available to preserve this precious shared resource while delivering a suitable UX and visual quality. 

Engineering for video encoding is about tradeoffs. The three primary levers are 1) bitrate, 2) resolution, 3) encoder performance. These levers are interconnected and dependent. For example, it’s not possible to achieve high bitrate efficiency at higher resolutions without affecting encoder performance (increasing CPU cycles).  

From a video quality perspective, lever one and lever two are the levers available to most video encoding engineers. While from an operational point of view, the third lever is what most impacts bitrate and quality.

The tools that we can use to reduce bandwidth include the use of advanced video codecs such as HEVC. HEVC (H.265) provides up to 50% reduction in bitrate at the same quality level as H.264, the current dominant codec used around the world. The other tool available is advanced technology, such as content-adaptive encoding, implemented inside the encoder. 

Beamr’s Content-Adaptive Bitrate (CABR) rate-control is an example of advanced technology that brings an additional 20-40% reduction in bitrate. Using HEVC and CABR, a 4K HDR video file can be as small as 10Mbps, an added savings of as much 6Mbps without CABR. With the promise of a 50% bitrate reduction using HEVC, and over 2 billion devices supporting HEVC decoding in hardware, it’s the obvious thing to do for a video service concerned about the sustainability of the Internet.

If a technical integration of a new codec is not possible, the three most popular methods for reducing bitrate are Per-Category, Per-Title, and Per-Frame Encoding optimization.

Per-Category Encoding optimization.

The Per-Category Encoding approach is least practical for premium movies, and TV shows since the range of encoding complexity within a category can vary significantly. Animated videos are typically easier to compress than video captured from a camera sensor, given the wide range of complexity. Animation techniques are highly diverse, from hand-drawn to 2D to 3D, and that makes it challenging to create an encoding ladder that works across animated content equally. 

Per-Category Encoding is the easiest of all the methods to implement, but also produces the lowest real bitrate reduction because of the variability of scenes. For example, a sports broadcast may include talking head in-studio shots along with fast action gameplay and slow-motion recaps, each requiring different bitrate values to preserve the quality level. 

Per-Title Encoding optimization.

Per-Title Encoding received a big boost when Netflix published a blog post explaining their encoding schema that creates a custom encoding ladder for each video file. The system performs a series of test encodes at different CRF levels and resolutions that are analyzed using the Video Multimethod Assessment Fusion (VMAF) quality metric. Netflix uses the scores to identify the best quality resolution at each applicable data rate. 

Though many video services have adopted their variation, Per-Title Encoding, or some variation of it can now be found in many video encoding workflows. It’s a great way to rethink fixed ABR recipes that are the primary source of wasted bandwidth, or poor video quality. Per-Title Encoding only works when you have a smaller library as it requires extensive computing resources to run the hundreds of fractional encodes needed for each title. 

Per-Title Encoding helps to reduce bitrate but is limited in its ability since the rudimentary VBR rate-control bounds the encoder QP setting with no additional intelligence. 

Per-Frame Encoding optimization.

The weakness of a category or title based optimization method is that this approach cannot adapt to the specific needs of the video at the frame level. Only by steering the encoder decisions frame by frame is it possible to achieve the ultimate result of producing high quality with the least number of bits required. 

Beamr’s CABR technology is the primary feature of the Beamr 4x and Beamr 5x encoding engines. CABR operates at the frame level to deliver the smallest possible size for each video frame while ensuring the highest overall quality of each frame within the video sequence. This approach avoids transient quality issues in other optimization techniques. The Beamr Quality Measure Analyzer has a higher correlation with subjective results than existing quality measures such as PSNR, and SSIM. CABR is protected by the majority of Beamr’s 48 granted patents. 

To learn more about Beamr’s Content-Adaptive Bitrate technology, you can hear Tamar Shoham, Head of Algorithms at Beamr, explain CABR here.

We must all play our part in preserving the integrity of the Internet.

Just as environmental sustainability is an essential initiative for companies who want to be good citizens of the world, in the COVID-19 world that we are living in, video sustainability is now an equally vital initiative. And this is likely to be unchanged in the future as the work from home and virtual meeting trends continue post COVID-19. Now is the time for the streaming video industry to play an active role in adopting more efficient codecs and content-adaptive bitrate technology so that we can ensure a great user experience without disrupting the Internet. 

eGamingUncategorized

They don’t collect baseball cards, but eSports super fans are giving traditional sports a run for their money.

Posted on by Dror Gill

For sports fans that grew up before the 90s, you likely have fond memories of collecting baseball memorabilia or going to your first game. To hear eSports fans rattle off statistics and kill ratios from their favorite Twitch stream or boasting about their latest Fortnite Skin may bring back memories of spilling over your favorite team’s stats in the newspaper on Sundays or trading cards with your friends. Today’s eSports fans may engage a little differently than the traditional sporting fans of yonder, but they may be the most engaged fans in history – and they’re making their mark.

With over 2.2M creators streaming on Amazon’s popular video game streaming service, Twitch, every month, 517M watching gaming on YouTube, and another 185M consuming their video content on Twitch, eSports viewership has surpassed HBO, Netflix, ESPN & Hulu, combined. The massive online gaming viewership is changing the sporting landscape and technology requirements for fans, content providers, and ISPs alike.

To meet online gamers demand for faster & higher quality gaming experiences, Cox Cable recently launched a trial of their Elite Gamer package. The Elite Gamer package is a premium offer that they claim will result in “34% less lag, 55% fewer ping spikes, and 45% less jitter” by speeding up the connection between the player and the desired gaming server.

We believe this marks one of the first in what will be standard practices for ISPs and content providers. When you factor in the massive amount of content delivered and consumed via Twitch & YouTube into perspective, it’s no wonder that vendors are starting to consider how they will address the bandwidth & technology requirements that are needed to maintain the eGaming industry. For the casual gamer, they are required to have a download speed of at least 3 Mbps, an upload speed of at least 1 Mbps, and a ping rate under 150 ms and those figures multiply with each concurrent player in your household.

At Beamr, we live and breathe optimization. For us, the quality and bandwidth challenges introduced by the gaming industry are an opportunity to see how far we can push the limits of balancing video compression with the highest video quality possible.   

If you are passionate about gaming and are curious about what it takes to deliver a high-quality cloud gaming experience, you will enjoy this episode from our podcast, The Video Insiders, where we interviewed Yuval Noimark from Electronic Arts. Listen to Episode 15 here.

Sources:

https://www.superdataresearch.com/market-data/gaming-video-content/

https://twitchadvertising.tv/audience/

https://www.highspeedinternet.com/resources/how-much-speed-do-i-need-for-online-gaming

Uncategorized

Why Game of Thrones is pushing video encoder capabilities to the edge

Posted on by Dror Gill

Game of Thrones entered its eighth and final season on April 14th, 2019. Though Game of Thrones has been a cultural phenomenon since the beginning of its airing, the attention and eyeballs on these final episodes have been higher than ever. Right now, every aspect of season eight is under a microscope and up for discussion, from the battle of Winterfell to theories on the Azor Ahai prophecy, super fans are taking to the internet and social media to debate and swap theories. Yet, even if you’ve installed the Chrome extension GameOfSpoilers to block Game of Thrones news from popular social networks, you probably did not miss all the fans who flocked to social media to report their dissatisfaction with the poor quality of Season 8 Episode 3, “The Longest Night.”

CHECK OUT THIS PODCAST EPISODE

Though not all viewers experienced degraded visual quality for “The Longest Night”, a sufficiently high number did report a poor viewer experience, which triggered TechCrunch to write an article titled, “Why did last night’s ‘Game of Thrones’ look so bad?”

And TechCrunch wasn’t alone, The Verge also wrote an entire piece on how to setup your TV for a rewatch of “The Longest Night”, something that seems hardly possible. After all how is it possible that fans could need to rewatch an episode, not because the plot was so twisted or complicated that they needed a second pass at deciphering it, but because they couldn’t see what was happening on the screen? And in fact, Game of Thrones super fans were not shy in taking to Twitter with their quality assessments.

Why does this look so bad?

Before you throw a Valyrian steel dagger at your TV, let’s take a close look at what happened to create this poor video quality by diving into the operational structure of the underlying video codecs that are used by all commercial streaming services.

The video compression schemes used in video streaming, including AVC which is also used by most PayTV cable and satellite distributors, utilize hybrid block-based video codecs. These codecs use block-based encoding methods which mean each video frame or picture is partitioned into blocks during the compression process, and they also apply motion estimation between frames. Though the effective compression that is made possible by these techniques is impressive, hybrid block-based compression schemes are inherently prone to creating blockiness and banding artifacts, which can be particularly evident in dark scenes.

Blockiness is a video artifact where areas of a video image appear to be comprised of small squares, rather than proper detail and smooth edges as the viewer would expect to see. The blockiness artifact happens when not enough detail is preserved in each of the coding blocks, resulting in inconsistencies between adjacent blocks and making each block appear separate from its neighbors. The video quality will suffer from blockiness when too much detail is lost within each block.

There are two main causes of blockiness. The first is when there is a mismatch between the content complexity and the target bitrate. It can be present either in highly complex content which is encoded at typical bitrates, or in standard content which is compressed to overly aggressive bitrates. Content providers can avoid this by using content adaptive solutions which match the encoder bitrate to the content complexity. The second cause of blockiness is from poor quality decisions made by the encoder, such as discarding information which is crucial for visual quality.

As noted by TechCrunch, for the specific Game of Thrones episode “The Longest Night”, the images are very dark and have a limited range of colors and brightness levels, basically between grey and dark grey. Encoding this limited range of grey shades, which filled up most of the screen, resulted in “banding” artifacts which is where there are visible transitions as a result of the video being represented by just a few shades of grey, which look like “stripes” instead of smooth gradients. Video suffers from banding when the color or brightness palette being used has too few values to accurately describe the shades present in part or all of the video frame.

The prevalent assumption even among some video engineers is that increasing bitrate is the cure-all to video quality problems. But as we’ll see in this case, it’s likely that even if the bitrate had been doubled, the systemic artifacts would still be present. Thus, the solution is not likely external to the video encoding process, but rather can only be addressed at the codec implementation level.

That is, the video encoding engine must be improved to prevent situations like this in the future. That, or HBO and other premium content owners could instruct their filmmakers to avoid dark scenes – We’ll stick with Option #1!

In this case, the video quality issues were not caused by the video encoding bitrate being too low. In fact, the bitrate used was more than sufficient to represent the limited range of colors and brightness. The issue was in the decisions made by the specific video encoder used by HBO. These are decisions regarding how to allocate the available bitrate, or how the bits should be used for different elements of the compressed video stream.

Without getting too deep into video compression technology, it is sufficient to say that a compressed video stream consists basically of two types of data.

The first type is prediction data which enables the creation of a prediction block from previously decoded pixels (in either the same or a reference frame). This prediction block acts as a rough estimate of the block and is complemented by the residual or error block, encoded in the stream. This is essentially a block that fills in the difference between the predicted block and the actual source video block.

The second type of data that is key to how a block-based video encoder works can be found in the rate-distortion algorithm which optimizes the selection of the prediction modes which the prediction data represents. This determines the level of detail to preserve in the residual block. The decisions are made in an attempt to minimize distortion, or maximize quality, for a specific bit allocation which is derived from the target bit-rate.

When a scene is very dark and consists of small variations of pixel colors, the numerical estimates of distortion may be skewed. Components of the video encoder including motion estimation and the rate-distortion algorithm should adapt to optimize the allocations and decisions for this particular use case.

For example, the motion estimation might classify the differences in pixel values as noise instead of actual motion, thus providing inferior prediction information. In another example, if the distortion measures are not tuned correctly, the residual may be considered noise rather than true pixel information and may be discarded or aggressively quantized.

Another common encoder technique that is affected and often “fooled” by very dark scenes is early terminations. Many encoders use this technique to “guess” what would be the best encoding decision they should make, instead of making an exhaustive search of all possibilities, and computing their cost. This technique improves the performance of the video encoder, but in the case of dark scenes with small variations, it can cause the encoder to make the wrong decision.

Some encoding engineers use a technique called “capped CRF” for encoding video at a constant quality instead of a pre-defined bitrate. This is a simple form of “content-adaptive” or “content-aware” encoding, which produces different bitrates for each video clip or scene based on its content. In some implementations, when this technique is used for dark scenes, it can also be “fooled” by the limited range of color and brightness values and may perform very aggressive quantization thus removing too much information form the residual blocks, resulting in these blockiness and banding artifacts.

In summary, we can conclude that dark scenes can lead to various encoding issues if the encoder is not “well prepared” for this type of content, and it seems that this is what happened with this Game of Thrones episode.

Better luck quality, next time.

In order to ensure good quality video across different content types, the encoder must be able to correctly adapt to each and every frame being encoded. In Beamr encoders we tackle this with a combination of tools and algorithms to provide the best video quality possible.

Beamr encoders use unique, patented and patent-pending approaches, to calculate psycho-visual distortions to be used by the rate control module when deciding on prediction modes and on the bit allocations for different components of the compressed video data. This means that the actual visual impact of different decisions is taken into account, resulting in the improved visual quality of the content across a variety of content types.

Beamr encoders offer a wide variety of encoding speeds for different use cases, ranging from lightning fast to enable a full 4K ABR stack to be generated on a single server for live or VOD. When airing premium content of the caliber of Game of Thrones, one should opt for the maximum quality by using slower encoder speeds.

In these speeds, the encoder is wary of invoking early termination methods and thus does not overlook the data that may be hiding in the small deviations of the dark pixel values. We invest a huge effort to discover the optimal combinations for all the internal controls of the algorithm such as the most optimum lambda values for the different rate-distortion cost calculations, optimal values for the deblocking filter (and SAO in the case of Beamr 5 for HEVC) , and many other details – none of which are overlooked.

Rather than use a CRF like based approach for constant quality encoding, Beamr employs a sophisticated technique for content-adaptive encoding called CABR. The content-adaptive bitrate mode of encoding operates in a closed loop and examines the quality of each frame using a patented perceptual quality measure. Our perceptual quality measure is also specifically tuned to adapt to the “darkness” of the frame and each region within the frame, which makes it highly effective even when processing very dark scenes such as the “The Longest Night”, or fade-in and fade-out sequences.

Looking to the Future

For content providers, viewer expectations and demands for quality will continue to rise each year.  A decade ago, you could slide by not delivering a consistent experience across devices. Today, not only is video degradation noticed by your viewers, it can have a massive impact on your audience and churn if you’re not delivering an experience inline with their quality expectations.  

To see what high quality at the lowest bitrate should look like, try Beamr Transcoder for free or contact our team by sending an email to sales@beamr.com to learn about our comparison tool Beamr View.

H.265HEVCMobile VideoOTTReduce BitrateUncategorized

How to deal with the tension of video on the mobile network – Part 1

Posted on by Dror Gill

Last week, the Internet erupted in furor over Verizon’s alleged “throttling” of video streaming services over their mobile network. With a quick glance at the headlines, and to the uninitiated, this could be perceived as an example of a wireless company taking their market dominance too far. Most commenters were quick to pontificate calling “interference” by Verizon a violation of net neutrality.

But this article isn’t about the argument for, or against, network neutrality. Instead, let’s examine the tension that exists as a result of the rapid increase in video consumption on mobile devices for the OTT and video streaming industry. Let’s explore why T-Mobile, Verizon, and others that have yet to come forward, feel the need to reduce the size of the video files that are streaming across their networks.

Cisco reports that by 2021, 82% of all Internet traffic will be video, and for the mobile network video is set to explode equally so that by 2022 75% of data flowing over a mobile network will be video according to Ericsson. This increase of video over the mobile network means by 2021, the average user is set to consume a whopping 8.9GB of data every month as reported by BGR. These data points reveal why escalating consumption of video by wireless subscribers is creating tension in the ecosystem.

So what are the wireless operators trying to achieve by reducing the bitrates of video that is being delivered on their network?

Many mobile service operators offer their own entertainment video service packages, which means they are free to deliver the content in the quality that is consistent with their service level positioning. For some, this may be low to medium quality, but most viewers won’t settle for anything short of medium to high quality.

As most mobile networks have internal video distribution customers such as AT&T with DirecTV Now, at the same time, AT&T delivers video for Netflix. Which means, DirecTV Now is free to modify the encoded files to the maximum extent in order to achieve a perfect blend of quality and low bitrate, while for premium services like Netflix, the video packets cannot be touched due to DRM and the widespread adoption of HTTPS encryption. The point is, mobile carriers don’t always control the formats or quality of video that they carry over the network and for this reason, every content owner and video distributor should have an equal interest in pre-packaging (optimizing) their content for the highest quality and smallest file size possible.

As consumers grow more savvy to the difference in video and service quality between content services, many are becoming less willing to compromise. After all, you don’t invest in a top-of-the-line phone with an AMOLED screen to watch blocky low resolution video. Yet, because of the way services deliver content to mobile devices, in some cases, the full quality of the devices’ screen is unable to be realized by the consumer.

We see this point accentuated when a mobile network operator implements technology designed to reduce the resolution, or lower video complexity, in order to achieve a reduced bandwidth target. Attempts are made to make these changes while preserving the original video quality as much as possible, but it stands to reason that if you start with 1080p (full HD) and reduce the resolution to 480p (standard definition), the customer experience will suffer. Currently, the way bandwidth is being reduced on mobile networks is best described as a brute force method. In scenarios where mobile operators force 480p, the bitrate is reduced at the expense of resolution. But is this the best approach? Let’s take a look.

Beamr published a case study with findings from M-GO where our optimization solution helped to reduce buffering events by up to 50%, and improved stream start times by as much as 20%. These are impressive achievements, and indicative of the value of optimizing video for the smallest size possible, provided the original quality is retained.

A recent study “Bit rate and business model” published by Akamai in conjunction with Sensum also supports M-GO and Conviva’s Viewer Experience Report findings. In the Akamai/Sensum study, the human reaction to quality was measured and the researchers found that three out of four participants would stop using a service after even a few re-buffering events.

For the study, viewers were split into two control groups with one group exposed only to a lower resolution (quality) stream that contained at least one stream interruption (re-buffering event). This group was 20% less likely to associate a positive word with the viewing experience as compared to viewers who watched the higher quality full resolution stream that played smoothly without buffering (resolutions displayed were up to 4K). Accordingly, lower quality streams lead to a 16% increase in negative emotions, while higher quality streams led to a 20% increase in emotional engagement.

There are those who claim “you can’t see 4K”, or use phrases like “smarter pixels not more pixels.” With the complexity of the human visual system and its interconnection to our brain, the Akamai study shows that our physiological systems are able to detect differences between higher resolution and lower resolution. These disruptions were validated by changes in the viewers eye movements, breathing patterns, and increased perspiration.

Balancing the needs of the network, video distributor, and consumer.

  • Consumers expect content at their fingertips, and they also expect the total cost of the content and the service needed to deliver it, to be affordable.
  • Service providers are driven by the need to deliver higher quality video to increase viewer engagement.
  • Mobile network operators welcome any application that drives more demand for their product (data) with open arms, yet, need to face the challenge of how to deal with this expanding data which is beginning to outstrip the customers willingness to pay.

Delivering content over the Internet is not free as some assume. Since the streaming video distributor pays the CDN by the size of the package, e.g. gigabytes delivered, they are able to exploit the massive network investments made by mobile operators. Meanwhile, they (or more specifically their end-customers) carry the expectation that the capacity needed to deliver their videos to meet demand, will always be available. Thus, a network operator must invest ahead of revenues with the promise that growth will meet the investment.

All of this can be summed up by this simple statement, “If you don’t take care of the bandwidth, someone else will.”

Video codecs are evolutionary with each progressive codec being more efficient than the last. The current standard is H.264 and though this codec delivers amazing quality with reasonable performance and bitrate reduction, it’s built on a standard that is now fourteen years old. However, as even entry level mobile phones now support 1080p, video encoding engineers are running into an issue with H.264 not able to reach the quality they need below 3 Mbps. In fact, some distributors are pushing their H.264 bitrates lower than  3Mbps for 1080p, but in doing so they must be willing to introduce noticeable artifacts. So the question is, how do we get to 2 Mbps or lower, but with the same quality of 3-4 Mbps, and with the original resolution?

Enter HEVC.

With Apple’s recent announcement to support HEVC across as many as 400 million devices with HW decoding, content owners should be looking seriously to adopt HEVC in order to realize the 40% reduction in bitrate that Apple is reporting, over H.264. But how exactly can HEVC bring relief to an overburdened mobile network?

In the future it can be argued that once HEVC has reached broad adoption, the situation we have today with bitrates being higher than we’d like, will no longer exist. After all, if you could flip a switch and reduce all the video traffic on the network by 40% with a more efficient compression scheme (HEVC), then it’s quite possible that we’ll push the bandwidth crunch out for another 3-5 years.

But this thinking is more related to fairytales and unicorns than real life. For one thing, video encoding workflows and networks do not function like light switches. Not only does it takes time to integrate and test new technology, but a big issue is that video consumption and advanced entertainment experiences, like VR, AR, and 360, will consume the new white space as quickly as it becomes available, bringing us back to where we are today.

Meeting the bandwidth challenge will require us working together.

In the above scenario, there is a shared responsibility on both the distributor and the network to each play their role in guaranteeing that quality remains high while not wasting bits. For those who are wondering, inefficient encoding methods, or dated codecs such as H.264 fall into the “inefficient” category.

The Internet is a shared resource and whether it stays under some modicum of government regulation, or becomes open again, it’s critical for all members of the ecosystem to recognize that the network is not of infinite capacity and those using it to distribute video should respect this by taking the following steps:

  1. Adopt HEVC across all platforms and resolutions. This step alone will yield up to a 40% reduction over your current H.264 bandwidths.
  2. Implement advanced content-adaptive technologies such as Beamr CABR (CABR stands for Content-Adaptive Bitrate) which can enable a further reduction of video bitrates over the 40% that HEVC affords, by an additional 30-50%.
  3. Adopt just in time encoding that can allow for real-time dynamic control of bitrate based on the needs of the viewing device and network conditions. Intel and Beamr have partnered to offer an ultra-high density and low cost HEVC 4K, live 10bit encoding solution using the E3 platform with IRIS PRO P580 graphics accelerator.

In conclusion.

  • With or without network neutrality, reducing video bandwidth will be a perpetual need for the foreseeable future. Whether to delay capex investment, or to meet competitive pressure on video quality, or simply to increase profitability and decrease opex, the benefits to always delivering the smallest file and stream sizes possible, are easy to model.
  • The current method of brute forcing lower resolutions, or transcoding to reduced framerate will not be sustainable as consumers are expecting the original experience to be delivered. The technical solutions implemented must deliver high quality and be ready for next generation entertainment experiences. At the same time, if you don’t work to trim the fat from your video files, someone else may do it, and it most certainly will be at the expense of video quality and user experience.
  • HEVC and Beamr CABR represent the state of the art in high quality video encoding and bitrate reduction (optimization) without compromise.

If you’d like to learn more, keep an eye out for part two in this series, or take a moment to read this relevant article: It’s Unreasonable to Expect ISP’s Alone to Finance OTT Traffic

In the meantime, you can download our VP9 vs. HEVC white paper, learn how to encode content for the future, or contact us at sales@beamr.com to talk further.

 

H.265HEVCHEVC/VP9UncategorizedVP9

Comparing HEVC & VP9 Made Easy

Posted on by Dror Gill

Apple recently announced their plans to enable HEVC on up to one billion devices with the release of iOS 11 and High Sierra later this year, causing video services to reevaluate their workflows and the codecs they are using. With video codecs in a constant state of ongoing development, it can be hard to know which one is right to choose.  But with the ever increasing pressure to reduce bitrates and upgrade quality, video service providers looking for a competitive edge, are examining next generation encoding platforms, namely HEVC and VP9.

To aid in the evaluation process, our software video encoding engineers have compiled a guide that will eliminate the guesswork of comparing HEVC and VP9. This guide provides a qualitative overview of the strengths, weaknesses, and key differences between the HEVC (H.265) and VP9 codec toolsets, including:

  • Partitioning approaches & block sizes
  • Variable length coding features
  • Transform types and sizes
  • Intra & inter prediction
  • In-loop filters

Click graphic to download the full white paper.

beamr-hevc-vp9-comparison-chart

4K TVBroadcastContent EverywhereEncoder ApplicationsH.265HEVCLiveOTTReduce BitrateUncategorizedV.265 Codec SDKVR

2016 Paves the Way for a Next-Gen Video Encoding Technology Explosion in 2017

Posted on by Dror Gill

2016 has been a significant year for video compression as 4K, HDR, VR and 360 video picked up steam, paving the road for an EXPLOSION of HEVC adoption in 2017. With HEVC’s ability to reduce bitrate and file sizes up to 50% over H.264, it is no surprise that HEVC has transitioned to be the essential enabler of high-quality and reliable streaming video powering all the new and exciting entertainment experiences being launched.

Couple this with the latest announcement from HEVC Advance removing royalty uncertainties that plagued the market in 2016 and we have a perfect marriage of technology and capability with HEVC.

In this post we’ll discuss 2016 from the lenses of Beamr’s own product and company news, combined with notable trends that will shape 2017 in the advanced video encoding space.  

>> The Market Speaks: Setting the Groundwork for an Explosion of HEVC

The State of 4K

With 4K content creation growing and the average selling price of UHD 4K TVs dropping (and being adopted faster than HDTVs), 4K is here and the critical mass of demand will follow closely. We recently did a little investigative research on the state of 4K and four of the most significant trends pushing its adoption by consumers:

  • The upgrade in picture quality is significant and will drive an increase in value to the consumer – and, most importantly, additional revenue opportunities for services as consumers are preconditioned to pay more for a premium experience. It only takes a few minutes viewing time to see that 4K offers premium video quality and enhances the entertainment experience.
  • Competitive forces are operating at scale – Service Providers and OTT distributors will drive the adoption of 4K. MSO are upping their game and in 2017 you will see several deliver highly formidable services to take on pure play OTT distributors. Who’s going to win, who’s going to lose? We think it’s going to be a win-win as services are able to increase ARPUs and reduce churn, while consumers will be able to actually experience the full quality and resolution that their new TV can deliver.
  • Commercially available 4K UHD services will be scaling rapidly –  SNL Kagan forecasts the number of global UHD Linear channels at 237 globally by 2020, which is great news for consumers. The UltraHD Forum recently published a list of UHD services that are “live” today numbering 18 VOD and 37 Live services with 8 in the US and 47 outside the US. Clearly, content will not be the weak link in UHD 4K market acceptance for much longer.
  • Geographic deployments — 4K is more widely deployed in Asia Pacific and Western Europe than in the U.S. today. But we see this as a massive opportunity since many people are traveling abroad and thus will be exposed to the incredible quality. They will then return home to question their service provider, why they had to travel outside the country to see 4K. Which means as soon as the planned services in the U.S. are launched, they will likely attract customer more quickly than we’ve seen in the past.

HDR adds WOW factor to 4K

High Dynamic Range (HDR) improves video quality by going beyond more pixels to increase the amount of data delivered by each pixel. HDR video is capable of capturing a larger range of brightness and luminosity to produce an image closer to what can be seen in real life. Show anyone HDR content encoded in 4K resolution, and it’s no surprise that content providers and TV manufacturers are quickly jumping on board to deliver content with HDR. Yes, it’s “that good.” There is no disputing that HDR delivers the “wow” factor that the market and consumers are looking for. But what’s even more promising is the industry’s overwhelmingly positive reaction to it. Read more here.

Beamr has been working with Dolby to enable Dolby Vision HDR support for several years now, even jointly presenting a white paper at SMPTE in 2014. The V.265 codec is optimized for Dolby Vision and HDR10 and takes into account all requirements for both standards including full support for VUI signaling, SEI messaging, SMPTE ST 2084:2014 and ITU-R BT.2020. For more information visit http://beamr.com/vanguard-by-beamr-content-adaptive-hevc-codec-sdk

Beamr is honored to have customers who are best in class and span OTT delivery, Broadcast, Service Providers and other entertainment video applications. From what we see and hear, studios are uber excited about HDR, cable companies are prepping for HDR delivery, Satellite distributors are building the capability to distribute HDR, and of course OTT services like Netflix, FandangoNow (formerly M-GO), VUDU, and Amazon are already distributing content using either Dolby Vision or HDR10 (or both). If your current video encoding workflow cannot fully support or adequately encode content with HDR, it’s time to update. Our V.265 video encoder SDK is a perfect place to start.

VR & 360 Video at Streamable Bitrates

360-degree video made a lot of noise in 2016.  YouTube, Facebook and Twitter added support for 360-degree videos, including live streaming in 360 degrees, to their platforms. 360-degree video content and computer-generated VR content is being delivered to web browsers, mobile devices, and a range of Virtual Reality headsets.  The Oculus Rift, HTC Vive, Gear VR and Daydream View have all shipped this year, creating a new market for immersive content experiences.

But, there is an inherent problem with delivering VR and 360 video on today’s platforms.  In order to enable HD video viewing in your “viewport” (the part of the 360-degree space that you actually look at), the resolution of the full 360 video delivered to you should be 4K or more.  On the other hand, the devices on the market today which are used to view this content, including desktops, mobile devices and VR headsets only support H.264 video decoding. So delivering the high-resolution video content requires very high bitrates – twice as much as using the more modern HEVC standard.

The current solution to this issue is lowered video quality in order to fit the H.264 video stream into a reasonable bandwidth. This creates an experience for users which is not the best possible, a factor that can discourage them from consuming this newly-available VR and 360 video content.  But there’s one thing we know for sure – next generation compression including HEVC and content adaptive encoding – and perceptual optimization – will be a critical part of the final solution. Read more about VR and 360 here.

Patent Pool HEVC Advance Announces “Royalty Free” HEVC software

As 4K, HDR, VR and 360 video gathers steam, Beamr has seen the adoption rate moving faster than expected, but with the unanswered questions around royalties, and concerns of who would shoulder the cost burden, distributors have been tentative. The latest move by HEVC Advance to offer a royalty free option is meant to encourage and accelerate the adoption (implementation) of HEVC, by removing royalty uncertainties.

Internet streaming distributors and software application providers can be at ease knowing they can offer applications with HEVC software decoders without incurring onerous royalties or licensing fees. This is important as streaming app content consumption continues to increase, with more and more companies investing in its future.

By initiating a software-only royalty solution, HEVC Advance expects this move to push the rest of the market i.e. device manufacturers and browser providers to implement HEVC capability in their hardware and offer their customers the best and most efficient video experience possible.

 

>> 2017 Predictions

Mobile Video Services will Drive the Need for Content-adaptive Optimization

Given the trend toward better quality and higher resolution (4K), it’s more important than ever for video content distributors to pursue more efficient methods of encoding their video so they can adapt to the rapidly changing market, and this is where content-adaptive optimization provides a massive benefit.

The boundaries between OTT services and traditional MSO (cable and satellite) are being blurred now that all major MSOs include TVE (TV Everywhere streaming services with both VOD and Linear channels) in their subscription packages (some even break these services out separately as is the case with SlingTV). And in October, AT&T CEO Randall Stephenson vowed that DirecTV Now would disrupt the pay-TV business with revolutionary pricing for an  Internet-streaming service at a mere $35 per month for a package with more than 100 channels.

And get this – AT&T wireless is adopting the practice of “zero rating” for their customers, that is, they will not count the OTT service streaming video usage toward the subscriber’s monthly data plan. This represents a great value for customers, but there is no doubt that it puts pricing pressure on the operational side of all zero rated services.

2017 is the year that consumers will finally be able to enjoy linear as well as VOD content anywhere they wish even outside the home.

Beamr’s Contribution to MSOs, Service Providers, and OTT Distributors is More Critical Than Ever

When reaching to consumers across multiple platforms, with different constraints and delivery cost models, Beamr’s content adaptive optimizer perfects the encoding process to the most efficient quality and bitrate combination.

Whether you pay by the bit delivered to a traditional CDN provider, or operate your own infrastructure, the benefits of delivering less traffic are realized with improved UX such as faster stream start times and reduced re-buffering events, in addition to the cost savings. One popular streaming service reported to us that after implementing our content-adaptive optimization solution their rebuffering events as measured on the player were reduced by up to 50%, while their stream start times improved 20%.

Recently popularized by Netflix and Google, content-adaptive encoding is the idea that not all videos are created equal in terms of their encoding requirements. Content-adaptive optimization complements the encoding process by driving the encoder to the lowest bitrate possible based on the needs of the content, and not a fixed target bitrate (as seen in traditional encoding processes and products).

A content-adaptive solution can optimize more efficiently by analyzing already-encoded video on a frame-by-frame and scene-by-scene level, detecting areas of the video that can be further compressed without losing perceptual quality (e.g. slow motion scenes, smooth surfaces).

Provided the perceptual quality calculation is performed at the frame level with an optimizer that contains a closed loop perceptual quality measure, the output can be guaranteed to be the highest quality at the lowest bitrate possible. Click the following link to learn how Beamr’s patented content adaptive optimization technology achieves exactly this result.

Encoding and Optimization Working Together to Build the Future

Since the content-adaptive optimization process is applied to files that have already been encoded, by combining an industry leading H.264 and HEVC encoder with the best optimization solution (Beamr Video), the market will be sure to benefit by receiving the highest quality video at the lowest possible bitrate and file size. As a result, this will allow content providers to improve the end-user experience with high quality video, while meeting the growing network constraints due to increased mobile consumption and general Internet congestion.

Beamr made a bold step towards delivering on this stated market requirement by disrupting the video encoding space when in April 2016 we acquired Vanguard Video – a premier video encoding and technology company. This move will benefit the industry starting in 2017 when we introduce a new class of video encoder that we call a Content Adaptive Encoder.

As content adaptive encoding techniques are being adopted by major streaming services and video platforms like YouTube and Netflix, the market is gearing up for more advanced rate control and optimization methods, something that fits our perceptual quality measure technology perfectly. This fact when combined with Beamr having the best in class HEVC software encoder in the industry, will yield exciting benefits for the market. Read the Beamr Encoder Superguide that details the most popular methods for performing content adaptive encoding and how you can integrate them into your video workflow.

One Year from Now…

In one year from now when you read our post summarizing 2017 and heralding 2018, what you will likely hear is that 2017 was the year that advanced codecs like HEVC combined with efficient perceptually based quality measures, such as Beamr’s, provide an additional 20% or greater bitrate reduction.

The ripple effect of this technology leap will be that services struggling to compete today on quality or bitrate, may fall so far behind that they lose their ability to grow the market. We know of many multi-service operator platforms who are gearing up to increase the quality of their video beyond the current best of class for OTT services. That is correct, they’ve watched the consumer response to new entrants in the market offering superior video quality, and they are not sitting still. In fact, many are planning to leapfrog the competition with their aggressive adoption of content adaptive perceptual quality driven solutions.  

If any one service assumes they have the leadership position based on bitrate or quality, 2017 may prove to be a reshuffling of the deck.

For Beamr, the industry can expect to see an expansion of our software encoder line with the integration of our perceptual quality measure which has been developed over the last 7 years, and is covered by more than 50 patents granted and pending. We are proud of the fact that this solution has been shipping for more than 3 years in our stand-alone video and photo optimizer solutions.

It’s going to be an exciting year for Beamr and the industry and we welcome you to join us. If you are intrigued and would like to learn more about our products or are interested in evaluating any of our solutions, check us out at beamr.com.

Content EverywhereHEVCOTTReduce BitrateUncategorized

Can we profitably surf the Video Zettabyte Tsunami?

Posted on by Dror Gill

Two key ingredients are in place. But we need to get started now.

In a previous post, we warned about the Zettabyte video tsunami – and the accompanying flood of challenges and opportunities for video publishers of all stripes, old and new. 

Real-life tsunamis are devastating. But California’s all about big wave surfing, so we’ve been asking this question: Can we surf this tsunami?

The ability to do so is going to hinge on economics. So a better phrasing is perhaps: Can we profitably surf this video tsunami?

Two surprising facts came to light recently that point to an optimistic answer, and so we felt it was essential to highlight them.

1. The first fact is about the Upfronts – and it provides evidence that 4K UHD content can drive growth in top-line sales for media companies.

The results from the Upfronts – the annual marketplace where networks sell ad inventory to premium brand marketers – provided TV industry watchers a major upside surprise. This year, the networks sold a greater share of ad inventory at their upfront events, and at higher prices too. As Brian Steinberg put it in his July 27, 2016 Variety1 article:

“The nation’s five big English-language broadcast networks secured between $8.41 billion and $9.25 billion in advance ad commitments for primetime as part of the annual “upfront” market, according to Variety estimates. It’s the first time in three years they’ve managed to break the $9 billion mark. The upfront finish is a clear signal that Madison Avenue is putting more faith in TV even as digital-video options abound.”

Our conclusion? Beautiful, immersive content environments with a more limited number of high-quality ads can fuel new growth in TV. And 4K UHD, including the stunning impact of HDR, is where some of this additional value will surely come from.

Conventional wisdom is that today’s consumers are increasingly embracing ad-free SVOD OTT content from premium catalogs like Netflix, even when they have to pay for it. Since they are also taking the lead on 4K UHD content programming, that’s a great sign that higher value 4K UHD content will drive strong economics. But the data from the Upfronts also seems to suggest that premium ad-based TV content can be successful as well, especially when the Networks create immersive, clutter-free environments with beautiful pictures. 

Indeed, if the Olympics are any measure, Madison Avenue has received the message and turned up their game on the creative. I saw more than a few head-turning :30-second spots. Have you seen the Chobani ads in pristine HD? They’re as powerful as it gets.2

Check out this link to see the ads.

2. The second fact is about the operational side of the equation.

Can we deliver great content at a reasonable cost to a large enough number of homes?  On that front, we have more good news. 

The Internet in the United States is getting much faster. This, along with advanced methods of compression including HEVC, Content Adaptive Encoding and Perceptual Quality Metrics, will result in a ‘virtual upgrade’ of existing delivery network infrastructure. In particular, Ookla’s Speedtest.net published data on August 3, 2016 contained several stunning nuggets of information. But before we reveal the data, we need to provide a bit of context.

It’s important to note that 4K UHD content requires bandwidth of 15 Mbps or greater. Let’s be clear, this assumes Content Adaptive Encoding, Perceptual Quality Metrics, and HEVC compression are all used in combination. However, in Akamai’s State of the Internet report released in Q1 of this year, only 35% of the US population could access broadband speeds of 15 Mbps.

(Note: We have seen suggestions that 4K UHD content requires up to 25 Mbps. Compression technologies improve over time and those data points may well be old news. Beamr is on the cutting edge of compression and we firmly believe that 10 – 15 Mbps is the bandwidth needed – today – to achieve stunning 4K UHD audio visual quality.)

And that’s what makes Ookla’s data so important. Ookla found that in the first 6 months of 2016, fixed broadband customers saw a 42% year-over-year increase in average download speeds to a whopping 54.97 Mbps. Even more importantly, while 10% of Americans lack basic access to FCC target speeds of 25 Mbps, only 4% of urban Americans lack access to those speeds. This speed boost seems to be a direct result of industry consolidation, network upgrades, and growth in fiber optic deployments.

After seeing this news, we also decided to take a closer look at that Akamai data. And guess what we found? A steep slope upward from prior quarters (see chart below).

To put it back into surfing terms: Surf’s Up!
time-based-trends-in-internet-connection-speeds-and-adoption-rates

References:

(1) “How TV Tuned in More Upfront Ad Dollars: Soap, Toothpaste and Pushy Tactics” Brian Steinberg, July 27, 2016: http://variety.com/2016/tv/news/2016-tv-upftont-networks-advertising-increases-1201824887/ 

(2)  Chobani ad examples from their YouTube profile: https://www.youtube.com/watch?v=DD5CUPtFqxE&list=PLqmZKErBXL-Nk4IxQmpgpL2z27cFzHoHu

Uncategorized

Net Neutrality Means Bye Bye HOV Lanes

Posted on by Dror Gill

What started as a rant from comedian John Oliver calling for support of an open internet, has now developed into a monumental philosophical debate riddled with controversy that is centered upon the Federal Communications Commission’s (FCC) attempt to reclassify broadband based on its role in modern day life. For the past two decades, the Internet has been unregulated and free, but that’s about to change and with this change, a light is shined on the importance of optimization solutions for bandwidth savings.

What is net neutrality?

In broad terms, net neutrality is the idea or principle of having the Internet equally accessible for all. Advocates proclaim it’s meant to stop internet service providers including mobile wireless services from discriminating against any application or content that is shared over their networks. It is the idea of keeping the internet free of “fast lanes” that give preferential treatment to companies and websites willing to pay more to deliver their content faster. It, therefore, prohibits internet service providers from putting everyone else in “slow lanes” that are said to discriminate against the “smaller” companies that cannot afford to pay a premium.

Broadband is now considered a utility

The decision by the United States Court of Appeals for the District of Columbia Circuit maintained the FCC’s view that broadband Internet should be reclassified as a utility, telecommunications service as opposed to information services, giving them more power to enforce stricter rules and regulations in a once free market.  Essentially, this ruling solidifies that the internet has become as important to the American way of life as the telephone and forces providers to adhere to some of the same regulations faced by phone companies.

But what’s the point? In theory, the reclassification will increase competition among service providers, which as we learned in economics class, should cut costs to consumers, and raise the quality of service as well as drive innovation. But opposing forces claim this new open policy will hurt the consumer as prices increase due to lack of incentive for companies to build out and update their networks as needed to meet consumer demand, which is growing exponentially due to video. In fact, Cisco reported that by 2019, video will represent 80% of all global consumer Internet traffic. 

Whatever side you fight for, in the end, what net neutrality comes down to is the commoditization of the internet… bits.  

Bandwidth savings is now a necessity making optimization a requirement

As a service provider, now that you can no longer pay to get your bits (content) delivered faster or more efficiently, this means everyone is on the same playing field.  We are all stuck in the same traffic jam.  Couple this fact with the increasing consumer demands for a reliable and stable experience and in one of the most infamous phrases ever uttered, “Houston we have a problem.”  But the good news is that companies have developed solutions to improve encoding efficiency which can lead to additional bitrate reductions of 20-40% without a perceptible change in viewing quality.

Optimization solutions, like Beamr’s JPEGmini and Beamr Video, have already become well entrenched in the fabric of media workflows, especially video, and are likely to grow even more in their importance over the years to come. Why? Because the name of the game is getting content to your consumer with the best quality and highest user experience possible, and now the only way to do that is to increase file efficiency by optimizing your file sizes to conserve bandwidth so you can cut through the clutter.

For example, with an image intensive website, JPEGmini can cut the file size of your photos by up to 80% without compromising quality, whereby giving your user faster page load times.  Which by the way, Google recently made a change in their ranking algorithms to weight website searches based on page load time and mobile performance.  In short, if your site is slow, you are going to see other sites rank more highly on Google search engine results pages.  For video, Beamr Video is able to reduce H.264 and HEVC bitrates by 20-40% without any compromise in quality, whereby enabling a smoother streaming experience. With the ruling as it stands, only companies that can lower file size and save bandwidth will break through congested networks to deliver content that meets consumer expectations for entertainment anytime, anywhere and on any device.

BONUS: Four secrets to keep in mind when selecting a video bitrate reduction solution

  1. Automation is your friend.  Though an encoder in the hands of a well-trained expert can yield amazingly small files with good quality, for nearly all streaming service providers, automation will be required to deploy a bitrate reduction solution at scale.  And anything less than scale will not have a meaningful impact. For this reason, you should select a product that can give you the ability to automate the analysis and workflow implementation.  
  2. Solutions that work at the frame level will give you the best overall video quality.  Products that analyze video to determine an optimum fixed bitrate cannot provide the level of granularity that you will want to have as methods that perform this analysis at the frame or scene level.  Beamr Video operates the analysis and compression parameter settings on a frame basis and in a closed loop, to ensure that the optimal allocation of bits per frame is achieved.  No other method of optimization can achieve the same level of quality across a disparate video library as Beamr Video.
  3. Leave neural networks to Google, you want a solution that uses a perceptual quality metric to validate video quality.  A sophisticated quality measure that is closely aligned to human vision will deliver a more accurate result across an entire library of content.  Quality measures such as PSNR and SSIM provide inconsistent results and cannot be used with a high degree of accuracy.  These quality measures have some application for classifying content, but they are less useful as a mechanism to drive an encoder to achieve the best quality possible.
  4. Guaranteed quality, is there any other choice?  Sophisticated encoder configurations that use CRF (Constant Rate Factor)  on first look can appear to be viable, however, without a quality verification step that is post-encode the final result will likely contain degradation or other transient quality issues.  If the product or technique that you have selected does not operate in a closed loop or contain some method for verifying quality, the potential to introduce artifacts will be unacceptably high.  

 

For more information or to receive a free trial of Beamr Video, please visit http://beamr.com/product  

 

Uncategorized

Dolby Vision and HEVC, an Introduction

Posted on by Dror Gill

Notice that some material from this post appeared originally in the article, “Integrating HEVC Video Compression with a High Dynamic Range Video Pipeline,” by Raul Diaz, Sam Blinstein, and Sheng Qu, SMPTE Motion Imaging Journal, 125 (1): 14-21, January/February 2016.

You can download the original paper here.

Advances in video capture and display are moving the state of the art beyond higher resolution to include high dynamic range or HDR.  Improvements in video resolution are now informed by advances in the area of color gamut and dynamic range.  New displays shipping today are capable of reproducing a much wider range of colors and brightness levels than can be represented by the video content being produced for streaming, download, digital broadcast, and even Blu-ray.  

By combining technical advances such as higher spatial resolution, larger temporal resolution, and higher dynamic range, distributors looking for differentiated offerings will be able to provide an improved video experience with the existing workflow and technology that is available today.  Content providers are taking notice of technology advances to deliver high-resolution content to viewers, and the newest video compression standard HEVC enables them to compress high-resolution digital video more efficiently for broadcast, mobile and Internet delivery enable exciting experiences such as HDR.

Higher frame rates are now widely supported in modern video compression and delivery standards, yet content producers and distributors largely use workflows and grading protocols that target the dynamic range standards set by technology from the early and mid 20th century. With modern cinema distribution moving to digital, and given the wholesale replacement of cathode ray tube displays by flat panels, the technology is now available to display and deliver a much wider dynamic and color range viewing experience. As a result, an opportunity exists to demonstrate high resolution, high frame rate, and high dynamic range in a single viewing environment.

Is your codec HDR ready?

The question then, is will the codec you are using be able to support HDR?  Native HDR support can be found in HEVC where extensions and new features were added to the standard in July 2014, many of which specifically address high dynamic range. Color profiles previously limited to sample sizes of 8 and 10 bits (named Main and Main10) were expanded. And because chroma subsampling was limited to 4:2:0 this created a significant restriction of chroma relative to luminance by a factor of 4 to 1, but the new HEVC range extensions can now support bit depths of 12, 14, and even 16 bits, along with lower subsampling of chroma to 2:1 (4:2:2) and 1:1 (4:4:4).

SEI messages have also been added to accommodate signal conversions from one color space to another and to facilitate conversions across varying dynamic ranges. With these tools, an end-to-end system for any arbitrary dynamic range and color gamut can be built, and transformations between disparate dynamic range systems implemented.

However, a challenging element is the need to address the widely varying dynamic range of display devices, including legacy systems. Efforts have been underway to develop a solution that can support the future while providing playback compatibility with legacy devices from various organizations and companies, one of which is Dolby Laboratories, who has developed a backwards compatible approach for HDR.

Dolby Vision

One example of this framework is Dolby Vision, the result of nearly a decade of research and development in dynamic range and color gamut scalability. Dolby Vision HDR exists in two flavors: dual layer and single layer.

The original architecture was a dual layer HDR schema, designed as a combination of a base layer containing video data constrained to current specifications, with an optional enhancement layer carrying a supplementary signal along with the Dolby Vision metadata.  This architecture implements a dual layer HDR workflow where the secondary data is an extension of a traditional signal format that provides backwards compatibility with legacy SDR devices (TVs, Set-top-boxes).  But it has a disadvantage of requiring two decoders for the client device to render video.

Single Layer Dolby Vision HDR was introduced later as an alternative to the original approach and to address competitive single layer technologies. It uses the similar metadata as a single layer, but has only one HDR video stream multiplexed with metadata. With this approach, Dolby loses compatibility with legacy devices, but the upside is that it is highly cost-effective for new deployments and operators as some consumer playback devices and TV’s can be upgraded to support single layer Dolby Vision HDR after they were originally sold.

When HDR and wide color gamut is supported on a given device, the two layers are simultaneously decoded and combined using metadata coefficients to present a highly engaging and compelling HDR viewing experience. On legacy equipment, the enhancement layer and metadata are ignored (or not transmitted) and the traditional viewing experience is unaffected. In this way, the dual layer HDR system offers multiple touch-points in the video pipeline to transition without the need for strict synchronization across multiple technologies. In contrast, single Layer HDR requires a hardware and software upgrade to the display device which is not always possible or easily achieved.

As HDR capable reference monitors become more cost effective and available, creative directors will be in a better position to review and master content in the higher range that the video was captured. Such a shift will preserve a much wider range of luminance and color, and invert the mastering stage bottleneck of grading to the lowest common denominator.

Naturally, a dual layer HDR architecture adds a certain complexity to the distribution of video content where both the encoding and decoding stages must handle the secondary layer Dolby requires for HDR. Also, color volume considerations of traditional SDR signals in the new HDR framework requires more than the traditional 8-bit signal.

By using a modular workflow in which augmented encoding and decoding systems can be integrated separately, introducing auxiliary metadata paths to support higher bit-depth requirements of a dual layer HDR system can leverage new video compression standards while simultaneously offering backwards compatibility with legacy equipment and signals.

From experience in production environments, offline encoded 4K HEVC content requires a bitrate between 10 and 15 Mbps to generate quality that is in line with viewer expectations. AVC has multiple inherent limitations relative to HEVC. These limitations make it particularly difficult for AVC to achieve acceptable quality levels at bitrates that can be streamed and delivered effectively to most viewers, particularly at 4K content bitrates.

Dolby Vision adds approximately 20% to 30% additional bitrate load for the higher dynamic range data, making the challenges even more severe for AVC. For 4K content combined with high dynamic range information, the HEVC video compression standard presents the optimum solution to generate acceptable video quality at the bitrates that are needed.

For traditional HD content, AVC video compression has been used successfully for many years for broadcast, the Internet, and mobile video delivery. By using HEVC video compression, this HD content can be compressed at least 30% more efficiently than with AVC video compression.  For high dynamic range HD content encoded with a dual layer HDR system, HEVC video compression can generate a bitrate that is equal to or less than the bitrate needed for a non-HDR HD bitstream encoded using AVC.

Consequently, any video delivery system that can deliver non-HDR HD content today with AVC can also distribute HDR HD content using HEVC video compression without altering the delivery infrastructure.

HEVC offers two primary advantages for HDR content delivery:

  1. An effective and viable method to deliver 4K content,
  2. Bandwidth compatibility with existing video delivery infrastructure to deliver HDR HD content.

If you have more questions about HEVC and encoding for next-generation high dynamic range solutions such as Dolby Vision, email info@beamr.com to learn how Beamr supports some of the largest streaming distributors in the world to deliver high-quality 4K UHD streams with Dolby Vision.

Uncategorized

Using Beamr’s V.265 HEVC Encoder to Generate MPEG-DASH Compliant Streams

Posted on by Dror Gill

Recent developments in video encoding and streaming technology have come together to supply two major tools to optimize the delivery of synchronized video streams across multiple devices.

The first development is the next generation video coding standard HEVC, which offers significant compression efficiency gains over AVC. And the second is MPEG-DASH, which gives key advantages over HLS, in managing adaptive bitrate streaming of synchronized resolutions and profiles across varying network bandwidths. The combination of HEVC and MPEG-DASH supports higher quality video delivery over limited bandwidth networks.

Apple’s HLS ABR standard is in broad use today, but MPEG-DASH is not that new having been standardized before HEVC, and being applied in the distribution of AVC (H.264) content. MPEG-DASH is codec and media format agnostic, and the magic of MPEG-DASH is that it splits content into a collection of file segments, each containing a short section of the content. The MPEG-DASH standard defines guidelines for implementing interoperable adaptive streaming services, and it describes specific media formats for use with the ISO Base Media File Format (e.g., MP4) or MPEG-2 Transport Stream containers, making the integration of HEVC into an MPEG-DASH workflow possible within existing standards.

MPEG-DASH targets OTT delivery and CDNs but is also finding a home in broadcast and MSO/MVPD environments as a replacement for MPEG-2 TS-based workflows. Through the exhaustive descriptions available in the MPD, MPEG-DASH clients can determine which media segments best fit their user’s preferences, device capability, and network conditions, guaranteeing a high-quality viewing experience and support for next-generation video services.

Early in the development of HEVC, Beamr realized the need for true adaptive bitrate (ABR) Multistreaming support in HEVC as a tool for content preparation for multistreaming services. In Version 3.0 of our HEVC encoder SDK, V.265, we introduced several API extensions supporting multistream encoding. This architecture allows for the encoding of a single master source video into multiple, GOP-aligned streams of various resolutions and bitrates with a single encoder instance. Moreover, newly exposed encoder input settings allow for the specification of individual settings and flags for each of the streams.

Supporting multiple streams of varying resolutions, bitrates, and settings from a single source in a single encoder instance, which guarantees GOP alignment and offers computational savings across shared processes, is critical for reliable ABR encoding/transcoding performance. Beamr’s V.265 encoding SDK offers service providers the opportunity to combine the advancements of HEVC coding with the versatility of MPEG-DASH.

This functionality offers two significant advantages for developing a multistreaming workflow. First, the architecture guarantees that the multiple streams generated by the encoder are 100% GOP-aligned, an essential requirement for any multistreaming workflow. Second, it simplifies the encoding process to a single input source and encoder instance, reducing command-and-control and resource management.

Despite the performance savings available from multistreaming as a result of the shared computing resources that can be leveraged, our implementation yields optimally synchronized streams that are nearly impossible to generate with separate encoders. Beamr’s Multistreaming capability positions V.265 as highly unique, a contributing factor to why V.265 is in use by leading OTT service providers who use our solution as the basis for encoding their 4k HDR and 1080p ABR profiles.