Encoder Applications

AVCBeamr 4Beamr 5Encoder ApplicationsH.264H.265HDRHEVC/VP9Uncategorizedx264x265

Automatically upgrade your video content to a new and improved codec

Posted on by Tamar Shoham

Easy & Safe Codec Modernization with Beamr using Nvidia GPUs 

Following a decade where AVC/H.264 was the clear ruler of the video encoding world, the last years have seen many video coding options battling to conquer the video arena. For some insights on the race between modern coding standards you can check out our corresponding blog post.

Today we want to share how easy it can be to upgrade your content to a new and improved codec in a fast, fully automatic process which guarantees the visual quality of the content will not be harmed. This makes the switchover to newer encoders a smooth, easy and low cost process which can help accelerate the adoption of new standards such as HEVC and AV1. When this transformation is done using a combination of Beamr’s technology with the Nvidia NVENC encoder, using their recently released APIs, it becomes a particularly cutting-edge solution, enjoying the benefits of the leading solution in hardware AV1 encoding.

The benefit of switching to more modern codecs lies of course in the higher compression efficiency that they offer. While the extent of improvement is very dependent on the actual content, bitrates and encoders used, HEVC is considered to offer gains of 30%-50% over AVC, meaning that for the same quality you can spend up to 50% fewer bits. For AV1 this increase is generally a bit higher.. As more and more on-device support is added for these newer codecs, the advantage of utilizing them to reduce both storage and bandwidth is clear. 

Generally speaking, performing such codec modernization involves some non-trivial steps. 

First, you need to get access to the modern encoder you want to use, and know enough about it in order to configure the encoder correctly for your needs. Then you can proceed to encoding using one of the following approaches.

The first approach is to perform bit-rate driven encoding. One possibility is to use conservative bitrates, in which case the potential reduction in size will not be achieved. Another possibility is to set target bitrates that reflect the expected savings, in which case there is a risk of losing quality. For example, In an experimental test of files which were converted from their AVC source to HEVC, we found that on average, a bitrate reduction of 50% could be obtained when using the Beamr CABR codec modernization approach. However, when the same files were all brute-force encoded  to HEVC at 50% reduced bitrate, using the same encoder and configuration, the quality took a hit for some of the files.

 

Source AVC
Beamr HEVC codec modernization
HEVC Brute-Force encode

This example shows the full AVC source frame on top, with the transcodes to HEVC below it. Note the distortion in the blind HEVC encode, shown on the left, compared to the true-to-source video transformed with CABR on the right.

The second approach is to perform the transcode using a quality driven encode, for instance using the constant QP (Quantization Parameter) or CRF (Constant Rate Factor) encoding modes with conservative values, which will in all likelihood preserve the quality. However, in this case you are likely to unnecessarily “blow up” some of your files to much higher bitrates. For example, for the UGC content shown below, transcoding to HEVC using a software encoder and CRF set to 21 almost doubled the file size.

Yet another approach is to use a trial and error encode process for each file or even each scene, manually verifying that a good target encoding setup was selected which minimizes the bitrate while preserving the quality. This is of course an expensive and cumbersome process, and entirely unscalable.

By using Beamr CABR this is all done for you under the hood, in a fully automatic process, which makes optimized choices for each and every frame in your video, selecting the lowest bitrate that will still perfectly preserve the source visual quality. When performed using the Nvidia NVENC SDK with interfaces to Beamr’s CABR technology, this transformation is significantly accelerated and becomes even more cost effective. 

The codec modernization flow is demonstrated for AVC to HEVC conversion in the above high-level block diagram. As shown here, the CABR controller interacts with NVENC, Nvidia’s hardware video encoder, using the new APIs Nvidia has created for this purpose. At the heart of the CABR controller lies Beamr’s Quality Measure, BQM, a unique, patented, Emmy award winning perceptual video quality measure. BQM has now been adapted and ported to the Nvidia GPU platform, resulting in significant acceleration of the optimization process .  

The Beamr optimization technology can be used not only for codec modernization, but also to reduce bitrate of an input video, or of a target encode, while guaranteeing the perceptual quality is preserved, thus creating encodes with the same perceptual quality at lower bitrates or file sizes. In any and every usage of the Beamr CABR solution, size or bitrate are reduced as much as possible while each frame of the optimized encode is guaranteed to be perceptually identical to the reference. The codec modernization use case is particularly exciting as it puts the ability to migrate to more efficient and sophisticated codecs, previously used primarily by video experts, into the hands of any user with video content.

For more information please contact us at info@beamr.com 

AOM AV1Apple joins AOMEncoder ApplicationsHEVC

HEVC today. AV1 tomorrow?

Posted on by Dror Gill

In case you missed it, Apple just snuck a little surprise into the first few days of 2018 in the form of their name appearing on the Alliance for Open Media (AOM) website as a Founding Member. As an ardent HEVC supporter, some may be shocked at this move by Apple. Blog post detailing Apple’s HEVC announcement. Now let’s see what it means for HEVC. 

As a codec engineering company, Beamr has invested heavily in our HEVC implementation and we are proud of the best in class customers who are using it around the world to distribute video that is higher quality and up to 50% smaller than the H.264 version.

This means we do have a “vested” interest in HEVC being successful. At the same time, whether VP9 or AV1, we are always tracking the development of new codec technology so that we are in the strongest position to bring our extensive codec development resources to bear on market leading solutions.

Though we have an interest in HEVC becoming successful, we have invested resources and continue to do so, in order to understand AV1 in the areas of market readiness and licensing preparedness including IP questions, playback support and more.

In this article I will share the Beamr perspective that Apple joining the AOM reinforces the possibility that AV1 will be the successor to HEVC. However, with 1 billion HEVC enabled end points in the market, HEVC has legs for many years before a sufficiently large AV1 ecosystem will be built.

This position is also in alignment with many of our customers who are serving hundreds of millions of end users and must make codec decisions based on streams they can reliably deliver today.

After all, speculating on what may be coming in the future is not a luxury most of the industry enjoys because if they bet wrong, it could impact tens of millions of users negatively. There is a huge difference between advanced technology development (e.g. what happens in the lab) and the realities of production (that which generates revenue).

Once the AOM locks down the AV1 spec, you can expect many shootouts and comparisons with HEVC to be published. But let’s take a look at how HEVC compares to AV1, given what we know now.

AV1 Readiness compared to HEVC.

HEVC was ratified in 2013 while the AV1 bitstream was set to be frozen in Q1 2017, yet even now the AV1 bitstream has not been completed. Developing software timelines, committing to them, and then meeting them, is far from an exact science. Thus the delay is not completely the fault of the AOM development community since it is endemic to the software development lifecycle. Innovation is difficult to schedule. The point is, AV1 will be ready when it is ready. Which means commercial plans that hinge on the delivery of AV1 before 2020 or 2021 could be at risk given the uncertainty of when the standard will be ratified.

AV1 Compression Efficiency compared to HEVC.

HEVC is recognized to be 40-50% more efficient than AVC (H.264), and AV1 is hoped to be up to 30% more efficient than HEVC (H.265).

However, while HEVC’s compression efficiency has already been reached by advanced encoder implementations such as Beamr 5, AV1’s 30% efficiency claim over HEVC has not been proven outside of an extremely limited (small) set of files.

In any case any improvement can be validated only after the spec is final and the tools included in AV1 are decided upon. At that point the race to realize these gains will start, balancing the computing resources needed and maturing the rate control algorithms. But just as HEVC did not reach its planned 50% efficiency in the first release, taking multiple years to achieve, the AOM developers will need to work very hard for the next 2 to 3 years before significant gains over HEVC will be seen.

AV1 Royalty and IP constraints compared to HEVC.

There are three HEVC Patent pools which license the technology used in HEVC implementations: MPEG-LA, HEVC Advance, and Velos Media. Both MPEG-LA and Velos Media do not charge license fees for content distribution (See the MPEG-LA HEVC License Summary and the Velos Media FAQ), and HEVC Advance does not charge a license fee for free content distribution, such as public broadcasts and ad-funded commercial broadcasts (see page 3 of the HEVC Royalty Rates document).

Even Technicolor, that licenses its HEVC patents outside of the 3 patent pools, has publicly declared that they will not charge license fees from content providers. In addition, royalty schedules are being (have been) amended down, and it seems the Patent pools are aware that a more friendly approach is needed.

AV1 cannot guarantee a royalty-free offer.

Yes, that’s correct, I said it! Now here’s why.

While AV1 claims to be royalty-free, many industry players have missed the fact that the Alliance for Open Media does not provide indemnification to companies who use AV1 against patent claim violations.

Since some of the algorithms used in AV1 bear a resemblance to corresponding H.264 and HEVC algorithms, there is some probability that the IP in AV1 could infringe on AVC and/or HEVC Patents. In fact, delays to the ratification of the AV1 standard might well be due to legal teams who are examining the final algorithms exactly for these cases.  

To be fair, IP questions are hardly ever cut and dried, and there are many unknowns and “what-if’s” to be discussed. But the lack of clarity regarding the AV1 IP situation, and the fact that AOM is not offering indemnification for IP infringement, makes the “royalty free” claim at this point more of a wish than a solid fact.

If you are still not convinced that AV1 offering a royalty-free codec could be problematic, consider that for VP8 and VP9 Google needed to license the H.264 patents from MPEG-LA. If an infringement action is identified with AV1, and if the courts rule in the plaintiff’s favor, the legal exposure will be of the magnitude that headlines will be penned and stock prices hammered. Velos Media, one of the 3 HEVC patent pools, has already warned in its FAQ:

“As it relates to royalties, we know that VP9 incorporates patented technologies, including some of the patents being licensed by Velos Media for HEVC. And, while AV1 has not yet been publicly released, it may also incorporate patented technology from many parties.”

Try slipping that little disclaimer by a corporate IP attorney! When a licensing body directly references a new technology as being possibly infringing, it is worth paying attention to.

Let’s look at why the AOM members feel it’s so important to have a royalty-free platform.

  • Royalties are a pain, that is, when you are the party that needs to pay. Not only do you need to factor this added cost into your business model but in some cases, the tracking and reporting burden by itself represents a real difficulty and may limit certain business models from being feasible.
  • You can never know what patent holder’s next demand will be. The HEVC fragmented IP pools and shifting fee structures prove the point. The AOM is absolutely correct in wanting to address this with AV1.
  • Confusion over who to pay. The lack of clarity overpayments is delaying adoption even further. It seems AOM members are seeking control over a critical component of their technology that is not royalty free.

While all these arguments make sense, AOM cannot guarantee its users that this is a royalty-free codec; all they can suggest is that AOM members will not ask for royalties. But the thousands of patent holders for block-based codecs that are not a part of AOM are still out there, and the generosity of AOM could come at their expense.

For an interesting analysis on the validity and value of the MPEG-LA and HEVC Advance HEVC patent pools, you will want to read this Unified Patents article as it provides a perspective on how the courts look at the patents that are contained in a pool. In short, it’s probably not a wise legal move to assume that the AOM has everyone adopting AV1 “covered.”

AV1 Encoding CPU Performance compared to HEVC.

HEVC encoding CPU performance is advancing at a rapid rate. At IBC 2017 Beamr demonstrated six simultaneous 10-bit 4Kp60 live channels being encoded on a single Intel Xeon Scalable Platinum 8180 dual-socket server.

This encoding speed is in contrast to AV1, which has not been optimized, but is currently running about 100 times slower than real-time on a single server. Aside from the fact that AV1 is not available for low latency live encoding workflows, the operational cost delta of running an AV1 encoding service versus HEVC is staggeringly higher for AV1.  

Download the Intel solution guides detailing Beamr HEVC codec SDK performance and applications on Intel processors.

Beamr’s HEVC encoder has been under active development for more than five years, and our CPU performance has been consistently improved by way of algorithmic and code optimizations. AV1 developers will bring improvements to the encoding speed, but it will most certainly follow the same development trend of every codec before it, including HEVC.

In other words, it can only happen over a period of years, and only if a group of dedicated engineers focuses on it day and night. Optimizing a codec is not a hobby. But even in the end after it is fully optimized AV1 will be slower because of the added mathematical complexity needed for it to achieve higher efficiency. 

Video distributors with capex, opex, or physical space constraints will find the bitrate efficiency gains of AV1 will come with a very high operational cost. This Jan Ozer article from Streaming Media provides further context on AV1 performance.

AV1 Playback and Decoding Performance compared to HEVC.

HEVC hardware decoder support exists today in more than 1 billion devices spanning the most popular computing and mobile operating systems in the market like iOS, macOS, Android, and Windows. And in addition, low power hardware implementations for HEVC exist on Intel and ARM-based chips, as well as hundreds of millions of SoC’s shipping in CE devices such as TV’s, media players and game consoles. HEVC is a de-facto standard in all UHD TV’s found in the market.

According to the Consumer Technology Association (CTA) between 2014 and thru 2017, somewhere in the neighborhood of 175 million UHD TV’s will have been sold. Source: CTA 2017 presentation, data from GfK

This is an impressive HEVC footprint, and it’s only going to grow stronger. In contrast, the only AV1 playback environment available today is an early alpha implementation of the Mozilla browser Nightly build.  

As the AV1 spec is not yet locked by the AOM it’s no wonder that when we reached out to the best-known silicon vendors who are supplying the media processing and video decoding chips to the most popular media player vendors, game console makers and TV OEM’s, everyone reported without exception that they cannot begin planning AV1 support in the absence of a ratified spec.

AV1 support in silicon is a minimum of 24 months out (Q1 2020). But silicon is just that, silicon. Chips have to be designed and integrated into consumer products before the advanced capabilities that chip vendors are including can be available. But, this is a chicken and egg situation. Afterall, why would a CE company go through the extra engineering cycles and increase their BOM to support a codec that is yet to be adopted across the ecosystem?

Should Apple influence your codec adoption decision?

With Apple’s commitment to HEVC, it’s a mystery of what their intentions are in joining the AOM. But we do know this – Apple has a solid HEVC roadmap with a vertically integrated HEVC video encoding, distribution and playback technology stack across all devices.

This means that today, you can reach the Apple ecosystem with H.264 and HEVC.

For video distributors looking for smaller bandwidth footprints, opting out of HEVC means they will lose a whopping 54% of the North American mobile market assuming the selected codec isn’t supported by Apple. This is hardly a prospect that any executive or encoding head will agree to, which means the question of whether to support AV1 or HEVC could come down to compatibility.  

Any video service unable to match the performance of HEVC on Apple, will have a difficult time competing with services delivering high quality 1080p HD video at bitrates well under 2 Mbps. HEVC is available today across the all too important Apple ecosystem, while AV1 is not supported (today).  

HEVC is the codec for today. Is AV1 the codec for the future?

Remember how I said that Beamr is actively tracking all new video technology and codec developments and that this applies to AV1? Well, we do believe that AV1 could be a factor at some point in the future.

There are those who always wait for next years model. The trouble with this approach is that by delaying, you miss out on technology leaps that could have afforded a significant advantage to your company. This is especially true with the decision to move ahead or hold on adopting HEVC in anticipation that AV1 will be cheaper and provide an added efficiency benefit. 

The reality is that HEVC is able to reach 40% to 50% efficiency gains over H.264 today. These aren’t theoretical numbers or only possible on a limited set of content. Beamr has customers distributing content around the world, and enjoying bitrate savings in this range. Even Apple in their WWDC2017 announcements of HEVC, used the numbers 40% and 50% savings when talking about their decision to adopt the HEVC standard.

Ask yourself, what is the opportunity cost incurred by continuing with H.264 for the next 24 to 36 months as you wait for a relatively small AV1 playback footprint to emerge?

This is why most of the industry operating a commercial service is opting to realize the benefits of HEVC today while keeping tabs on the development of AV1 (for the future).

Still not sure…

Consider that Amazon Prime and Netflix are both members of AOM. And both are active in AV1 development and testing, yet Amazon and Netflix are users of HEVC.

Why would they do this? It’s simple. HEVC serves them well by being compatible with more than 1 billion devices and enabling premium video experiences at bitrates that are 40 to 50% less than H.264. Now, one may think that HEVC is only being used for 4K content by these services, but we know that Amazon is using HEVC in emerging markets for lower resolutions.

But what about Google, they are Founding members of the AOM and ardent supporters of alternative codecs? It’s interesting to note that Google supports HEVC in Chromecast which is clearly required for content services (some who are in the AOM like Netflix and Amazon) to stream 4K HDR video. But will Google use HEVC for their own services on a wider basis in the future? We do not know.

One situation in the market that must be faced by Google and YouTube is what will happen if Apple deprecates H.264, and mandates all apps that stream video to Apple devices leverage HEVC. Remember Flash? A similar situation occurred with HLS, the mandatory streaming protocol for Apple devices.  

Remember how I pointed out that Apple is vertically integrating with HEVC for video and HEIC (HEVC I-frame) for mobile image capture and display? I think now you can see how a codec selection decision by a vendor like Apple can move the entire industry.

Apple has chosen HEVC for production and any video distributor can encode in HEVC and transmit to a user with iOS 11 or macOS High Sierra and know that it will play perfectly.

HEVC is a robust standard that has broad support with extensive development from the largest encoding vendors in the industry. And HEVC is widely adopted on the device side with major services like Amazon, Apple, and Netflix using it now. HEVC is not going away.

Beamr’s view is that HEVC is the codec for today, and AV1 may possibly be the codec of tomorrow. But for sure, with HEVC, we can all enjoy more video and better quality as HEVC enables new applications, experiences, and innovations to be transmitted to users today.

4K TVBeamr 5Content-AdaptiveEncoder ApplicationsH.265HDRHEVCIBCMobile VideoOTTReduce Bitrate

How to deal with the tension on the mobile network – part 2 (VIDEO Interview)

Posted on by Dror Gill

In late July, I reported on the “news” that Verizon was throttling video traffic for some users. As usual, the facts around this seemingly punitive act were not fully understood, which triggered this blog post.

At IBC last month (September 2017), I was interviewed by RapidTV where much of the conversation was around the Apple news of their support for HEVC across the device ecosystem running iOS 11 and High Sierra. As I was reviewing this interview, it seemed natural to publish it as a follow up to the original post.

There is no doubt that mobile operators are under pressure as a result of the network crushing video traffic they are being forced to deliver. But the good news is that for those operators who adopt HEVC, they are going to enjoy significant bitrate efficiencies, possibly as high as 50%. And for many services, though they will chose to take some savings, this means they’ll be able to upgrade their resolutions to full 1080p while simultaneously improving the video quality they are delivering.

I hope you find this video insightful. Our team has a very simple evaluation offer to discuss with all qualified video services and video distributors. Just send an email to sales@beamr.com and we’ll get in touch with the details.

4K TVBroadcastContent EverywhereEncoder ApplicationsH.265HEVCLiveOTTReduce BitrateUncategorizedV.265 Codec SDKVR

2016 Paves the Way for a Next-Gen Video Encoding Technology Explosion in 2017

Posted on by Dror Gill

2016 has been a significant year for video compression as 4K, HDR, VR and 360 video picked up steam, paving the road for an EXPLOSION of HEVC adoption in 2017. With HEVC’s ability to reduce bitrate and file sizes up to 50% over H.264, it is no surprise that HEVC has transitioned to be the essential enabler of high-quality and reliable streaming video powering all the new and exciting entertainment experiences being launched.

Couple this with the latest announcement from HEVC Advance removing royalty uncertainties that plagued the market in 2016 and we have a perfect marriage of technology and capability with HEVC.

In this post we’ll discuss 2016 from the lenses of Beamr’s own product and company news, combined with notable trends that will shape 2017 in the advanced video encoding space.  

>> The Market Speaks: Setting the Groundwork for an Explosion of HEVC

The State of 4K

With 4K content creation growing and the average selling price of UHD 4K TVs dropping (and being adopted faster than HDTVs), 4K is here and the critical mass of demand will follow closely. We recently did a little investigative research on the state of 4K and four of the most significant trends pushing its adoption by consumers:

  • The upgrade in picture quality is significant and will drive an increase in value to the consumer – and, most importantly, additional revenue opportunities for services as consumers are preconditioned to pay more for a premium experience. It only takes a few minutes viewing time to see that 4K offers premium video quality and enhances the entertainment experience.
  • Competitive forces are operating at scale – Service Providers and OTT distributors will drive the adoption of 4K. MSO are upping their game and in 2017 you will see several deliver highly formidable services to take on pure play OTT distributors. Who’s going to win, who’s going to lose? We think it’s going to be a win-win as services are able to increase ARPUs and reduce churn, while consumers will be able to actually experience the full quality and resolution that their new TV can deliver.
  • Commercially available 4K UHD services will be scaling rapidly –  SNL Kagan forecasts the number of global UHD Linear channels at 237 globally by 2020, which is great news for consumers. The UltraHD Forum recently published a list of UHD services that are “live” today numbering 18 VOD and 37 Live services with 8 in the US and 47 outside the US. Clearly, content will not be the weak link in UHD 4K market acceptance for much longer.
  • Geographic deployments — 4K is more widely deployed in Asia Pacific and Western Europe than in the U.S. today. But we see this as a massive opportunity since many people are traveling abroad and thus will be exposed to the incredible quality. They will then return home to question their service provider, why they had to travel outside the country to see 4K. Which means as soon as the planned services in the U.S. are launched, they will likely attract customer more quickly than we’ve seen in the past.

HDR adds WOW factor to 4K

High Dynamic Range (HDR) improves video quality by going beyond more pixels to increase the amount of data delivered by each pixel. HDR video is capable of capturing a larger range of brightness and luminosity to produce an image closer to what can be seen in real life. Show anyone HDR content encoded in 4K resolution, and it’s no surprise that content providers and TV manufacturers are quickly jumping on board to deliver content with HDR. Yes, it’s “that good.” There is no disputing that HDR delivers the “wow” factor that the market and consumers are looking for. But what’s even more promising is the industry’s overwhelmingly positive reaction to it. Read more here.

Beamr has been working with Dolby to enable Dolby Vision HDR support for several years now, even jointly presenting a white paper at SMPTE in 2014. The V.265 codec is optimized for Dolby Vision and HDR10 and takes into account all requirements for both standards including full support for VUI signaling, SEI messaging, SMPTE ST 2084:2014 and ITU-R BT.2020. For more information visit http://beamr.com/vanguard-by-beamr-content-adaptive-hevc-codec-sdk

Beamr is honored to have customers who are best in class and span OTT delivery, Broadcast, Service Providers and other entertainment video applications. From what we see and hear, studios are uber excited about HDR, cable companies are prepping for HDR delivery, Satellite distributors are building the capability to distribute HDR, and of course OTT services like Netflix, FandangoNow (formerly M-GO), VUDU, and Amazon are already distributing content using either Dolby Vision or HDR10 (or both). If your current video encoding workflow cannot fully support or adequately encode content with HDR, it’s time to update. Our V.265 video encoder SDK is a perfect place to start.

VR & 360 Video at Streamable Bitrates

360-degree video made a lot of noise in 2016.  YouTube, Facebook and Twitter added support for 360-degree videos, including live streaming in 360 degrees, to their platforms. 360-degree video content and computer-generated VR content is being delivered to web browsers, mobile devices, and a range of Virtual Reality headsets.  The Oculus Rift, HTC Vive, Gear VR and Daydream View have all shipped this year, creating a new market for immersive content experiences.

But, there is an inherent problem with delivering VR and 360 video on today’s platforms.  In order to enable HD video viewing in your “viewport” (the part of the 360-degree space that you actually look at), the resolution of the full 360 video delivered to you should be 4K or more.  On the other hand, the devices on the market today which are used to view this content, including desktops, mobile devices and VR headsets only support H.264 video decoding. So delivering the high-resolution video content requires very high bitrates – twice as much as using the more modern HEVC standard.

The current solution to this issue is lowered video quality in order to fit the H.264 video stream into a reasonable bandwidth. This creates an experience for users which is not the best possible, a factor that can discourage them from consuming this newly-available VR and 360 video content.  But there’s one thing we know for sure – next generation compression including HEVC and content adaptive encoding – and perceptual optimization – will be a critical part of the final solution. Read more about VR and 360 here.

Patent Pool HEVC Advance Announces “Royalty Free” HEVC software

As 4K, HDR, VR and 360 video gathers steam, Beamr has seen the adoption rate moving faster than expected, but with the unanswered questions around royalties, and concerns of who would shoulder the cost burden, distributors have been tentative. The latest move by HEVC Advance to offer a royalty free option is meant to encourage and accelerate the adoption (implementation) of HEVC, by removing royalty uncertainties.

Internet streaming distributors and software application providers can be at ease knowing they can offer applications with HEVC software decoders without incurring onerous royalties or licensing fees. This is important as streaming app content consumption continues to increase, with more and more companies investing in its future.

By initiating a software-only royalty solution, HEVC Advance expects this move to push the rest of the market i.e. device manufacturers and browser providers to implement HEVC capability in their hardware and offer their customers the best and most efficient video experience possible.

 

>> 2017 Predictions

Mobile Video Services will Drive the Need for Content-adaptive Optimization

Given the trend toward better quality and higher resolution (4K), it’s more important than ever for video content distributors to pursue more efficient methods of encoding their video so they can adapt to the rapidly changing market, and this is where content-adaptive optimization provides a massive benefit.

The boundaries between OTT services and traditional MSO (cable and satellite) are being blurred now that all major MSOs include TVE (TV Everywhere streaming services with both VOD and Linear channels) in their subscription packages (some even break these services out separately as is the case with SlingTV). And in October, AT&T CEO Randall Stephenson vowed that DirecTV Now would disrupt the pay-TV business with revolutionary pricing for an  Internet-streaming service at a mere $35 per month for a package with more than 100 channels.

And get this – AT&T wireless is adopting the practice of “zero rating” for their customers, that is, they will not count the OTT service streaming video usage toward the subscriber’s monthly data plan. This represents a great value for customers, but there is no doubt that it puts pricing pressure on the operational side of all zero rated services.

2017 is the year that consumers will finally be able to enjoy linear as well as VOD content anywhere they wish even outside the home.

Beamr’s Contribution to MSOs, Service Providers, and OTT Distributors is More Critical Than Ever

When reaching to consumers across multiple platforms, with different constraints and delivery cost models, Beamr’s content adaptive optimizer perfects the encoding process to the most efficient quality and bitrate combination.

Whether you pay by the bit delivered to a traditional CDN provider, or operate your own infrastructure, the benefits of delivering less traffic are realized with improved UX such as faster stream start times and reduced re-buffering events, in addition to the cost savings. One popular streaming service reported to us that after implementing our content-adaptive optimization solution their rebuffering events as measured on the player were reduced by up to 50%, while their stream start times improved 20%.

Recently popularized by Netflix and Google, content-adaptive encoding is the idea that not all videos are created equal in terms of their encoding requirements. Content-adaptive optimization complements the encoding process by driving the encoder to the lowest bitrate possible based on the needs of the content, and not a fixed target bitrate (as seen in traditional encoding processes and products).

A content-adaptive solution can optimize more efficiently by analyzing already-encoded video on a frame-by-frame and scene-by-scene level, detecting areas of the video that can be further compressed without losing perceptual quality (e.g. slow motion scenes, smooth surfaces).

Provided the perceptual quality calculation is performed at the frame level with an optimizer that contains a closed loop perceptual quality measure, the output can be guaranteed to be the highest quality at the lowest bitrate possible. Click the following link to learn how Beamr’s patented content adaptive optimization technology achieves exactly this result.

Encoding and Optimization Working Together to Build the Future

Since the content-adaptive optimization process is applied to files that have already been encoded, by combining an industry leading H.264 and HEVC encoder with the best optimization solution (Beamr Video), the market will be sure to benefit by receiving the highest quality video at the lowest possible bitrate and file size. As a result, this will allow content providers to improve the end-user experience with high quality video, while meeting the growing network constraints due to increased mobile consumption and general Internet congestion.

Beamr made a bold step towards delivering on this stated market requirement by disrupting the video encoding space when in April 2016 we acquired Vanguard Video – a premier video encoding and technology company. This move will benefit the industry starting in 2017 when we introduce a new class of video encoder that we call a Content Adaptive Encoder.

As content adaptive encoding techniques are being adopted by major streaming services and video platforms like YouTube and Netflix, the market is gearing up for more advanced rate control and optimization methods, something that fits our perceptual quality measure technology perfectly. This fact when combined with Beamr having the best in class HEVC software encoder in the industry, will yield exciting benefits for the market. Read the Beamr Encoder Superguide that details the most popular methods for performing content adaptive encoding and how you can integrate them into your video workflow.

One Year from Now…

In one year from now when you read our post summarizing 2017 and heralding 2018, what you will likely hear is that 2017 was the year that advanced codecs like HEVC combined with efficient perceptually based quality measures, such as Beamr’s, provide an additional 20% or greater bitrate reduction.

The ripple effect of this technology leap will be that services struggling to compete today on quality or bitrate, may fall so far behind that they lose their ability to grow the market. We know of many multi-service operator platforms who are gearing up to increase the quality of their video beyond the current best of class for OTT services. That is correct, they’ve watched the consumer response to new entrants in the market offering superior video quality, and they are not sitting still. In fact, many are planning to leapfrog the competition with their aggressive adoption of content adaptive perceptual quality driven solutions.  

If any one service assumes they have the leadership position based on bitrate or quality, 2017 may prove to be a reshuffling of the deck.

For Beamr, the industry can expect to see an expansion of our software encoder line with the integration of our perceptual quality measure which has been developed over the last 7 years, and is covered by more than 50 patents granted and pending. We are proud of the fact that this solution has been shipping for more than 3 years in our stand-alone video and photo optimizer solutions.

It’s going to be an exciting year for Beamr and the industry and we welcome you to join us. If you are intrigued and would like to learn more about our products or are interested in evaluating any of our solutions, check us out at beamr.com.

4K TVEncoder ApplicationsH.265HEVCOTTReduce BitrateV.265 Codec SDK

Before you evaluate x265, read this!

Posted on by Dror Gill

With video consumption rising and consumer preferences shifting to 4K UHD this is contributing to an even faster adoption rate than what we saw with the move to HD TV. Consumer demand for a seamless (buffer-free) video experience is a new expectation, and with the latest announcement from HEVC Advance removing royalty uncertainties in the market it’s time to start thinking about building and deploying an HEVC workflow, starting with a robust HEVC encoder.

As you may know, Beamr’s V.265 was the first commercially deployed HEVC codec SDK and it is in use today by the world’s largest OTT streaming service. Even still, we receive questions regarding V.265 in comparison to x265 and in this post we’d like to address a few of them.

In future posts, we will discuss the differences in two distinct categories, performance (speed) and quality, but in this post we’ll focus on feature-related differences between V.265 and x265.

Beginning with our instruction set, specifically support for X86/x64 SMP Architecture, V.265 is able to improve encoding performance by leveraging a resource efficient architecture that is used by most multiprocessors today. Enabling this type of support allows each processor to execute different programs while working on discrete data sets to afford the capability of sharing common resources (memory, I/O device interrupt system and so on) that are connected using a system bus or a crossbar. The result is a notable increase in overall encoding speed with V.265 over x265. For any application where speed is important, V.265 will generally pull ahead as the winner.

Another area V.265 shines compared to x265 is with its advanced preprocessing algorithm support that provides resizing and de-interlacing. As many of you know, working with interlaced video can lead to poor video quality so to try and minimize the various visual defects V.265 uses a variety of techniques like line doubling where our smart algorithms are able to detect and fill in an empty row by averaging the line above and the line below. The advantages of having a resizing feature is recognizable, largely saving time and resources, and out of the box V.265 allows you to easily convert video from one resolution to another (i.e. 4K to HD). One note, we are aware that x265 supports these features via FFMPEG. However in the case that a user is not able to use FFMPEG, the fact that V.265 supports them directly is a benefit.

V.265 boasts an unmatched pre-analysis library with fading detection and complexity analysis capabilities not supported in x265. An application for the V.265 library is video segmentation that is problematic with many encoders because of the different ways two consecutive shots may be linked. In V.265, the fading detection method detects the type of gradual transition, fade type etc. which is needed to detect hard to recognize soft cuts. V.265’s complexity analysis is able to discriminate temporal and spatial complexity in video sequences with patented multi-step motion estimation methods that are more advanced than standard “textbook” motion estimation algorithms. The information gained from doing a video complexity analysis is used during the encoding process to improve encoding quality especially during transitions between scenes.

One of the most significant features V.265 offers compared to x265 is multistreaming (ABR) support. V.265 can produce multiple GOP-aligned video output streams that are extremely important when encoding for adaptive streaming. It is critical that all bitrates have IDRs aligned to enable seamless stream switching, which V.265 provides.

Additionally, with V.265 users can produce multiple GOP-aligned HEVC streams from a single input. This is extremely important for use cases when a user has one chance to synchronize video of different resolutions and bitrates.  Multistreaming helps to provide encoded data to HLS or DASH packagers in an optimal way and it provides performance savings – especially when the service must output multiple streams of the same resolution, but at varying bitrates.


Another significant feature V.265 has over x265 is its content adaptive speed settings that makes codec configuration more convenient such as real-time compared to VOD workflows. Currently we offer presets ranging from ultra fast for extremely low latency live broadcast streams to the highest quality VOD.

To combat packet losses and produce the most robust stream possible, V.265 supports slicing by slice compressed size which produces encoded slices of limited sized (typically the size of a network packet) for use in an error prone network. This is an important feature for anyone distributing content on networks with highly variable QoS.

Continuing on to parallel processing features, V.265 offers support for tiles that divides the frame into a grid of rectangular regions that can be independently decoded and encoded. Enabling this feature increases encoding performance.

V.265 is regarded as one of the most robust codecs in the market because of its ability to suit both demanding real-time and offline file based workflows. To deliver the industry leading quality that makes V.265 so powerful, it offers motion estimation features like patented high performance search algorithms and motion vectors over a picture boundary to provide additional quality improvements over x265.

For encoding by frame-type, V.265 offers Bi- and uni-directional non-reference P-frames which is useful where low-delay encoding is needed to improve temporal scalability

As for encoding tools, V.265 offers a unique set of tools over x265:

  1. Joint bi-directional Motion Vector Search which is an internal motion estimation encoding technique that provides a better bi-direction motion vector search.
  2. Sub-LCU QP modulation that allows the user to change QP from block to block inside LCU as a way to control in-frame bits/quality more precisely.
  3. Support for up to 4 temporal layers of multiple resolutions in the same bitstream to help with changing network conditions.
  4. Region of Interest (ROI) control which allows for encoding of a specific ROI with a particular encoding parameter (qp) to add flexibility and improve encoding quality.

Another major advantage over x265 is the proprietary rate control implementation offered with V.265. This ensures target bitrates are always maintained.

The more supplemental enhancement information (SEI) messages a codec supports the more video usability information (VUI) metadata that may be delivered to the decoder in an encoded bitstream. For this reason, Beamr found it necessary to include in V.265 support for Recovery point, Field indication, Decoded Picture Hash, User data unregistered, and User data as specified by ITU-T T.35.

V.265’s ability to change encoding parameters on the fly is another extremely important feature that sets it apart from x265. With the ability to change encoder resolution, bitrate, and other key elements of the encoding profile, video distributors can achieve a significant advantage by creating recipes appropriate to each piece of content without needing to interrupt their workflows or processing cycles to reset and restart an encoder.

We trust this feature comparison was useful. In the event that you require more information or would like to evaluate the V.265, feel free to reach out to us at http://beamr.com/info-request and someone will get in touch to discuss your application and interest.

Encoder ApplicationsHEVCReduce BitrateV.265 Codec SDKVR

Immersive VR and 360 video at streamable bitrates: Are you crazy?

Posted on by Dror Gill

There have been many high-profile experiments with VR and 360 video in the past year. Immersive video is compelling, but large and unwieldy to deliver. This area will require huge advancements in video processing – including shortcuts and tricks that border on ‘magical’.

Most of us have experienced breathtaking demonstrations that provide a window into the powerful capacity of VR and 360 video – and into the future of premium immersive video experiences.

However, if you search the web for an understanding of how much bandwidth is required to create these video environments, you’re likely to get lost in a tangled thicket of theories and calculations.

Can the industry support the bitrates these formats require?

One such post on Forbes in February 2016 says No.

It provides a detailed mathematical account of why fully immersive VR will require each eye to receive 720 million pixels at 36 bits per pixel and 60 frames per second – or a total of 3.1 trillion bits per second.1

We’ve taken a poll at Beamr, and no one in the office has access to those kinds of download speeds. And some of these folks pay the equivalent of a part-time salary to their ISP!

Thankfully the Forbes article goes on to explain that it’s not quite that bad.

Existing video compression standards will be able to improve this number by 300, according to the author, and HEVC will compress that by 600 down to what might be 5.2 Gbps.

The truth is, the calculations put forth in the Forbes piece are very ambitious indeed. As the author states:

“The ultimate display would need a region of 720 million pixels for full coverage because even though your foveal vision has a more narrow field of view, your eyes can saccade across that full space within an instant. Now add head and body rotation for 360 horizontal and 180 vertical degrees for a total of more than 2.5 billion (giga) pixels.”

A more realistic view of the way VR will rollout was presented by Charles Cheevers of network equipment vendor ARRIS at INTX in May of this year.2

Great VR experiences including a full 360 degree stereoscopic video environment at 4K resolutions could easily require a streaming bandwidth of 500 Mbps or more.

That’s still way too high, so what’s a VR producer to do?

Magical illusion, of course. 

In fact, just like your average Vegas magician, the current state of the art in VR delivery relies on tricks and shortcuts that leverage the imperfect way we humans see.

For example, Foveated Rendering can be used to aggressively compress the areas of a VR video where your eyes are not focused.

This technique alone, and variations on this theme – can take the bandwidth required by companies like NextVR dramatically lower, with some reports that an 8 Mbps stream can provide a compelling immersive experience. The fact is, there are endless ways to configure the end-to-end workflow for VR and much will depend on the hardware and software and networking environments in which it is deployed.

Compression innovations utilizing perceptual frame by frame rate control methodologies, and some involving the mapping of spherical images to cubes and pyramids, in an attempt to transpose images into 5 or 6 viewing planes, and ensure the highest resolution is always on the plane where the eyes are most intensely focused, are being tried.3

At the end of the day, it’s going to be hard to pin down your nearest VR dealer on the amount of bandwidth that’s required for a compelling VR experience. But there’s one thing we know for sure – next generation compression including HEVC and content adaptive encoding – and perceptual optimization – will be a critical part of the final solution.

References:

(1) Found on August 10, 2016 at the following URL: http://www.forbes.com/sites/valleyvoices/2016/02/09/why-the-internet-pipes-will-burst-if-virtual-reality-takes-off/#ca7563d64e8c

(2) Start at 56 minutes. https://www.intxshow.com/session/1041/  — Information and a chart is also available online here: http://www.onlinereporter.com/2016/06/17/arris-gives-us-hint-bandwidth-requirements-vr/ 

(3) Facebook’s developer site gives a fascinating look at these approaches, which they call dynamic streaming techniques. Found on August 10, 2016 at the following URL:  https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/

4K TVAVCEncoder ApplicationsH.264HEVCOTTV.264 Codec SDKV.265 Codec SDKVR

Will Virtual Reality Determine the Future of Streaming?

Posted on by Dror Gill

As video services take a more aggressive approach to virtual reality (VR), the question of how to scale and deliver this bandwidth intensive content must be addressed to bring it to a mainstream audience.

While we’ve been talking about VR for a long time you can say that it was reinvigorated when Oculus grabbed the attention of Facebook who injected 2 billion in investment based on Mark Zuckerberg’s vision that VR is a future technology that people will actively embrace. Industry forecasters tend to agree, suggesting VR will be front and center in the digital economy within the next decade. According to research by Canalys, vendors will ship 6.3 million VR headsets globally in 2016 and CCS Insights suggest that as many as 96 million headsets will get snapped up by consumers by 2020.

One of VR’s key advantages is the fact that you have the freedom to look anywhere in 360 degrees using a fully panoramic video in a highly intimate setting. Panoramic video files and resolution dimensions are large, often 4K (4096 pixels wide, 2048 pixels tall, depending on the standard) or bigger.

While VR is considered to be the next big revolution in the consumption of media content, we also see it popping up in professional fields such as education, health, law enforcement, defense telecom and media. It can provide a far more immersive live experience than TV, by adding presence, the feeling that “you are really there.”

Development of VR projects have already started to take off and high-quality VR devices are surprisingly affordable. Earlier this summer, Google announced that 360-degree live streaming support was coming to YouTube.

Of course, all these new angles and sharpness of imagery creates new and challenging sets of engineering hurdles which we’ll discuss below.

Resolution and, Quality?

Frame rate, resolution, and bandwidth are affected by the sheer volume of pixels that VR transmits. Developers and distributors of VR content will need to maximize frame rates and resolution throughout the entire workflow. They must keep up with the wide range of viewers’ devices as sporting events in particular, demand precise detail and high frame rates, such as what we see with instant replay, slow motion, and 360-degree cameras.

In a recent Vicon industry survey, 28 percent of respondents stated that high-quality content was important to ensuring a good VR experience. Let’s think about simple file size comparisons – we already know that ultra HD file sizes take up considerably more storage space than SD and the greater the file size, the greater a chance it will impede the delivery. VR file sizes are no small potatoes.  When you’re talking about VR video you’re talking about four to six times the foundational resolution that you are transmitting. And, if you thought that Ultra HD was cumbersome, think about how you’re going to deal with resolutions beyond 4K for an immersive VR HD experience.

In order to catch up with the file sizes we need to continue to develop video codecs that can quickly interpret the frame-by-frame data. HEVC is a great starting point but frankly given hardware device limitations many content distributors are forced to continue using H.264 codecs. For this reason we must harness advanced tools in image processing and compression. An example of one approach would be content adaptive perceptual optimization.

I want my VR now! Reaching End Users

Because video content comes in a variety of file formats including combinations of stereoscopic 3D, 360 degree panoramas and spherical views – they all come with obvious challenges such as added strain on processors, memory, and network bandwidth. Modern codecs today use a variety of algorithms to quickly and efficiently detect these similarities, but they are usually tailored to 2D content. However, a content delivery mechanism must be able to send this to every user and should be smart to optimize the processing and transmitting of video.

Minimizing latency, how long can you roll the boulder up the hill?

We’ve seen significant improvements in the graphic processing capabilities of desktops and laptops. However, to take advantage of the immersive environment that VR offers, it’s important that high-end graphics are delivered to the viewer as quickly and smoothly as possible. The VR hardware also needs to display large images properly and with the highest fidelity and lowest latency. There really is very limited room for things like color correction or for adjusting panning from different directions for instance. If you have to stitch or rework artifacts, you will likely lose ground. You need to be smart about it. Typical decoders for tablets or smart TVs are more likely to cause latency and they only support lower framerates. This means how you build the infrastructure will be the key to offering image quality and life-like resolution that consumers expect to see.

Bandwidth, where art thou?

According to Netflix, for an Ultra HD streaming experience, your Internet connection must have a speed of 25 Mbps or higher. However, according to Akamai, the average Internet speed in the US is only approximately 11 Mbps. Effectively, this prohibits live streaming on any typical mobile VR device which to achieve the quality and resolution needed may need 25 Mbps minimum.

Most certainly the improvements in graphic processing and hardware will continue to drive forward the realism of the immersive VR content, as the ability to render an image quickly becomes easier and cheaper. Just recently, Netflix jumped on the bandwagon and became the first of many streaming media apps to launch on Oculus’ virtual reality app store. As soon as all the VR display devices are able to integrate with these higher resolution screens, we will see another step change in the quality and realism of virtual environments. But will the available bandwidth be sufficient, is a very real question. 

To understand the applications for VR, you really have to see it to believe it

A heart-warming campaign from Expedia recently offered children at a research hospital in Memphis Tennessee the opportunity to be taken on a journey of their dreams through immersive, real-time virtual travel – all without getting on a plane:  https://www.youtube.com/watch?time_continue=179&v=2wQQh5tbSPw

The National Multiple Sclerosis Society also launched a VR campaign that inventively used the tech to give two people with MS the opportunity to experience their lifelong passions. These are the type of immersive experiences we hope will unlock a better future for mankind. We applaud the massive projects and time spent on developing meaningful VR content and programming such as this.

Frost & Sullivan estimates that $1.5 billion is the forecasted revenue from Pay TV operators delivering VR content by 2020. The adoption of VR in my estimation is only limited by the quality of the user experience, as consumer expectation will no doubt be high.

For VR to really take off, the industry needs to address some of these challenges making VR more accessible and most importantly with unique and meaningful content. But it’s hard to talk about VR without experiencing it. I suggest you try it – you will like it.

AVCEncoder ApplicationsH.264HEVCV.264 Codec SDKV.265 Codec SDK

Applications for On-the-Fly Modification of Encoder Parameters

Posted on by Dror Gill

As video encoding workflows modernize to include content adaptive techniques, the ability to change encoder parameters “on-the-fly” will be required. With the ability to change encoder resolution, bitrate, and other key elements of the encoding profile, video distributors can achieve a significant advantage by creating recipes appropriate to each piece of content.

For VOD or file-based encoding workflows, the advantages of on-the-fly reconfigurability are to enable content specific encoding recipes without resetting the encoder and disrupting the workflow. At the same time, on-the-fly functionality is a necessary feature for supporting real-time encoding on a network with variable capacity.  This way the application can take appropriate steps to react to changing bandwidth, network congestion or other operational requirements.

Vanguard by Beamr V.264 AVC Encoder SDK and V.265 HEVC Encoder SDK have supported on-the-fly modification of the encoder settings for several years. Let’s take a look at a few of the more common applications where having the feature can be helpful.

On-the-fly control of Bitrate

Adjusting bitrate while the encoder is in operation is an obvious application. All Vanguard by Beamr codec SDKs allow for the maximum bitrate to be changed via a simple “C-style” API.  This will enable bitrate adjustments to be made based on the available bandwidth, dynamic channel lineups, or other network conditions.

On-the-fly control of Encoder Speed

Encoder speed control is an especially useful parameter which directly translates into video encoding quality and encoding processing time. Calling this function triggers a different set of encoding algorithms, and internal codec presets. This scenario applies with unicast transmissions where a service may need to adjust the encoder speed for ever-changing network conditions and client device capabilities.

On-the-fly control of Video Resolution

A useful parameter to access on the fly is video resolution. One use case is in telecommunications where the end user may shift his viewing point from a mobile device operating on a slow and congested cellular network, to a broadband WiFi network, or hard wired desktop computer. With control of video resolution, the encoder output can be changed during its operation to accommodate the network speed or to match the display resolution, all without interrupting the video program stream.

On-the-fly control of HEVC SAO and De-blocking Filter

HEVC presents additional opportunities to enhance “on the fly” control of the encoder and the Vanguard by Beamr V.265 encoder leads the market with the capability to turn on or off SAO and De-blocking filters to adjust quality and performance in real-time.

On-the-fly control of HEVC multithreading

V.265 is recognized for having superior multithreading capability.  The V.265 codec SDK provides access to add or remove encoding execution threads dynamically. This is an important feature for environments with a variable number of tasks running concurrently such as encoding functionality that is operating alongside a content adaptive optimization process, or the ABR packaging step.

Beamr’s implementation of on-the-fly controls in our V.264 Codec SDK and V.265 Codec SDK demonstrate the robust design and scalable performance of the Vanguard by Beamr encoder software.

For more information on Vanguard by Beamr Codec SDK’s, please visit the V.264 and V.265 pages.  Or visit http://beamr.com for more on the company and our technology.