With numerous advantages, AV1 is now supported on about 60% of devices and all major web browsers. To accelerate its adoption – Beamr has introduced an easy, automated upgrade to the codec that is in the forefront of today’s video technology
Four years ago we explored the different video codecs, analyzing their strengths and weaknesses, and took a look at current and predicted market share. While it is gratifying to see that many of our predictions were pretty accurate, that is accompanied by some degree of disappointment: while AV1 strengths are well known in the industry, significant change in adoption of new codecs has yet to materialize.
The bottom line of the 2020 post was: “Only time will tell which will have the highest market share in 5 years’ time, but one easy assessment is that with AVC current market share estimated at around 70%, this one is not going to disappear anytime soon. AV1 is definitely gaining momentum, and with the giants backing we expect to see it used a fair bit in online streaming. “
Indeed we are living in a multi-codec reality, where AVC still accounts for, by far, the largest percentage of video content, but adoption of AV1 is starting to increase with large players such as Netflix and YouTube incorporating it into their workflows, and many others using it for specific high value use cases.
Thus, we are faced with a mixture of the still dominant AVC, HEVC (serving primarily UHD and HDR use cases), AV1 and some additional codecs such as VP9, VVC which are being used in quite small amounts.
The Untapped Potential of AV1
So while AV1 adoption is increasing, there is still significant untapped potential. One of the causes for slower than hoped rollout of AV1 is the obstacle present for adoption of any new standard – critical mass of decoding support in H/W on edge devices.
While for AVC and HEVC the coverage is very extensive, for AV1 that has only recently become the case, with support across an estimate of 60% of devices and all major web browsers, and complementing the efficient software decoding offered by Dav1d.
Another obstacle AV1 faces involves the practicalities of deployment. While there is extensive knowledge, within the industry and available online, of how best to configure AVC encoding, and what presets and encoding parameters work well for which use cases – there is no such equivalent knowledge for AV1. Thus, in order to deploy it, extensive research is needed by those who intend to use it.
Additionally, AV1 encoding is complicated, resulting in much higher processing power required to perform software encoding. In a world that is constantly trying to cut back costs, and use lower power solutions, this can pose a problem. Even when using software solutions at the fastest settings, the compute required is still significantly slower than AVC encoding at typical speeds. This is a strong motivator to upgrade to AV1 using H/W accelerated solutions (Learn more about Beamr solution to the challenge).
The upcoming codec possibilities are also a deterrent for some. With AV2 in the works, VVC finalized and gaining some traction, and various groups working on AI based encoding solutions, there will always be players waiting for ‘the next big thing’, rather than having to switch out codecs twice.
In a world where JPEG, a 30+ year old standard, is still used in over 70% of websites and is the most popular format on the web for photographic content, it is no surprise that adoption of new video codecs is taking time.
While a multi codec reality is probably going to stay with us, we can at least hope that when we revisit this topic in a blog a few years down the line, the balance between deployed codecs leans more towards the higher efficiency codecs, like AV1, to yield the best bitrate – quality options for the video world.
This year at IBC 2024 in Amsterdam, we are excited to demonstrate Live 4K p60 optimized streaming with our Content-Adaptive Bitrate (CABR) technology on NVIDIA Holoscan for Media, a software-defined, AI-enabled platform that allows live video pipelines to run on the same infrastructure as AI. Using the CABR GStreamer plugin, premiered at the NAB Show earlier this year, we now support live, quality-driven optimized streaming for 4Kp60 video content.
It is no secret that savvy viewers are coming to expect the high-quality experience of 4K Ultra-High-Definition streamed at 60 frames per second for premium events. What started with a drizzle a few years back has become the high end norm for recent events such as the 2024 Olympics, where techies were sharing insights on where it could be accessed.
Given the fact that 4K means a whopping four times the pixels compared to full HD resolution, keeping up with live encoding of 4K at 60 fps can be quite challenging, and can also result in bitrates that are too high to manage.
One possible solution for broadcasters is to encode and transmit at 1080p and rely on the constantly improving upscalers available on TVs to provide the 4K experience, but this of course means they cannot control the user experience. A better solution is to have a platform that is super fast, and can create live 4Kp60 encodes, which combine excellent quality with an optimization process that minimizes the required bitrate for transmission.
Comparison of 4K Live video before and after optimization
Beamr CABR on Holoscan for Media offers exactly that, by combining the fast data buses and easy-to-use architecture of Holoscan for Media with Beamr hardware-accelerated, quality-driven optimized AV1 encoding. Together, it is possible to stream super efficient, 4K, lower bitrate encodes at top notch quality.
Content Adaptive Bitrate encoding, or CABR, is Beamr’s patented and award-winning technology that uses a quality measure to select the best candidate with the lowest bitrate and the same perceptual quality as a reference frame. In other words, users can enjoy 30-50% lower bitrate, faster delivery of files or live video streams and improved user experience – all with exactly the same quality as the original video.
In order to achieve aggressive bitrates which are feasible for broadcast of live events, we configure the system to use AV1 encoding. The advanced AV1 format has been around since 2018. However, its full potential has not been fully realized by many players in the video arena. AV1 is raising the bar significantly in comparison to previous modern codecs, such as AVC (H.264) or HEVC (H.265), in terms of efficiency, performance with GPUs and high quality for real-time video. When combined with CABR – AV1 is offering up even more. According to our tests, AV1 can reduce data by 50% compared to AVC and by 30% compared to HEVC. We also showed that CABR optimized AV1 is beneficial for machine learning tasks.
Putting all three of these technologies together, namely deploying Holoscan for Media with the Beamr CABR solution inside, which in turn is using NVIDIA’s hardware-accelerated AV1 encoder, provides a platform that offers spectacular benefits. With the rise in demand for high-quality live streaming at high resolution, high fps and manageable bitrates, while keeping an eye on the encoding costs – this solution is definitely an interesting prospect for companies looking to boost their streaming workflows.
Now available: Hardware accelerated, unsupervised, codec modernization to AV1 for increased efficiency video AI workflows
AV1, the new kid on the block of video encoders, is definitely starting to gain traction due to its high compression efficiency and increasing adoption on browsers and end devices. As we mentioned in our previous blog, H/W accelerated AV1 encoding is a particularly attractive prospect due to the combination of increased efficiency and light speed performance. H/W accelerated codec modernization – using Beamr’s Content Adaptive Bit-Rate (CABR) video optimization process running with NVIDIA video encoding – allows for fast, fully automatic, upgrade of legacy encodes to perceptually identical AV1 encodes.
Codec modernization is essentially the ability to get double the benefit – both the increased compression efficiency of codecs such as AV1, and the bitrate efficiencies of Beamr’s perceptually driven optimization. Over the years we have consistently validated that Beamr CABR technology creates optimized files that are perceptually identical, meaning they look the same to the human eye. While we have consistently demonstrated that the visual quality is indeed preserved, in this blog post we continue to explore how Beamr’s optimization lends itself to AI based workflows.
In our previous case studies, we looked at how the reduced bitrate, optimized videos, behave in Machine Learning (ML) tasks such as face detectionand action recognition training. We showed that the results when using optimized AVC and HEVC encodes are stable, despite reducing file sizes significantly with an average reduction of 24% on the source files, and an amazing x3 decrease in size of the cropped AVC encoded files created by openCV.
Now we add codec modernization to the mix, which allows to reduce the sizes of the cropped encodes further. The AV1 encoded files are smaller by a factor of 4, while still providing very similar training and inference results, as shown by the maximal and average accuracy results obtained in the different experiments and presented in the following table:
Tested on AVC
Tested on optimized AV1
Trained on AVC
67.5% (53%)
66.4% (53%)
Trained on optimized AV1
66.4% (52%)
64.8% (53.5%)
Next we decided to ramp up the fun factor and play around with some cool AI applications. Using the open source Face Fusion project, we took 10 source AVC videos, an image containing our target face and proceeded to swap the faces in the source videos with our target person. Now, while this is a fun experiment in itself, imagine how much easier it becomes when the source videos are reduced by a factor of 4, with the results looking just the same.
Below is an example showing a frame from the source video, the target face image, and side by side comparison of the video with the replaced or fused face – when using the original AVC encode (on the left) or the AV1 optimized by Beamr (on the right), looking just as good:
We are just starting to scratch the surface on how Beamr’s technology and offerings, including codec modernization to AV1, can help make AI workflows more efficient without compromising quality or accuracy. We are excited to be on this journey and will continue to explore and add on to the synergies between video optimization, video modernization and video AI solutions.
Beamr’s Content Adaptive Bit Rate solution enables significantly decreasing video file size or bitrates without changing the video resolution or compromising perceptual quality. Since the optimized file is fully standard compliant, it can be used in your workflow seamlessly, whatever your use case, be it video streaming, playback or even part of an AI workflow.
Beamr first launched Beamr cloud earlier this year, and we are now super excited to announce that our valued partnership with Oracle Cloud Infrastructure (OCI) is enabling us to offer to OCI customers more features and better performance.
The performance improvements are due in part to the availability of the powerful NVIDIA L40S GPUs on OCI. In preliminary testing we found that running our video encoding workflows can be up to 30% faster when using these cards, than when running on the cards we currently use in the Beamr Cloud solution.
This was derived from testing AVC and HEVC NVENC driven encodes for a set of nine 1080p classic test clips with eight different configurations, and comparing encoding wall times on an A10G vs. a L40S GPU. Speedup factors of up to 55% were observed, with an average just above 30%. The full test data is available here.
Another exciting feature about these cards is that they support AV1 encoding, which means Beamr Cloud will now offer to turn your videos into optimized AV1 encodes, offering even higher bitrate/file size savings.
What’s the fuss about AV1?
In order to store and transmit video, substantial compression is needed. From the very earliest efforts to standardize video compression in the 90s, there has been a constant effort to create video compression standards offering increasing efficiency – meaning that the same video quality can be achieved with smaller files or lower bitrates.
As shown in the schematic illustration below, AV1 has come a long way in improving over H.264/AVC, the most widely adopted standard today, despite being 20 years old. However, the increased compression efficiency is not free – the computational complexity of newer codecs is also significantly higher, motivating the adoption of hardware accelerated encoding options.
With the demand and need for Video AI workflows continuing to rise, the ability to perform fully automatic, fast, efficient, optimized video encoding is an important enabler.
The Beamr GPU powered video compression and optimization occur within the GPU on OCI, right at the heart of these AI workflows, making them extremely well placed to offer benefits to such workflows. We have previously shown in a number of case studies that there is no negative impact on inference or training results when using the optimized files – making the integration of this optimization process into AI workflows a natural choice for cost savvy developers.
This year at the NAB Show 2024 in Las Vegas, we are excited to demonstrate our Content-Adaptive Bitrate (CABR) technology on the NVIDIA Holoscan for Media platform. By implementing CABR as a GStreamer plugin, we have, for the first time, made bitrate optimization of live video streams easily achievable in the cloud or premise.
Building on the NVIDIA DeepStream software development kit, which can extends GStreamer’s capabilities, significantly reduced the amount of code required to develop the Holoscan for Media based application. Using DeepStream components for real-time video processing and NMOS (Networked Media Open Specifications) signaling, we were able to keep our focus on the CABR technology and video processing.
The NVIDIA DeepStream SDK provides an excellent framework for developers to build and customize dynamic video processing pipelines. DeepStream provides pipeline components that make it very simple to build and deploy live video processing pipelines that utilize the hardware decoders and encoders available on all NVIDIA GPUs.
Beamr CABR dynamically adjusts video bitrate in real-time, optimizing quality and bandwidth use. It reduces data transmission without compromising video quality, making the video streaming more efficient. Recently we released our GPU implementation which uses the NVIDIA NVENC, encoder, providing significantly higher performance compared to previous solutions.
Taking our GPU implementation for CABR to the next level, we have built a GStreamer Plugin. With our GStreamer Plugin, users can now easily and seamlessly incorporate the CABR solution into their existing DeepStream pipelines as a simple drop-in replacement to their current encoder component.
Holoscan For Media
A GStreamer Pipeline Example
To illustrate the simplicity of using CABR, consider a simple DeepStream transcoding pipeline that reads and writes from files.
By simply replacing the nvv4l2av1enc component with our CABR component, the encoding bitrate is adapted in real-time, according to the content, ensuring optimal bitrate usage for each frame, without any loss of perceptual quality.
Similarly, we can replace the encoder component used in a live streaming pipeline with the CABR component to optimize live video streams, dynamically adjusting the output bitrate and offering up to a 50% reduction in data usage without sacrificing video quality.
The Broad Horizons of CABR Integration in Live Media
Beamr CABR, demonstrated using NVIDIA Holoscan for Media at NAB show, marks just the beginning. This technology is an ideal fit for applications running on NVIDIA RTX GPU-powered accelerated computing and sets a new standard for video encoding.
Lowering the video bitrate reduces the required bandwidth when ingesting video to the cloud, creating new possibilities where high resolution or quality were previously costly or not even possible. Similarly, reduced bitrate when encoding on the cloud allows for streaming of higher quality videos at lower cost.
From file-based encoding to streaming services — the potential use cases are diverse, and the integration has never before been so simple. Together, let’s step into the future of media streaming, where quality and efficiency coexist without compromise.
When it comes to comparing video codecs, it’s easy to get caught up in the “codec war” mentality. If analyzing and purchasing codecs was as easy as comparing fuel economy in cars, it would undoubtedly take a lot of friction out of codec comparison, but the reality is that it’s not that simple.
In Episode 02, The Video Insiders go head-to-head comparing two of the leading codecs in a three against one standoff over whether AV1 is more efficient than HEVC.
So, which is more efficient?
Listen in to this week’s episode, “Codec Efficiency Is in the Eye of the Measurer,” to find out.
Tune in here or click play on the video below to listen now.
https://youtu.be/ThkoOHuPEvo
Want to join the conversation? Reach out to TheVideoInsiders@beamr.com.
TRANSCRIPTION (lightly edited to improve readability only)
Mark Donnigan: 00:41 Hi everyone I am Mark Donnigan and I want to welcome you to episode two of the Video Insiders.
Mark Donnigan: 00:50 In every episode of the Video Insiders we bring the latest inside information about what’s happening in the video technology industry from encoding, to packaging, to delivery, and playback, and even the business behind the video business. Every aspect of the video industry is covered in detail on the Video Insiders podcast.
Dror Gill: 01:11 Oh yeah, we usually do cover everything from pixels, to blocks, to microblocks, to frames, to sequences. We go all the way up and down the video delivery chain and highlight the most important things you should know before you send any video bits over the wire.
Mark Donnigan: 01:28 In our first episode we talked about a very hot topic which asked, “Hasn’t this kind of been worn out?” The whole HEVC, AV1 discussion. But I think it was very interesting. I sure enjoyed the talk. What about you Dror?
Dror Gill: 01:47 Yeah, yeah, yeah. I sure did. It was great talking about the two leading codecs. I don’t want to say the word, codec war.
Mark Donnigan: 01:58 No, no, we don’t believe in codec wars.
Mark Donnigan: 02:00 Yeah, that’s true. Why is it so complicated to compare video codecs? Why can’t it be as simple as fuel economy of cars, this one gets 20 miles per gallon and that one gets 30 and then I make a decision based on that.
Dror Gill: 02:15 I wish it was that simple with video codecs. In video compression you have so many parameters to consider. You have the encoding tools, tools are grouped into what’s called profiles and levels, or as AV1 calls them “experiments.”
Dror Gill: 02:35 When you compare the codecs which profiles and levels do you use. What rate control method? Which specific parameters do you set for each codec? And each codec can have hundreds, and hundreds of parameters. Then there is the question of implementation. Which software implementation of the codec do you use? Some implementations are reference implementations that are used for research, and others are highly performance optimized commercial implementations. Which one do you select for the test? And then, which operating system, what hardware do you run on, and obviously what test content? Because encoding two people talking, or encoding an action scene for a movie, is completely different.
Dror Gill: 03:13 Finally, when you come to evaluate your video, what quality measure do you use? There’re various objective quality measures and some people use actual human viewers and they assesses subjective quality of the video. On that front also, there’re many possibilities that you need to choose from.
Mark Donnigan: 03:32 Yeah, so many questions and no wonder the answers are not so clear. I was quite surprised when I recently read three different technical articles published at IBC actually, effectively comparing AV1 versus HEVC and I can assume that each of the authors did their research independently. What was surprising was they came to the exact same conclusion, AV1 has the same compression efficiency as HEVC. This is surprising because some other studies and one in particular (I think we’ll talk about) out there says the contrary. So can you explain what this means exactly, Dror.
Dror Gill: 04:16 By saying that they have the same compression efficiency, this means that they can reach the same quality at the same bitrate or the other way round. You need the same bitrate to reach that same quality. If you need for example, two and a half megabits per second to encode an HD video file using HEVC at a certain quality, then with AV1 you would need roughly the same bitrate to reach that same quality and this means that AV1 and HEVC provide the same compression level. In other words, this means that AV1 does not have any technical advantage over HEVC because it has the same compression efficiency. Of course that’s if we put aside all the loyalty issues but we discussed that last time. Right?
Mark Donnigan: 04:56 That’s right. The guys who wrote the three papers that I’m referencing are really top experts in the field. It’s not seminar work done by a student, not to downplay those papers, but the point is these are professionals. One was written by the BBC in cooperation with the Multimedia and Vision Group at the Queen Mary University of London. I think nobody is going to say that the BBC doesn’t know a thing or two about video. The second was written by Ateme, and the third by Harmonic, leading vendors.
Mark Donnigan: 05:29 I actually pulled out a couple of phrases from each that I’d like to quote. First the BBC and Queen Mary University, here is a conclusion that they wrote, “The results obtained show in general a similar performance between AV1 and the reference HEVC both objectively and subjectively.” Which is interesting because they did take the time to both do the visual assessment as well as use a quality measure.
Mark Donnigan: 06:01 Ateme said, “Results demonstrate AV1 to have equivalent performance to HEVC in terms of both objective and subjective video quality test results.”
Mark Donnigan: 06:16 And then here is what Harmonic said, “The findings are that AV1 is not more advantageous today than HEVC on the compression side and much more complex to encode than HEVC.” What do you make of this?
Dror Gill: 06:32 I don’t know. It sounds pretty bad to me, even two of those papers also analyzed subjective quality so they used actual human viewers to check out the quality. But Mark what if I told you that researchers from the University of Klagenfurt in Austria together with Bitmovin published a paper which showed completely different results. What would you say about that?
Dror Gill: 06:58 Last month in Athens I was the ICIP conference that’s the IEEE International Conference on Image Compression and Image Processing. There was this paper presented by this University in Austria with Bitmovin and their conclusion was, let me quote, “When using weighted PSNR, AV1 performs consistently better for bit rate compared to AVC, HEVC, and VP9.” So they claim AV1 is better than three codecs but specifically it’s better than HEVC. And then they have a table in their article that compares AV1 to HEVC for six different video clips. The table shows that with AV1 you get up to 25% lower bitrate at the same quality than HEVC.
Dror Gill: 07:43 I was sitting there in Athens last month when they presented this and I was shocked.
Mark Donnigan: 07:50 What are the chances that three independent papers are wrong and only this paper got it right? And by the way, the point here is not three against one because presumably there’re some other papers. I’m guessing other research floating around that might side with Bitmovin. The point is that three companies who no one is going to say that any of them are not experts and not highly qualified to do a video assessment, came up with such a different result. Tell us what you think is going on here?
Dror Gill: 08:28 I was thinking the same thing. How can that be. During the presentation I asked one of the authors who presented the paper a few questions and it turned out that they made some very questionable decisions in all of that sea of possibility that I talked about before. Decisions related to coding tools, codec parameters, and quality measures.
Dror Gill: 08:51 First of all, in this paper they didn’t show any results of subjective viewing. Only the objective metrics. Now we all know that you should always your eyes, right?
Dror Gill: 09:04 Objective metrics, nice numbers, but obviously you need to view the video because that’s how the actual viewers are going to assess the (video) quality. The second thing is that they only used the single objective metric and this was PSNR. PSNR, it stands for peak signal-to-noise ratio and basically this measure is a weighted average of the difference in peaks between pixel values of the two images.
Dror Gill: 09:30 Now, we’re Video Insiders, but even if you’re not an insider you know that PSNR is not a very good quality measure because it does not correlate very well with human vision. This is the measure that they choose to look at but what was most surprising is that there is a flag in the HEVC open source encoder which they used that if chosen, the result is improved PNSR. What it does, it turns off some psycho-visual optimizations which make the video look better but reduce the PSNR, and that’s turned on by default. So you would expect that they’re measuring PSNR they would turn that flag on so you would get higher PSNR. Well, they didn’t. They didn’t turn the flag on!
Dror Gill: 10:17 Finally, even then AV1 is much slower than HEVC, and they also reported in this data that it was much, much slower than HEVC but still they did not use the slowest encoding standing of HEVC, which would provide the best quality. There’s always a trade off between performance and quality. The more tools you employ the better quality you can squeeze out of the video, of course that takes you more CPU cycles but they used for HEVC, the third slowest setting which means this is the third best quality you can get with that codec and not the very best quality. When you handicap an HEVC encoder in this way, it’s not surprising that you get such poor results.
Dror Gill: 11:02 I think based on all these points everybody can understand why the results of this comparison were quite different than all of the other comparison that were published a month earlier at IBC (by Ateme, BBC, Harmonic).
Mark Donnigan: 11:14 Another critical topic that we have to cover is performance. If you measure the CPU performance on encoding time of AV1, I believe that it’s pretty universally understood that you are going to find it currently is a hundred times slower than HEVC. Is that correct?
Dror Gill: 11:32 Yeah, that’s right. Typically, you measure the performance of an encoder and FPS which is frames per second. For HEVC it’s common to measure an FPM which is frames per minute.
Mark Donnigan: 11:42 Frames per minute, (more like) frames per hour, FPH.
Dror Gill: 11:45 A year and a half ago or a year ago when there were very initial implementation, it was really FPD or FPH, Frames per hour or per day and you really needed to have a lot of patience, but now after they’ve done some work it’s only a hundred times slower than HEVC.
Mark Donnigan: 12:02 Yeah, that’s pretty good. They’re getting there. But some people say that the open source implementation of AV1 I believe it’s AOM ENC.
Mark Donnigan: 12:16 ENC exactly has not been optimized for performance at all. One thing I like about speed is either your encoder produces X number of frames per second or per minute, or it doesn’t. It’s really simple. Here is my next question for you. Proponents of AV1 are saying, “well it’s true it’s slow but it hasn’t been optimized, the open source implementation,” which is to imply that there’s a lot of room (for improvement) and that we’re just getting started, “don’t worry we’ll close the gap.” But if you look at the code, and by the way I may be a marketing guy but my formal education is computer science.
Mark Donnigan: 13:03 You can see it already includes performance optimizations. I mean eptimizations like MMX, SSE, there’s AVX instructions, there’s CPU optimization, there’s multithreading. It seems like they’re already trying to make this thing go faster. So how are they going to close this a hundred X (time) gap?
Dror Gill: 13:22 I don’t think they can. I mean a hundred X, that’s a lot and you know even the AV1 guys they even admit that they won’t be able to close the gap. I talked to a few senior people who’re involved in the Alliance for Open Media and even they told me that they expect AV1 to five to 10 times more complex than HEVC at the end of the road. In two to three years after all optimization are done, it’s still going to be more complex than HEVC.
Dror Gill: 13:55 Now, if you ask me why it’s so complex I’ll tell you my opinion. Okay, this is my personal opinion. I think it’s because they invested a lot of effort in side stepping the patents (HEVC).
Dror Gill: 14:07 They need to get that compression efficiency which is the same as HEVC but they need to use algorithms that are not patented. They have methods that use much more CPU resources than the original patent algorithms to reach the same results. You can call it kind of brute force implementation of the same thing to avoid the patent issue. That’s my personal opinion, but the end result I think is clear, it’s going to be five to 10 times slower than HEVC. It has the same compression efficiency so I think it’s quite questionable. This whole notion of using AV1 to get better results.
Mark Donnigan: 14:45 Absolutely. If you can encode let’s say on a single computer with HEVC a full ABR stack, this is what people want to do. But here we’re talking speeds that are so slow let’s just try and do (encode) one stream. Literally what you’re saying is you’ll need five to 10 computers to do the same encode with AV1. I mean, that’s just not viable. It doesn’t make sense to me.
Dror Gill: 15:14 Yeah, why would you invest so much encoding into getting the same results. If you look at another aspect of this, let’s talk about hardware encode. Companies that have large data centers, companies that are encoding vast amount of video content are not looking into moving from the traditional software encoding and CPUs and GPUs, to dedicated hardware. We’re hearing talks about FPGAs even ASICs … by the way this is a very interesting trend in itself that we’ll probably cover in one of the next episodes. But in the context of AV1, imagine a chip that is five to 10 times larger than an HEVC chip and which is the same complexity efficiency. The question I ask again is why? Why would anybody design such a chip, and why would anybody use it when HEVC is available today? It’s much easier to encode, royalty issues have been practically solved so you know?
Mark Donnigan: 16:06 Yeah, it’s a big mystery for sure. One thing I can say is the Alliance for Open Media has done a great service to HEVC by pushing the patent holders to finalize their licensing terms … and ultimately make them much more rational shall we say?
Mark Donnigan: 16:25 Let me say that as we’re an HEVC vendor and speaking on behalf of others (in the industry), we’re forever thankful to the Alliance for Open Media.
Dror Gill: 16:36 Definitely, without the push from AOM and the development of AV1 we would be stuck with HEVC royalty issue until this day.
Mark Donnigan: 16:44 That was not a pretty situation a few years back, wow!
Dror Gill: 16:48 No, no, but as we said in the last episode we have a “happy ending” now. (reference to episode 1)
Dror Gill: 16:52 Billions of devices support HEVC and royalty issues are pretty much solved, so that’s great. I think we’ve covered HEVC and AV1 pretty thoroughly in two episodes but what about the other codecs? There’s VP9, you could call that the predecessor of AV1, and then there’s VVC, which is the successor of HEVC. It’s the next codec developed by MPEG. Okay, VP9 and VVC I guess we have a topic for our next episode, right?
Narrator: 17:23 Thank you for listening to the Video Insider podcast a production of Beamr limited. To begin using Beamr codecs today go to beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.
If you spend the majority of your time pondering hot topics in the streaming media space, you’re not alone.
Our team of image scientists and video encoding engineers are equally passionate about all things video related. Which means, not surprisingly, we spend our days discussing everything from video encoding, playback, workflow architecture, codecs, compression techniques, and more.
But as we looked around the industry for sources of information on these topics we noticed a dearth of information, which is why we thought to ourselves, “let’s do something about this,” and launched The Video Insiders podcast.
Hosted by our CTO Dror Gill and me, Mark Donnigan, the show takes on a refreshingly non-partisan tone as the purpose is to equip the industry with information that is free of bias, spin, and agenda.
Join us as we dive into today’s top of mind discussions: compression, codecs, encoding, transcoding, and video workflows. We cover it all, and we do hope that you find the information valuable as you contribute to the industry in your role, whether it be engineering, operations, or business.
Mark Donnigan: 00:15 Hi everyone, my name is Mark Donnigan and I am super excited to be opening this first episode of The Video Insiders. But before I say anything more I want to introduce you to my esteemed co-host, Dror Gill. Welcome Dror.
Dror Gill: 00:31 Thank you Mark. You know if you could do some e-motion estimation on me, you would also find out that I’m pretty excited to be opening this podcast and I think it’s gonna be awesome. Mark maybe we should tell everyone what they’re getting themselves into.
Mark Donnigan: 00:45 So, we are the Video Insiders and in every episode we will bring the latest inside information about what’s happening in the video technology industry from codecs both past, present and maybe even some not invented yet, you know. I don’t know, maybe we’ll invent one. What do you think Dror?
Mark Donnigan: 01:05 And obviously we’ll cover video in coding, trans coding and you know I’m a little bit more of a business guy. You’re a serious technologist and our listeners will get to know you but at the end of the day business drives this. So, we’ll cover business and I can’t think of too much that’s really off limits, right?
Dror Gill: 01:27 No, no, we’re gonna talk about everything, everything related to video. But first Mark I have to tell the listeners a small secret about you.
Dror Gill: 01:37 Those who know you best probably have been privileged to pound the keyboard.
Mark Donnigan: 01:43 Pound the keyboard, hmm, well you know when I, you know my new MacBook, you know it is pretty noisy. Man, what’s up with that keyboard, wow.
Dror Gill: 01:55 Not that kind of keyboard, I just happened to know that you’ve studied some classical piano performance and jazz in college, right?
Mark Donnigan: 02:03 Oo, this is true. Not too many people know that about me. That’s true, that is true. But it was a long time ago. But while we’re on the subject of music, you know, discussing beta release schedules kinda has a dual meaning for you, doesn’t it Dror?
Dror Gill: 02:21 Oh, yeah, yeah. You got me there. So, yeah, Beta is the name of my rock band. I’m a land singer of a rock band called Beta.
Dror Gill: 02:31 Yeah, we do some gigs, we’ve recorded a few tracks. So, when you discuss a beta schedule with me that could be just our tour schedule and not our milestones.
Mark Donnigan: 02:42 Well, there you have it. You know, look if we do our jobs this podcast just may rock our listeners video world, right?
Dror Gill: 02:50 Yeah, yeah, for sure. And if not, at least it will be the number one destination for their latest news and analysis about video compression, protection, monetization, applications, you know we’re gonna talk about all those buzzy words. Buzz words such as HDR, UHD, HEVC, AV1.
START OF DISCUSSION
HEVC triumph?
Mark Donnigan: 03:13 That’s pretty buzzy. That’s pretty buzzy. Well, hey HEVC and AV1, I’m glad you brought those codecs up, because some may say the topic is really been run into the ground. So, what do you think Dror?
Dror Gill: 03:26 I think it’s still a great topic, you know, because it has all the elements of a good story. It’s like a true hero’s journey. You know, you might say it has Hollywood glamour, has some tragedy, has its ups and downs and finally does have a happy ending.
Mark Donnigan: 03:42 Well, happy endings are good, so to kick this off for our listeners who don’t have a deep technical background. What is HEVC?
Dror Gill: 03:51 HEVC is the latest video standard. It was developed by the MPEG committee. MPEG is short for Motion Pictures Experts Group. Notice the motion pictures, yes they don’t call it video. We’re still in like the old days of when cinema just started. It’s not a video it’s just a bunch of moving pictures.
Dror Gill: 04:19 So, MPEG developed a lot of video standards. All the MPEG standards, there was MPEG 1, MPEG 2, MPEG 4 and then MPEG 4 AVC. Which some people know by the name H.264. And finally MPEG HEVC, which some people call H.265. But don’t do that Mark. Don’t call it H.265 because Leonardo might ban you.
Mark Donnigan: 04:43 Now, wait a second, how will Leonardo DiCaprio ban me?
Dror Gill: 04:46 Oh no, not Leonardo DiCaprio, Mark. You’re taking the Hollywood reference a bit too far. I’m talking about Leonardo Chiariglione. He’s been the founder and chairman of the MPEG committee for the past 30 years.
Dror Gill: 05:03 You know, many people call him the father of MPEG because you know he founded the committee and he’s been leading it. So, I heard a rumor that if he used the term H.265 instead of HEVC than he might ban you from the MPEG mail reflectors forever. So, you know, be careful.
Mark Donnigan: 05:19 Well, I mean pretty successful stuff that the MPEG committee has developed. MPEG 2, you know it’s used in all digital cable, satellite, terrestrial broadcast, DVD. MPEG 4, Blu-ray discs. Everything on the internet is streamed with MPEG 4.
Mark Donnigan: 05:53 Now, can you imagine the Leonardo that I’m thinking of and all the engineering geeks from MPEG standing there on the same stage as the other Leonardo from the MPEG committee. That’s really amazing. That’s pretty cool actually.
Dror Gill: 06:09 Yeah, yeah it is. You know, what’s even more amazing? Every generation of the standard they essentially double the compression efficiency. So, for example with HEVC you can get the same quality as H.264 but at half the bit rate. Now, I’m not saying you can do this all the time and not for every content and it’s not true for every resolution. I have to be very careful here cause you know, some people in the industry spread information that is not 100 percent accurate.
Dror Gill: 06:39 And sometimes it is possible that we will discuss such occasions on a podcast. But I’d like to be on the safe side so let’s say between 40 and 50 percent, depending on the resolution. That’s the improvement in compression efficiency would get with HEVC.
Mark Donnigan: 06:57 Sure, which is really great and pretty amazing. So, why isn’t everyone using HEVC instead of H.264? I mean, with video occupying, I mean the studies keep on, this number goes up seems like by the month by 70 percent the internet is video traffic. Netflix is like 15 percent of bandwidth consumed at peak viewing time. That’s staggering if you think about it. I would think everyone would just want to switch to HEVC and either enjoy the savings or be able to double their capacity. What’s going on there?
Dror Gill: 07:31 Yeah, yeah, that would have been really what you would expect with HEVC being much more efficient than H.264 but this is really where the tragedy part of our story comes in. There’s a lot of sophisticated algorithms which actually enable this great compression efficiency and many companies would jointly created the HEVC standard. They obviously have patents on all of these technologies that they developed. So, with H.264, the situation was simple. There were companies that developed algorithms, they had patents but there was a single patent pool that was organized by a company called MPEG LA. And you could just license all the patents very easily in this one stop shop. But with HEVC the problem was that the patent holders joined three different pool and some of them didn’t join any pools. So, you act of this unreasonable licensing terms which were published by one of the patent pools initially and their half of the industry was in complete chaos regarding HEVC licensing.
Mark Donnigan: 08:40 Wow, sounds pretty bad. Leonard Chiariglione wrote a pretty famous blog post called, ‘The Crisis, the Causes and it’s Solutions’, if I remember correctly.
Dror Gill: 08:55 Yes. It was very sad and he said that from his point of view the MPEG way has reached an end. They developed this great technology and then nobody uses them because of licensing issues. So, that was the peak of the tragedy but luckily for us and I think for the whole industry, finally, the patent holders came to their senses. And now everything is much more clear. I mean the two major patent’s pools which are MPEG LA and HEVC Advance, both of them are not charging any royalties for content.
Dror Gill: 09:34 And content providers around the industry and the third one Velos Media. They haven’t officially announced their licensing terms but everybody’s expecting them also to be reasonable and I’ve heard some things off the record that they’re also not going to charge for content. I mean, they don’t want to cut the branch they’re sitting on. If they’re unreasonable then nobody will license it and they get no money. So, I think basically where we are today you can say that HEVC patent world, these are mostly paid by the vendor of devices such as mobile phones and TVs. All those companies will put chips that support HEVC decoding and the devices and I understand the, Mark, that many of them are already supporting HEVC, right? A lot of devices out there.
Mark Donnigan: 10:18 You know earlier this year Beamr actually sponsored an industry survey which Tim Siglin a contributor to Streaming Media helped us with and also Streaming Media promoted it and helped put it together. The findings were really positive. This was completely a non partisan, fair and balanced survey. 66 percent of those that responded of which was a very high percentage of engineers by the way, more than 400. So, these are industry professionals – reported that HEVC was already in production. So, the reality is that though in some circles there still maybe a little bit of FUD. You know, that’s fear, uncertainty and doubt. The fact is is that more than half the industry has HEVC encoding pipelines running or they’re in the implementation stage. These things don’t happen overnight, so in some cases maybe they’re not encoding HEVC content yet but they will be. It’s a foregone conclusion.
Mark Donnigan: 11:33 77 percent so more than three out of four said HEVC was a very viable replacement for H.264. That’s obviously good news but if we’re making files that can’t be played then I suppose you could file this under the fake news category.
Dror Gill: 11:49 Yeah, if you encode the files somebody has to play them.
Mark Donnigan: 11:53 Exactly. And we’re gonna get to that when we talk about AV1, but when we look at the situation for device support it’s a little bit hard to get the exact numbers but industry seems to be kinda gathering around two billion devices is where most people, you know nod their head and say yes, we believe that number. That’s two billion today that support HEVC in hardware. So, if we think about, first of all it’s a massive number right? But you think about iPhones. Everything from the 6S forward supports HEVC and some markets iPhone penetration is way over 50 percent. And then when you factor in Samsung like the S8, S9, Note 8, Note 9, Galaxy Tab S3. I mean phones from LG, phones from Sony. You look at Macs, you look at PCs especially if they have the Intel Skylake or the Kaby Lake chips. Every 4K TV sold today, I mean I could go on and on.
Mark Donnigan: 12:54 Roku boxes, the new Apple TV, that is the Apple TV 4K. Chromecast even, Amazon Fire TV. So, all of those support HEVC. We heard from a very well placed industry insider that one MSO recently discovered of their users which number tens of millions, (that) more than 50 percent of their customers had at least one device in the home that supported HEVC. Which is very, encouraging.
Dror Gill: 13:32 Yeah, it’s very encouraging. It means that HEVC support is out there. I mean two billion devices, that’s a really large number. So, if you look at it, the situation with HEVC I would say it’s pretty clear. I mean, it’s half the bit rate of H.264 AVC. Royalty issues have mostly been solved, a few billion devices supported in hardware. So, there it is Mark. I think that’s the happy ending I was looking for. So, okay, that’s HEVC, where are we with AV1?
AV1 tragedy?
Mark Donnigan: 14:04 That’s a good question. You know, it’s unclear at the moment. Look, we are definitely fair and balanced here on the Video Insiders so we don’t present things from an agenda. First of all AV1’s open source, right? It is a supposedly a royalty free codec. It’s been developed by a extremely strong group of companies. Companies that if anybody can pull this off it’s Google, it’s Netflix, it’s Facebook, it’s Intel, it’s Microsoft, it’s Apple and there are others. These people are our friends and they’re good, they can build a codec. They could do this. They set out to develop the AV1 codec really in response to the royalty issue. You know, you might get different responses, I haven’t really gone around and asked privately any of these individual companies. But I’m guessing that off the record most would say yeah, if HEVC Advance had been rational, the Alliance for Open Media wouldn’t have been needed and AV1 wouldn’t exist and things would be more clear.
Dror Gill: 15:11 Right, and I understand they want to develop a royalty free codec but I think I heard you say “supposedly royalty free.” Cause I thought AV1 was royalty free, I mean look what it says here. I’m on the AOM website, I’m reading from the website, it says, “Collaborative effort to offer open royalty free and interoperable solutions for the next generation of media delivery.” So, it says royalty free, doesn’t it? I mean, can it be more clearer than that?
Mark Donnigan: 15:36 You can challenge me on that but it’s not that simple. Here’s the deal, the members of the Alliance for Open Media all agreed to contribute their own IP to the AV1 standard and not charge royalties for it, but it’s possible. Again, I’m saying “possible”, I’m not a lawyer, we’re not giving any legal advice so I wanna be careful, but it is possible that other parties who are not members of the Alliance for Open Media could have IP claims or at least could attempt to assert IP claims and to charge royalties on the codec. Now, alliance for open media, AOM, has set up a fund to indemnify licensees, or so I’ve heard. At the end of the day there’s just a lot of questions that don’t have answers. It’s one thing when it’s, you know, engineers sitting around at an industry conference or on a panel having a debate and discussion. It’s a whole different thing when a fortune 50 or a fortune 25 media company who could be facing hundreds of millions if not billions of dollars of fines or levees or whatever. You just can’t sort of sweep this stuff under the rug.
Mark Donnigan: 16:56 I think unfortunately that’s kinda what is happening a little bit in my opinion.
Dror Gill: 17:03 At the end of the day I think we need to encourage our listeners to do their own due diligence, you know, because in some industry conferences you can be led to believe the situation is very secure but really nobody is indemnifying you here. So, you really need to be careful. Anyway, on the device side Mark, how many devices support AV1 in hardware?
Mark Donnigan: 17:24 Yeah, so I’m glad you asked, and you know this is what my background is in, consumer electronics. I have a lot of experience there. We get into this situation where it’s easy to sit around and talk about creating files with these new exciting codecs but people kind of forget that VLC is not a viable solution to play back. I’ve been doing some research though and I actually was able to connect via email and in some cases have voice conversations with the SOC vendors, that is the system-on-the-chip vendors who are the ones at the end of the day that have to be convinced to integrate AV1 decode into their silicon.
Mark Donnigan: 18:25 But you know it’s these SOC vendors that at the end of the day is where I’m gonna get the real information. I heard what I anticipated to hear that obviously support today is zero. I heard varying degrees of responses such as we’re thinking about it. It’s getting onto the road map to a slightly more optimistic where some people said in 24 months we’ll have it, maybe sooner. But you should anticipate two years. And then on the other side I got the usual hey look, you know, when there’s a business case that is when I can’t sell a chip because it doesn’t have AV1, that’s when we’ll support it. You know the situation is really this, I am very confident again because the companies behind AV1 that somewhere in 2020 – It’s just not even possible to complete the engineering cycles, the fab cycles for the silicon any sooner than really the end of 2020. So, we’re really talking 24 months, we’re recording this podcast on the last day of October, October 31st, 2018. I anticipate that Christmas season 2020 there will be an (AV1) device.
Mark Donnigan: 19:44 Will it be a Roku box? Will it be a TV? Will it be a game console? Or even just some new product all together, I don’t know.
Mark Donnigan: 19:54 Maybe an Android phone, exactly. There will be an AV1 device and there’s no doubt that certain folks that are incented for AV1 to succeed will write articles saying that the world is about to flip. At the end of the day one device, a device does not build an ecosystem. It takes two billion devices that are in real consumer’s hands.
Mark Donnigan: 20:20 You know, that somebody doesn’t even have to download an app or install something, they just turn it on and it just works. And that we are years and years and years away.
Dror Gill: 20:33 Yeah, I think I agree with you. It will take time until the chips will come out and then tested and devices will come out. So, I’m very bullish about HEVC, I think the message for the industry now is very clear. HEVC is happening today and it’s here to stay.
Mark Donnigan: 21:06 Well let’s bring this to a close. It’s been an awesome discussion about how to compare HEVC and AV1.
Dror Gill: 21:13 Well, comparing codecs that’s a pretty big question and you never get the same answer. It really depends on who you ask so it’s a really interesting topic, but it seems like our time is up so how about we discuss that in our next episode?
Mark Donnigan: 21:29 I want to thank everyone for listening to The Video Insiders’ podcast. This is a production of Beamr Imaging Limited. Please subscribe on iTunes and we will be expanding to other platforms shortly. If you’d like to try out Beamr codecs in your lab or production environment we’re giving away up to 100 hours of HEVC and H.264 encoding every month. Go to https://beamr.com/free to get started immediately.
If you would like to join the discussion as a guest, send an email to thevideoinsiders@beamr.com with a suggested topic that you would like to share.
In case you missed it, Apple just snuck a little surprise into the first few days of 2018 in the form of their name appearing on the Alliance for Open Media (AOM) website as a Founding Member. As an ardent HEVC supporter, some may be shocked at this move by Apple. Blog post detailing Apple’s HEVC announcement. Now let’s see what it means for HEVC.
As a codec engineering company, Beamr has invested heavily in our HEVC implementation and we are proud of the best in class customers who are using it around the world to distribute video that is higher quality and up to 50% smaller than the H.264 version.
This means we do have a “vested” interest in HEVC being successful. At the same time, whether VP9 or AV1, we are always tracking the development of new codec technology so that we are in the strongest position to bring our extensive codec development resources to bear on market leading solutions.
Though we have an interest in HEVC becoming successful, we have invested resources and continue to do so, in order to understand AV1 in the areas of market readiness and licensing preparedness including IP questions, playback support and more.
In this article I will share the Beamr perspective that Apple joining the AOM reinforces the possibility that AV1 will be the successor to HEVC. However, with 1 billion HEVC enabled end points in the market, HEVC has legs for many years before a sufficiently large AV1 ecosystem will be built.
This position is also in alignment with many of our customers who are serving hundreds of millions of end users and must make codec decisions based on streams they can reliably deliver today.
After all, speculating on what may be coming in the future is not a luxury most of the industry enjoys because if they bet wrong, it could impact tens of millions of users negatively. There is a huge difference between advanced technology development (e.g. what happens in the lab) and the realities of production (that which generates revenue).
Once the AOM locks down the AV1 spec, you can expect many shootouts and comparisons with HEVC to be published. But let’s take a look at how HEVC compares to AV1, given what we know now.
AV1 Readiness compared to HEVC.
HEVC was ratified in 2013 while the AV1 bitstream was set to be frozen in Q1 2017, yet even now the AV1 bitstream has not been completed. Developing software timelines, committing to them, and then meeting them, is far from an exact science. Thus the delay is not completely the fault of the AOM development community since it is endemic to the software development lifecycle. Innovation is difficult to schedule. The point is, AV1 will be ready when it is ready. Which means commercial plans that hinge on the delivery of AV1 before 2020 or 2021 could be at risk given the uncertainty of when the standard will be ratified.
AV1 Compression Efficiency compared to HEVC.
HEVC is recognized to be 40-50% more efficient than AVC (H.264), and AV1 is hoped to be up to 30% more efficient than HEVC (H.265).
However, while HEVC’s compression efficiency has already been reached by advanced encoder implementations such as Beamr 5, AV1’s 30% efficiency claim over HEVC has not been proven outside of an extremely limited (small) set of files.
In any case any improvement can be validated only after the spec is final and the tools included in AV1 are decided upon. At that point the race to realize these gains will start, balancing the computing resources needed and maturing the rate control algorithms. But just as HEVC did not reach its planned 50% efficiency in the first release, taking multiple years to achieve, the AOM developers will need to work very hard for the next 2 to 3 years before significant gains over HEVC will be seen.
AV1 Royalty and IP constraints compared to HEVC.
There are three HEVC Patent pools which license the technology used in HEVC implementations: MPEG-LA, HEVC Advance, and Velos Media. Both MPEG-LA and Velos Media do not charge license fees for content distribution (See the MPEG-LA HEVC License Summary and the Velos Media FAQ), and HEVC Advance does not charge a license fee for free content distribution, such as public broadcasts and ad-funded commercial broadcasts (see page 3 of the HEVC Royalty Rates document).
Even Technicolor, that licenses its HEVC patents outside of the 3 patent pools, has publicly declared that they will not charge license fees from content providers. In addition, royalty schedules are being (have been) amended down, and it seems the Patent pools are aware that a more friendly approach is needed.
AV1 cannot guarantee a royalty-free offer.
Yes, that’s correct, I said it! Now here’s why.
While AV1 claims to be royalty-free, many industry players have missed the fact that the Alliance for Open Media does not provide indemnification to companies who use AV1 against patent claim violations.
Since some of the algorithms used in AV1 bear a resemblance to corresponding H.264 and HEVC algorithms, there is some probability that the IP in AV1 could infringe on AVC and/or HEVC Patents. In fact, delays to the ratification of the AV1 standard might well be due to legal teams who are examining the final algorithms exactly for these cases.
To be fair, IP questions are hardly ever cut and dried, and there are many unknowns and “what-if’s” to be discussed. But the lack of clarity regarding the AV1 IP situation, and the fact that AOM is not offering indemnification for IP infringement, makes the “royalty free” claim at this point more of a wish than a solid fact.
If you are still not convinced that AV1 offering a royalty-free codec could be problematic, consider that for VP8 and VP9 Google needed to license the H.264 patents from MPEG-LA. If an infringement action is identified with AV1, and if the courts rule in the plaintiff’s favor, the legal exposure will be of the magnitude that headlines will be penned and stock prices hammered. Velos Media, one of the 3 HEVC patent pools, has already warned in its FAQ:
“As it relates to royalties, we know that VP9 incorporates patented technologies, including some of the patents being licensed by Velos Media for HEVC. And, while AV1 has not yet been publicly released, it may also incorporate patented technology from many parties.”
Try slipping that little disclaimer by a corporate IP attorney! When a licensing body directly references a new technology as being possibly infringing, it is worth paying attention to.
Let’s look at why the AOM members feel it’s so important to have a royalty-free platform.
Royalties are a pain, that is, when you are the party that needs to pay. Not only do you need to factor this added cost into your business model but in some cases, the tracking and reporting burden by itself represents a real difficulty and may limit certain business models from being feasible.
You can never know what patent holder’s next demand will be. The HEVC fragmented IP pools and shifting fee structures prove the point. The AOM is absolutely correct in wanting to address this with AV1.
Confusion over who to pay. The lack of clarity overpayments is delaying adoption even further. It seems AOM members are seeking control over a critical component of their technology that is not royalty free.
While all these arguments make sense, AOM cannot guarantee its users that this is a royalty-free codec; all they can suggest is that AOM members will not ask for royalties. But the thousands of patent holders for block-based codecs that are not a part of AOM are still out there, and the generosity of AOM could come at their expense.
For an interesting analysis on the validity and value of the MPEG-LA and HEVC Advance HEVC patent pools, you will want to read this Unified Patents article as it provides a perspective on how the courts look at the patents that are contained in a pool. In short, it’s probably not a wise legal move to assume that the AOM has everyone adopting AV1 “covered.”
AV1 Encoding CPU Performance compared to HEVC.
HEVC encoding CPU performance is advancing at a rapid rate. At IBC 2017 Beamr demonstrated six simultaneous 10-bit 4Kp60 live channels being encoded on a single Intel Xeon Scalable Platinum 8180 dual-socket server.
This encoding speed is in contrast to AV1, which has not been optimized, but is currently running about 100 times slower than real-time on a single server. Aside from the fact that AV1 is not available for low latency live encoding workflows, the operational cost delta of running an AV1 encoding service versus HEVC is staggeringly higher for AV1.
Download the Intel solution guides detailing Beamr HEVC codec SDK performance and applications on Intel processors.
Beamr’s HEVC encoder has been under active development for more than five years, and our CPU performance has been consistently improved by way of algorithmic and code optimizations. AV1 developers will bring improvements to the encoding speed, but it will most certainly follow the same development trend of every codec before it, including HEVC.
In other words, it can only happen over a period of years, and only if a group of dedicated engineers focuses on it day and night. Optimizing a codec is not a hobby. But even in the end after it is fully optimized AV1 will be slower because of the added mathematical complexity needed for it to achieve higher efficiency.
Video distributors with capex, opex, or physical space constraints will find the bitrate efficiency gains of AV1 will come with a very high operational cost. This Jan Ozer article from Streaming Media provides further context on AV1 performance.
AV1 Playback and Decoding Performance compared to HEVC.
HEVC hardware decoder support exists today in more than 1 billion devices spanning the most popular computing and mobile operating systems in the market like iOS, macOS, Android, and Windows. And in addition, low power hardware implementations for HEVC exist on Intel and ARM-based chips, as well as hundreds of millions of SoC’s shipping in CE devices such as TV’s, media players and game consoles. HEVC is a de-facto standard in all UHD TV’s found in the market.
According to the Consumer Technology Association (CTA) between 2014 and thru 2017, somewhere in the neighborhood of 175 million UHD TV’s will have been sold. Source: CTA 2017 presentation, data from GfK
This is an impressive HEVC footprint, and it’s only going to grow stronger. In contrast, the only AV1 playback environment available today is an early alpha implementation of the Mozilla browser Nightly build.
As the AV1 spec is not yet locked by the AOM it’s no wonder that when we reached out to the best-known silicon vendors who are supplying the media processing and video decoding chips to the most popular media player vendors, game console makers and TV OEM’s, everyone reported without exception that they cannot begin planning AV1 support in the absence of a ratified spec.
AV1 support in silicon is a minimum of 24 months out (Q1 2020). But silicon is just that, silicon. Chips have to be designed and integrated into consumer products before the advanced capabilities that chip vendors are including can be available. But, this is a chicken and egg situation. Afterall, why would a CE company go through the extra engineering cycles and increase their BOM to support a codec that is yet to be adopted across the ecosystem?
Should Apple influence your codec adoption decision?
With Apple’s commitment to HEVC, it’s a mystery of what their intentions are in joining the AOM. But we do know this – Apple has a solid HEVC roadmap with a vertically integrated HEVC video encoding, distribution and playback technology stack across all devices.
This means that today, you can reach the Apple ecosystem with H.264 and HEVC.
For video distributors looking for smaller bandwidth footprints, opting out of HEVC means they will lose a whopping 54% of the North American mobile market assuming the selected codec isn’t supported by Apple. This is hardly a prospect that any executive or encoding head will agree to, which means the question of whether to support AV1 or HEVC could come down to compatibility.
Any video service unable to match the performance of HEVC on Apple, will have a difficult time competing with services delivering high quality 1080p HD video at bitrates well under 2 Mbps. HEVC is available today across the all too important Apple ecosystem, while AV1 is not supported (today).
HEVC is the codec for today. Is AV1 the codec for the future?
Remember how I said that Beamr is actively tracking all new video technology and codec developments and that this applies to AV1? Well, we do believe that AV1 could be a factor at some point in the future.
There are those who always wait for next years model. The trouble with this approach is that by delaying, you miss out on technology leaps that could have afforded a significant advantage to your company. This is especially true with the decision to move ahead or hold on adopting HEVC in anticipation that AV1 will be cheaper and provide an added efficiency benefit.
The reality is that HEVC is able to reach 40% to 50% efficiency gains over H.264 today. These aren’t theoretical numbers or only possible on a limited set of content. Beamr has customers distributing content around the world, and enjoying bitrate savings in this range. Even Apple in their WWDC2017 announcements of HEVC, used the numbers 40% and 50% savings when talking about their decision to adopt the HEVC standard.
Ask yourself, what is the opportunity cost incurred by continuing with H.264 for the next 24 to 36 months as you wait for a relatively small AV1 playback footprint to emerge?
This is why most of the industry operating a commercial service is opting to realize the benefits of HEVC today while keeping tabs on the development of AV1 (for the future).
Still not sure…
Consider that Amazon Prime and Netflix are both members of AOM. And both are active in AV1 development and testing, yet Amazon and Netflix are users of HEVC.
Why would they do this? It’s simple. HEVC serves them well by being compatible with more than 1 billion devices and enabling premium video experiences at bitrates that are 40 to 50% less than H.264. Now, one may think that HEVC is only being used for 4K content by these services, but we know that Amazon is using HEVC in emerging markets for lower resolutions.
But what about Google, they are Founding members of the AOM and ardent supporters of alternative codecs? It’s interesting to note that Google supports HEVC in Chromecast which is clearly required for content services (some who are in the AOM like Netflix and Amazon) to stream 4K HDR video. But will Google use HEVC for their own services on a wider basis in the future? We do not know.
One situation in the market that must be faced by Google and YouTube is what will happen if Apple deprecates H.264, and mandates all apps that stream video to Apple devices leverage HEVC. Remember Flash? A similar situation occurred with HLS, the mandatory streaming protocol for Apple devices.
Remember how I pointed out that Apple is vertically integrating with HEVC for video and HEIC (HEVC I-frame) for mobile image capture and display? I think now you can see how a codec selection decision by a vendor like Apple can move the entire industry.
Apple has chosen HEVC for production and any video distributor can encode in HEVC and transmit to a user with iOS 11 or macOS High Sierra and know that it will play perfectly.
HEVC is a robust standard that has broad support with extensive development from the largest encoding vendors in the industry. And HEVC is widely adopted on the device side with major services like Amazon, Apple, and Netflix using it now. HEVC is not going away.
Beamr’s view is that HEVC is the codec for today, and AV1 may possibly be the codec of tomorrow. But for sure, with HEVC, we can all enjoy more video and better quality as HEVC enables new applications, experiences, and innovations to be transmitted to users today.
![Codec Efficiency is in the Eye of the Measurer. [podcast]](https://blog.beamr.com/wp-content/uploads/2018/12/Beamr-Podcast-banner-B-1-1-800x544.jpg)
![Will the Story End a Tragedy or Triumph? [podcast]](https://blog.beamr.com/wp-content/uploads/2018/12/Beamr-Podcast-banner-v3a-1-800x544.jpg)
