Easy & Safe Codec Modernization with Beamr using Nvidia GPUs
Following a decade where AVC/H.264 was the clear ruler of the video encoding world, the last years have seen many video coding options battling to conquer the video arena. For some insights on the race between modern coding standards you can check out our corresponding blog post.
Today we want to share how easy it can be to upgrade your content to a new and improved codec in a fast, fully automatic process which guarantees the visual quality of the content will not be harmed. This makes the switchover to newer encoders a smooth, easy and low cost process which can help accelerate the adoption of new standards such as HEVC and AV1. When this transformation is done using a combination of Beamr’s technology with the Nvidia NVENC encoder, using their recently released APIs, it becomes a particularly cutting-edge solution, enjoying the benefits of the leading solution in hardware AV1 encoding.
The benefit of switching to more modern codecs lies of course in the higher compression efficiency that they offer. While the extent of improvement is very dependent on the actual content, bitrates and encoders used, HEVC is considered to offer gains of 30%-50% over AVC, meaning that for the same quality you can spend up to 50% fewer bits. For AV1 this increase is generally a bit higher.. As more and more on-device support is added for these newer codecs, the advantage of utilizing them to reduce both storage and bandwidth is clear.
Generally speaking, performing such codec modernization involves some non-trivial steps.
First, you need to get access to the modern encoder you want to use, and know enough about it in order to configure the encoder correctly for your needs. Then you can proceed to encoding using one of the following approaches.
The first approach is to perform bit-rate driven encoding. One possibility is to use conservative bitrates, in which case the potential reduction in size will not be achieved. Another possibility is to set target bitrates that reflect the expected savings, in which case there is a risk of losing quality. For example, In an experimental test of files which were converted from their AVC source to HEVC, we found that on average, a bitrate reduction of 50% could be obtained when using the Beamr CABR codec modernization approach. However, when the same files were all brute-force encoded to HEVC at 50% reduced bitrate, using the same encoder and configuration, the quality took a hit for some of the files.
This example shows the full AVC source frame on top, with the transcodes to HEVC below it. Note the distortion in the blind HEVC encode, shown on the left, compared to the true-to-source video transformed with CABR on the right.
The second approach is to perform the transcode using a quality driven encode, for instance using the constant QP (Quantization Parameter) or CRF (Constant Rate Factor) encoding modes with conservative values, which will in all likelihood preserve the quality. However, in this case you are likely to unnecessarily “blow up” some of your files to much higher bitrates. For example, for the UGC content shown below, transcoding to HEVC using a software encoder and CRF set to 21 almost doubled the file size.
Yet another approach is to use a trial and error encode process for each file or even each scene, manually verifying that a good target encoding setup was selected which minimizes the bitrate while preserving the quality. This is of course an expensive and cumbersome process, and entirely unscalable.
By using Beamr CABR this is all done for you under the hood, in a fully automatic process, which makes optimized choices for each and every frame in your video, selecting the lowest bitrate that will still perfectly preserve the source visual quality. When performed using the Nvidia NVENC SDK with interfaces to Beamr’s CABR technology, this transformation is significantly accelerated and becomes even more cost effective.
The codec modernization flow is demonstrated for AVC to HEVC conversion in the above high-level block diagram. As shown here, the CABR controller interacts with NVENC, Nvidia’s hardware video encoder, using the new APIs Nvidia has created for this purpose. At the heart of the CABR controller lies Beamr’s Quality Measure, BQM, a unique, patented, Emmy award winning perceptual video quality measure. BQM has now been adapted and ported to the Nvidia GPU platform, resulting in significant acceleration of the optimization process .
The Beamr optimization technology can be used not only for codec modernization, but also to reduce bitrate of an input video, or of a target encode, while guaranteeing the perceptual quality is preserved, thus creating encodes with the same perceptual quality at lower bitrates or file sizes. In any and every usage of the Beamr CABR solution, size or bitrate are reduced as much as possible while each frame of the optimized encode is guaranteed to be perceptually identical to the reference. The codec modernization use case is particularly exciting as it puts the ability to migrate to more efficient and sophisticated codecs, previously used primarily by video experts, into the hands of any user with video content.
For more information please contact us at info@beamr.com
Video engineers dedicated to engineering encoding technologies are highly skilled and hyper-focused on developing the foundation for future online media content. Such a limited pool of experts in this field creates a lot of opportunity for growth and development, it also means there must be a level of camaraderie and cooperation between different methodologies.
In past episodes, you’ve seen The Video Insiders compare codecs head-to-head and debate over their strengths and weaknesses. Today, they are tackling a deeper debate between encoding experts: the advantages and disadvantages of proprietary technology vs. community-driven open source.
In Episode 05, Tom Vaughan surprises The Video Insiders as he talks through his take on open source vs. proprietary technology.
Want to join the conversation? Reach out to TheVideoInsiders@beamr.com
TRANSCRIPTION (lightly edited to improve readability only)
Mark Donnigan: 00:00 In this episode, we talk with a video pioneer who drove a popular open source codec project before joining a commercial codec company. Trust me, you want to hear what he told us about proprietary technology, open source, IP licensing, and royalties.
Announcer: 00:18 The Video Insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation codec nerd and a marketing guy who knows what iframes and macroblocks are. Here are your hosts, Mark Donnigan and Dror Gill.
Mark Donnigan: 00:56 You know, we’re like encoding geeks. I mean, are there even 180 of us in the world?
Dror Gill: 01:01 I don’t know. I think you should count the number of people who come to Ben Wagoner’s compressionist breakfast at NAB, that’s about the whole industry, right?
Mark Donnigan: 01:09 Yeah. That’s the whole industry.
Mark Donnigan: 01:11 Hey, we want to thank, seriously in all seriousness, all the listeners who have been supporting us and we just really appreciate it. We have an amazing guest lined up for today. This is a little personal for me. It was IBC 2017, I had said something about a product that he was representing, driving, developing at the time. In fact, it was factually true. He didn’t like it so much and we exchanged some words. Here’s the ironic thing, this guy now works for us. Isn’t that amazing, Dror?
Mark Donnigan: 01:52 You know what, and we love each other. The story ended well, talk about a good Hollywood ending.
Mark Donnigan: 01:58 Well, we are talking today with Tom Vaughn. I’m going to let you introduce yourself. Tell the listeners about yourself.
Tom Vaughn: 02:10 Hey Mark, hey Dror. Good to be here.
Tom Vaughn: 02:12 As Mark mentioned, I’m Beamr’s VP of strategy. Joined Beamr in January this year. Before that I was Beamr’s, probably, primary competitor, the person who started and led the x265 project at MulticoreWare. We were fierce competitors, but we were always friendly and always friends. Got to know the Beamr team when Beamr first brought their image compression science from the photo industry to the video industry, which was three or four years ago. Really enjoyed collaborating with them and brainstorming and working with them, and we’ve always been allies in the fight to make new formats successful and deal with some of the structural issues in the industry.
Dror Gill: 03:02 Let me translate. New formats, that means HEVC. Structural issues, that means patent royalties.
Tom Vaughn: 03:13 Yeah, we had many discussions over the years about how to deal with the challenging macro environment in the codec space. I decided to join the winning team at Beamr this year, and it’s been fantastic.
Mark Donnigan: 03:28 Well, we’re so happy to have you aboard, Tom.
Mark Donnigan: 03:32 I’d like to just really jump in. You have a lot of expertise in the area of open source, and in the industry, there’s a lot of discussion and debate, and some would even say there’s religion, around open source versus proprietary technology, but you’ve been on both sides and I’d really like to jump into the conversation and have you give us a real quick primer as to what is open source.
Tom Vaughn: 04:01 Well, open source is kind of basic what it says is that you can get the full source code to that software. Now, there isn’t just one flavor of open source in terms of the software license that you get, there are many different open source licenses. Some have more restrictions and some have less restrictions on what you can do. There are some well known open source software programs and platforms, Linux is probably the most well known in the multimedia space, there’s FFmpeg and Libav. There’s VLC, the multimedia player. In the codec space, x264, x265, VP9, AV1, et cetera.
Dror Gill: 04:50 I think the main attraction of open source, I think, the main feature is that people from all over the world join together, collaborate, each one contributes their own piece, then somehow this is managed together. Every bug that is discovered, anyone can fix it, because the source is open. This creates kind of a community and together a piece of software is created that is much larger and more robust than anything that a single developer could do on his own.
Tom Vaughn: 05:23 Yeah, ideally the fact that the source code is open means that you have many sets of eyes, not only trying the program, but able to go through the source code and see exactly how it was written and therefore more code review can happen. On the collaboration side, you’re looking for volunteers, and if you can find and energize many, many people worldwide to become enthusiastic and devote time or get their companies motivated to allocate developers full- or part-time to a particular open source project, you get that collaboration from many different types of people with different individual use cases and motivations. There are patches submitted from many different people, but someone has to decide, does that patch get committed or are there problems with that? Should it be changed?
Tom Vaughn: 06:17 Designed by a committee isn’t always the optimal, so someone or some small group has to decide what should be included, what should be left out.
Dror Gill: 06:27 It’s interesting to see, actually, the difference between x264 and x265 in this respect, because x264, the open source implementation of x264 was led by a group of developers, really independent developers, and no single company was owning or leading the development of that open source project. However, with x265, which is the open source implementation of HEVC, your previous company, MulticoreWare, has taken the lead and devoted, I assume, most of the development resources that have gone into the open source development, most of the contributions came from that company, but it is still an open source project.
Tom Vaughn: 07:06 That’s right. x264 was started by some students at a French university, and when they were graduating, leaving the university, they convinced the university to enable them to take the code with them, essentially under an open source license. It was very much grassroots open source beginnings and execution where developers may come and go, but it was a community collaboration.
Tom Vaughn: 07:31 I started x265 at MulticoreWare with a couple of other individuals, and the way we started it was finding some commercial companies who expressed a strong interest in such a thing coming to life and who were early backers commercially. It was quite different. Then, because there’s a small team of full-time developers on it working 40 hours plus a week, that team is moving very fast, it’s organized, it’s within a company. There was less of a need for a community. While we did everything we could to attract more external contributors, attracting contributors is always a challenge of open source projects.
Mark Donnigan: 08:14 What I hear you saying, Tom, is it sounds like compared to the x264 project, the x265 project didn’t have as large of a independent group of contributors. Is that …?
Tom Vaughn: 08:29 Well, x264 was all independent contributors.
Tom Vaughn: 08:33 And still is, essentially. There are many companies that fund x264 developers explicitly. Chip companies will fund individual developers to optimize popular open source software projects for their instruction set. AVX, AVX2, AVX512, essentially, things like that.
Tom Vaughn: 08:58 HEVC is significantly more complex than AVC, and I think, if I recall correctly, x265 already has three times the number of commits than x264, even though it’s only been in existence for one third of the life.
Dror Gill: 09:12 So Tom, what’s interesting to me is everybody’s talking about open source software being almost synonymous with free software. Is open source really free? Is it the same?
Tom Vaughn: 09:23 It can be at times. One part depends on the license and the other part depends on how you’re using the software. For example, if it’s a very open license like Apache, or BSD, or UIUC, that’s an attribution only license, and you’re pretty much free to create modifications, incorporate the software in your own works and distribute the resulting system.
Tom Vaughn: 09:49 Software programs like x264 and x265 are licensed under the GNU GPL V2, that is an open source license that has a copyleft requirement. That means if you incorporate that in a larger work and distribute that larger work, you have to open source not only your modifications, but you have to open source the larger work. Most commercial companies don’t want to incorporate some open source software in their commercial product, and then have to open source the commercial product. The owners of the copyright of the GPL V2 code, x264 LLC or MulticoreWare, also offer a commercial license, meaning you get access to that software, not under the GNU GPL V2, but under a separate, different license, in which case for you, it’s not open source anymore. Your commercial license dictates what you can and can’t do. Generally that commercial license doesn’t include the copyleft requirement, so you can incorporate it in some commercial product and distribute that commercial product without open sourcing your commercial product.
Dror Gill: 10:54 Then you’re actually licensing that software as you would license it from a commercial company.
Tom Vaughn: 10:59 Exactly. In that case it’s not open source at all, it’s a commercial license.
Dror Gill: 11:04 It’s interesting what you said about the GPL, the fact that anything that you compile with it, create derivatives of, incorporate into your software, you need to open source those components that you integrate with as well. I think this is what triggered Steve Ballmer to say in 2001, he said something like, “Open source is a cancer that spreads throughout your company and eats your IP.” That was very interesting. I think he meant mostly GPL because of that requirement, but the interesting thing is that he said that in 2001, and in 2016 in an interview, he said, “I was wrong and I really love Linux.” Today Microsoft itself open sources a lot of its own development.
Mark Donnigan: 11:48 That’s right. Yeah, that’s right.
Mark Donnigan: 11:50 Well Tom, let’s … This has been an awesome discussion. Let’s bring it to a conclusion. When is proprietary technology the right choice and when is open source maybe the correct choice? Can you give the listeners some guidelines?
Tom Vaughn: 12:08 Sure, people are trying to solve problems. Engineers, companies are trying to build products and services, and they have to compete in their own business environment. Let’s say you’re a video service and you run a video business. The quality of that video and the efficiency that you can deliver that video matters a lot. We know what those advantages of open source are, and all things being equal, people gravitate towards open source a lot because engineers feel comfortable actually seeing the source code, being able to read through it, find bugs themselves if pushed to the limit.
Tom Vaughn: 12:45 At the end of the day, if an open source project can’t produce the winning implementation of something, you shouldn’t necessarily use it just because it’s open source. At the end of the day you have a business to run and what you want is the most performant libraries and platforms to build your business around. If you find that a proprietary implementation in the long run is more cost effective, more efficient, higher performance, and the company that is behind that proprietary implementation is solid and is going to be there for you and provide a contractual commitment to support you, there’s no reason to not choose some proprietary code to incorporate into your product or service.
Tom Vaughn: 13:32 When we’re talking about codecs, there are particular qualities I’m looking for, performance, how fast does it run? How efficiently does it utilize compute resources? How many cores do I need in my server to run this in real time? And compression efficiency, what kind of video quality can I get at a given bit rate under a given set of conditions? I don’t want the second best implementation, I want the best implementation of that standard, because at scale, I can save a lot of money if I have a more efficient implementation of that standard.
Mark Donnigan: 14:01 Those are excellent pointers. It just really comes back to we’re solving problems, right? It’s easy to get sucked into religious debates about some of these things, but at the end of the day we all have an obligation to do what’s right and what’s best for our companies, which includes selecting the best technology, what is going to do the best job at solving the problems.
Mark Donnigan: 14:24 Thank you again for joining us.
Mark Donnigan: 14:33 Well, we want to thank you the listener for, again, joining The Video Insiders. We hope you will subscribe. You can go to thevideoinsiders.com and you can stream from your browser, you can subscribe on iTunes. We’re on Spotify. We are on Google Play. We’re expanding every day.
Announcer: 14:57 Thank you for listening to The Video Insiders podcast, a production of Beamr Limited. To begin using Beamr’s codecs today, go to Beamr.com/free to receive up to 100 hours of no-cost HEVC and H.264 transcoding every month.
Apple recently announced their plans to enable HEVC on up to one billion devices with the release of iOS 11 and High Sierra later this year, causing video services to reevaluate their workflows and the codecs they are using. With video codecs in a constant state of ongoing development, it can be hard to know which one is right to choose. But with the ever increasing pressure to reduce bitrates and upgrade quality, video service providers looking for a competitive edge, are examining next generation encoding platforms, namely HEVC and VP9.
To aid in the evaluation process, our software video encoding engineers have compiled a guide that will eliminate the guesswork of comparing HEVC and VP9. This guide provides a qualitative overview of the strengths, weaknesses, and key differences between the HEVC (H.265) and VP9 codec toolsets, including:
Jan Ozer, one of the world’s top experts on video compression and streaming and Contributing Editor of Streaming Media Magazine, has done a thorough evaluation of Beamr Video using test clips that he encoded. The evaluation included analyzing the quality of the resulting clips using objective measures such as PSNR, SSIM and VQM, and conducting extensive viewing of the optimized clips, assessing their subjective quality compared to the original clips. Following this evaluation, Jan Ozer concludes that “for 99.99% of the expert and average viewers, the Beamr-processed clips would be perceptually identical to their source”.
As for bitrate reduction, Jan has found that Beamr Video was able to reduce the bitrate of his test clips by 50% on average in High Quality mode, and 35% on average in Best Quality mode. Providing such significant bitrate reduction, while producing video clips that are perceptually identical to the source, brings Jan to the following conclusion: “If you’re spending big dollars on bandwidth, Beamr represents an opportunity to reduce it significantly, essentially delivering the benefit of HEVC/VP9 without the implementation hassles”.
Jan Ozer has worked in digital video since 1990, and is the author of over 20 books on video compression, editing and production, which have been adopted as textbooks by multiple colleges and universities. As Contributing Editor at Streaming Media Magazine, Jan has reviewed most cloud, enterprise and desktop encoding tools, and has worked with many online video platforms (OVPs) and live streaming services.
You can read Jan’s article summarizing his findings about Beamr video here and download his full report here.
![In the battle between open source & proprietary technology, does video win? [podcast]](https://blog.beamr.com/wp-content/uploads/2019/01/Beamr-Podcast-banner-E2-1-1-800x544.jpg)
