Anyone familiar with the streaming video industry knows that we love our acronyms. You would be hard-pressed to have a conversation about the online video industry without bringing one up…
But before we can look at the future, we have to take a moment to revisit the past.
The year 2018 marks the 15-year anniversary of AVC and in this episode, we visit the process and lifecycle of standardization to adoption and what that means for the future of these codecs.
Tune in to Episode 03: The Future of 3 Character Codecs or watch video below.
Want to join the conversation?
Reach out to TheVideoInsiders@beamr.com.
TRANSCRIPTION (lightly edited for improved readability)
Mark Donnigan: 00:52 It is, it is episode three. So, today we have a really exciting discussion as we consider the future of codecs named with three characters.
Dror Gill: 01:03 Three character codecs, okay, let’s see.
Mark Donnigan: 01:06 Three character codecs.
Dror Gill: 01:09 I can think of …
Mark Donnigan: 01:09 How many can you name?
Mark Donnigan: 01:21 Well, you just named three that I was thinking about and we’re gonna discuss today! We’ve already covered AV1. Yeah, yeah, you answered correctly, but we haven’t really considered where AVC, VP9, and VVC fit into the codec stew. So when I think about AVC, I’m almost tempted to just skip it because isn’t this codec standard old news? I mean, c’mon. The entire video infrastructure of the internet is enabled by AVC, so what is there to discuss?
Dror Gill: 01:57 Yeah. You’re right. It’s like the default, but in fact, the interesting thing is that today, we’re (in) 2018 and this is the twenty year anniversary of AVC. I mean, ITU issued the call for proposals, their video coding expert group, issued the call for proposal for a project. At the time was called H26L, and their target was to double the coding efficiency, which effectively means halving the bit rate necessary for given level of fidelity. And that’s why it was called H26L, it was supposed to be low bit rate.
Mark Donnigan: 02:33 Ah! That’s an interesting trivia question.
Dror Gill: 02:35 That’s where the L came from!
Mark Donnigan: 02:36 I wonder how many of our listeners knew that? That’s kind of cool. H26L.
Dror Gill: 02:42 But they didn’t go alone. It was the first time they joined forces in 2001 with the ISO MPEG, that’s the same Motion Pictures Experts Group, you know we discussed in the first episode.
Mark Donnigan: 02:56 That’s right.
Dror Gill: 02:57 And they came together, they joined forced, and they created JVT, that was the Joint Video Team, and I think it’s a great example of collaboration. There are standards by dealing with video communication standards, and ISO MPEG, which is a standards body dealing with video entertainment standards. So, finally they understood that there’s no point in developing video standards for these two different types of applications, so they got all the experts together in the JVT and this group developed what was the best video compression standard at the time. It was launched May 30, 2003.
Mark Donnigan: 03:35 Wow.
Dror Gill: 03:36 There was one drawback with this collaboration in that the video standard was known by two names. There was the ITU name which is H.264. And then there’s the ISO MPEG name which is AVC, so these created some confusion at the start. I think by now, most of our listeners know that H.264 and AVC are two of the same.
Mark Donnigan: 03:57 Yeah, definitely. So, AVC was developed 15 years ago and it’s still around today.
Dror Gill: 04:02 Yeah, yeah. I mean, that’s really impressive and it’s not only around, it’s the most popular video compression standard in the world today. I mean, AVC is used to deliver video over the internet, to computers, televisions, mobile devices, cable, satellite, broadcast, and even blu-ray disks. This just shows you how long it takes from standardization to adoption, right? 15 years until we get this mass market adoption market dominance of H.264, AVC as we have today.
Dror Gill: 04:31 And the reason it takes so long is that, we discussed it in our first episode, first you need to develop the standard. Then, you need to develop the chips that support the standard, then you need to develop devices that incorporate the chip. Even when initial implementation of the codec got released, they are still not as efficient as they can be, and it takes codec developers more time to refine it and improve the performance and the quality. You need to develop the tools, all of that takes time.
Mark Donnigan: 04:59 It does. Yeah, I have a background in consumer electronics and because of that I know for certainty that AVC is gonna be with us for a while and I’ll explain why. It’s really simple. Decoding of H.264 is fully supported in every chip set on the market. I mean literally every chip set. There is not a device that supports video which does not also support AVC today. It just doesn’t exist, you can’t find it anywhere.
Mark Donnigan: 05:26 And then when you look at in coding technologies for AVC, H.264, (they) have advanced to the point where you can really achieve state of the art for very low cost. There’s just too much market momentum where the encode and decode ecosystems are just massive. When you think about entertainment applications and consumer electronics, for a lot of us, that’s our primary market (that) we play in.
Mark Donnigan: 05:51 But, if you consider the surveillance and the industrial markets, which are absolutely massive, and all of these security cameras you see literally everywhere. Drone cameras, they all have AVC encoders in them. Bottom line, AVC isn’t going anywhere fast.
Dror Gill: 06:09 You’re right, I totally agree with that. It’s dominant, but it’s still here to stay. The problem is that, we talked about this, video delivery over the internet. The big problem is the bandwidth bottleneck. With so much video being delivered over the internet, and then the demand for quality is growing. People want higher resolution, they want HDR which is high dynamic range, they want higher frame rate. And all this means you need more and more bit rate to represent the video. The bit rate efficiency that is required today is beyond the standard in coding in AVC and that’s where you need external technologies such as content adaptive encoding perceptual optimization that will really help you push AVC to its limits.
Mark Donnigan: 06:54 Yeah. And Dror, I know you’re one of the inventors of a perceptual optimization technique based on a really unique quality measure, which I’ve heard some in the industry believe could even extend the life of AVC from a bit rate efficiency perspective. Tell us about what you developed and what you worked on.
Dror Gill: 07:13 Yeah, that’s right. I did have some part in this. We developed a quality measure and a whole application around it, and this is a solution that can reduce the bit rate of AVC by 30%, sometimes even 40%. It doesn’t get us exactly to where HEVC starts, 50% is pretty difficult and not for every content (type). But content distributors that recognize AVC will still be part of their codec mix for at least five years, I think what we’ve been able to do can really be helpful and a welcome relief to this bandwidth bottleneck issue.
Mark Donnigan: 07:52 It sounds like we’re in agreement that for at least the midterm horizon, the medium horizon, AVC is gonna stay with us.
Dror Gill: 08:01 Yeah, yeah. I definitely think so. For some applications and services and certain regions of the world where the device penetration of the latest, high end models is not as high as in other parts, AVC will be the primary codec for some time to come.
Dror Gill: 08:21 Okay, that’s AVC. Now, let’s talk about VP9.
Mark Donnigan: 08:24 Yes, let’s do that.
Dror Gill: 08:25 It’s interesting to me, essentially, it’s mostly a YouTube codec. It’s not a bad coded, it has some efficiency advantages over AVC, but outside of Google, you don’t see any large scale deployments. By the way, if you look at Wikipedia, you read about the section that says where is VP9 used, it says VP9 is used mostly by YouTube, some uses by Netflix, and it’s being used by Wikipedia.
Mark Donnigan: 08:50 VP9 is supported fairly well in devices. Though, it’s obviously hard to say exactly what the penetration is, I think there is support in hardware for decode for VP9. Certainly it’s ubiquitous on Android, and it’s in many of the UHD TV chip sets as well. So, it’s not always enabled, but again, from my background on the hardware side, I know that many of those SOCs, they do have a VP9 decoder built into them.
Mark Donnigan: 09:23 I guess the question in my mind is, it’s talked about. Certainly Google is a notable both developer and user, but why hasn’t it been adopted?
Dror Gill: 09:33 Well, I think there are several issues here. One of them is compression efficiency. VP9 brings maybe 20, 30% improvement in compression efficiency over AVC, but it’s not 50%. So, you’re not doubling your compression efficiency. If you want to replace the codec, that’s really a big deal. That’s really a huge investment. You need to invest in coding infrastructure, new players. You need to do compatibility testing. You need to make sure that your packaging and your DRM work correctly and all of that.
Dror Gill: 10:04 You really want to get a huge benefit to offset this investment. I think people are really looking for that 50% improvement, to double the efficiency, which is what you get with HEVC but not quite with VP9. I think the second point is that VP9, even though it’s an open source coder, it’s developed and the standard is maintained by Google. And some industry players are kind of afraid of the dominance of Google. Google has taken over the advertising market online.
Mark Donnigan: 10:32 Yes, that’s a good point.
Dror Gill: 10:34 You know, and search and mobile operating systems, except Apple, it’s all Android. So, those industry players might be thinking, I don’t want to depend on Google for my video compression format. I think this is especially true for traditional broadcasters. Cable companies, satellite companies, TV channels that broadcast over the air. These companies traditionally like to go with established, international standards. Compression technologies that are standardized, they have the seal of approval by ITU and ISO.
Dror Gill: 11:05 They’re typically following that traditional codec developer past. ISO MPEG too, now it’s AVC, starting with HEVC. What’s coming next?
Mark Donnigan: 11:16 Well, our next three letter codec is VVC. Tell us about VVC, Dror.
Dror Gill: 11:21 Yeah, yeah, VVC. I think this is another great example of collaboration between ITU and ISO. Again, they formed a joint video experts team. This time it’s called JVET.
Dror Gill: 12:10 So, JVET has launched a project to develop a new video coding standard. And you know, we had AVC that was advanced video coding. Then we had HEVC which is high efficiency video coding. So, they thought, what would be the next generation? It’s already advanced, it’s high efficiency. So, the next one, they called it VVC, which is versatile video code. The objective of VVC is obviously to provide a significant improvement in compression efficiency over the existing HEVC standard. Development already started. The JVET group is meeting every few in months in some exotic place in the world and this process will continue. They plan to complete it before the end of 2020. So, essentially in the next two years they are gonna complete the standard.
Dror Gill: 13:01 Today, already, even though VVC is in early development and they haven’t implemented all the tools, they already report a 30% better compression efficiency than HEVC. So, we have high hopes that we’ll be able to fight the video tsunami that is coming upon us with a much improved standard video coder which is VVC. I mean, its improved at least on the technical side and I understand that they also want to improve the process, right?
Mark Donnigan: 13:29 That’s right, that’s right. Well, technical capabilities are certainly important and we’re tracking of course VVC. 30% better efficiency this early in the game is promising. I wonder if the JVET will bring any learnings from the famous HEVC royalty debacles to VVC because I think what’s in everybody’s mind is, okay, great, this can be much more efficient, technically better. But if we have to go round and round on royalties again, it’s just gonna kill it. So, what do you think?
Dror Gill: 14:02 Yeah, that’s right. I think it’s absolutely true and many people in the industry have realized this, that you can’t just develop a video standard and then handle the patent and royalty issues later. Luckily some companies have come together and they formed an industry group called The Media Coding Industry Forum, or MC-IF. They held their first meeting a few weeks ago in Macau during empic meeting one through four. Their purpose statement, let me quote this from their website, and I’ll give you my interpretation of it. They say the media coding industry forum (MC-IF) is an open industry forum with a purpose of furthering the adoption of standards initially focusing on VVC, but establishing them as well accepted and widely used standards for the benefit of consumers and the industry.
Dror Gill: 14:47 My interpretation is that the group was formed in an effort for companies with interest in this next generation video codec to come together and attempt to influence the licensing policy of VVC and try to agree on a reasonable patent licensing policy in advance to prevent history from repeating itself. We don’t want that whole Hollywood story with the tragedy that took a few years until they reached the happy ending. So, what are you even talking about? This is very interesting. They’re talking about having a modular structure for the codec. These tools of the codecs, the features, can be plugged in and out, very easily.
Dror Gill: 15:23 So, if some company insists on reasonable licensing terms, this group can just decide not to support the feature and it will be very easily removed from the standard, or at least from the way that companies implement that standard.
Mark Donnigan: 15:37 That’s an interesting approach. I wonder how technically feasible it is. I think we’ll get into that in some other episodes.
Dror Gill: 15:46 Yeah. That may have some effect on performance.
Mark Donnigan: 15:49 Exactly. And again, are we back in the situation that the Alliance for Open Media is in with AV1. Where part of the issue of the slow performance is trying to work around patents. At the end of the day you end up with a solution that is hobbled technically.
Dror Gill: 16:10 Yeah. I hope it doesn’t go there.
Mark Donnigan: 16:13 Yeah, I hope we’re not there. I think you heard this too, hasn’t Apple joined the consortium recently?
Dror Gill: 16:21 Yeah, yeah, they did. They joined silently as they always do. Silently means that one day somebody discovers their logo… They don’t make any announcement or anything. You just see a logo on the website, and then oh, okay.
Mark Donnigan: 16:34 Apple is in the building.
Mark Donnigan: 16:41 You know, maybe it’s good to kind of bring this discussion back to Earth and close out our three part series by giving the listeners some pointers. About how they should be thinking about the next codec that they adopt. I’ve been giving some thought as we’ve been doing these episodes. I think I’ll kick it off here Dror if you don’t mind, I’ll share some of my thoughts. You can jump in.
Mark Donnigan: 17:11 These are complex decisions of course. I completely agree, billing this as codec wars and codec battles, it’s not helpful at the end of the day. Maybe it makes for a catchy headline, but it’s not helpful. There’s real business decisions (to be made). There are technical decisions. I think a good place to start for somebody who’s listening and saying “okay great, I now have a better understanding of the lay of the land of HEVC, for AV1, I can understand VP9, I can understand AVC and what some of my options are to even further reduce bit rate. But now, what do I do?”
Mark Donnigan: 17:54 And I think a good place to start is to just look at your customers, and do they lean towards early adopters. Are you in a strong economic environment, which is to say quite frankly, do most of your customers carry around the latest devices? Like an iPhone X, or Galaxy 9. If largely your customers lean towards early adopter and they’re carrying around the latest devices, then you have an obligation to serve them with the highest quality and the best performance possible.
Dror Gill: 18:26 Right. If your customers can receive HEVC, and it’s half the bit rate, then why not deliver it to them better quality, or say when you see the end cost with this more efficient codec and everybody is happy.
Mark Donnigan: 18:37 Absolutely, and again, I think just using pure logic. If somebody could afford a more than $1000 device in their pocket, probably the TV hanging on the wall is a very new, UHD capable (one). They probably have a game console in the house. The point is that you can make a pretty strong argument and an assumption that you can go, what I like to think of as all in HEVC including even standard definition, just SDR content.
Mark Donnigan: 19:11 So, the industry has really lost sight in my mind of the benefits of HEVC as they apply across the board to all resolutions. All of the major consumer streaming services are delivering 4K using HEVC, but I’m still shocked at how many, it’s kind of like oh, we forget that the same advantages of bit rate efficiency that work at 4K apply at 480p. Obviously, the absolute numbers are smaller because the file sizes are smaller, etc.
Mark Donnigan: 19:41 But the point is, 30, 40, 50% savings applies at 4K as it does at 480p. I understand there’s different applications in use cases, right? But would you agree with that?
Dror Gill: 19:55 Yeah, yeah, I surely agree with that. I mean, for 4K, HEVC is really an enabler.
Mark Donnigan: 20:00 That’s right.
Dror Gill: 20:01 For HEVC, you would need like 30, 40 megabits of video. Nobody can stream that to the home, but change it to 10, 15, that’s reasonable, and you must use HEVC for 4k otherwise it won’t even fit the pipe. But for all other resolutions, you get the bang with the advantage or you can trade it off for a quality advantage and deliver higher quality to your users, or higher frame rate, or enable HDR. If all of these possibilities that you can do with HD and even SD content, give them a better experience using HEVC and being able to stream on devices that your users already have. So yeah, I agree. I think it’s an excellent analysis. Obviously if you’re up in an emerging market, or your consumers don’t have high end devices, then AVC is a good solution. If there are network constraints, and there are many places in the world that network conductivity isn’t that great. Or in rural areas where we have very large parts of the population spread out (in these cases) bandwidth is low and you will get into a bottleneck even with HD.
Mark Donnigan: 21:05 That’s right.
Dror Gill: 21:06 That’s where perceptual optimization can help you reduce the bit rate even for AVC and keep within the constraints that you have. When your consumers can upgrade their devices and when the cycle comes in a few years when every device has HEVC support, then obviously you upgrade your capability and support HEVC across the board.
Mark Donnigan: 21:30 Yeah, that’s a very important point Dror, is that this HEVC adoption curve in terms of silicon, on devices. It is in full motion. Just the planning life cycles. If you look at what goes into hardware, and especially on the silicon side, it doesn’t happen that way. Once these technologies are in the designs, once they are in the dies, once the codec is in silicon, it doesn’t get arbitrarily turned on and off like light switches.
Mark Donnigan: 22:04 How should somebody be looking at VP9, VVC, and AV1?
Dror Gill: 22:13 Well, VP9 is an easy one. Unless you’re Google, you’re very likely gonna skip over this codec. Not just that the VP9 isn’t the viable choice, it simply doesn’t go so far as HEVC in terms of bit rate efficiency and quality. Maybe two years back we would consider it as an option for reducing bit rate, but now with the HEVC support that you have, there’s no point in going to VP9. You might as well go to HEVC. If you talk about VVC, (the) standard is still a few years from being ratified so, we actually don’t have anything to talk about.
Dror Gill: 22:49 The important point is again to remember, even when VVC launches, it will still be another 2 to 3 years after ratifying the standard before you have even a very basic playback ecosystem in place. So, I would tell our listeners if you’re thinking, why should I adopt HEVC, because VVC is just around the corner, well, that corner is very far. It’s more like the corner of the Earth than the corner of the next block.
Mark Donnigan: 23:15 That’s right.
Dror Gill: 23:18 So, HEVC today, VVC will be the next step in a few years. And then there’s AV1. You know, we talked a lot about AV1. No doubt, AV1 has support from huge companies. I mean Google, Facebook, Intel, Netflix, Microsoft. And those engineers, they know what they’re doing. But now, it’s quite clear that compression efficiency is the same as HEVC. Meanwhile, after removing other royalty cost for content delivery, HEVC Advance removed it. The license situation is much more clear now. You add to this the fact that at the end of the day, two to three years, you’re gonna need five to ten times more compute power to encode AV1, reaching effectively the same result. Now Google, again. Google may be that they have unlimited compute resources, they will use it. They developed it.
Dror Gill: 24:13 The smaller content providers, all the other ones, the non Googles of the world and other broadcasters with growing support for HEVC that we expect in a few years. I think it’s obvious. They’re gonna support HEVC and then a few years later when VVC is ratified, when it’s supported in devices, they’re gonna move to VVC. Because this codec does have the required compression efficiency improvement over HEVC.
Mark Donnigan: 24:39 Yeah, that’s an excellent summary Dror. Thank you for breaking this all down for our listeners so succinctly. I’m sure this is really gonna provide massive value. I want to thank our amazing audience because without you, the Video Insiders Podcast would just be Dror and me taking up bits on a server somewhere.
Dror Gill: 24:59 Yeah, talking to ourselves.
Mark Donnigan: 25:01 As you can tell, video is really exciting to us and so we’re so happy that you’ve joined us to listen. And again, this has been a production of Beamr Imaging Limited. Please, subscribe on iTunes and if you would like to try out beamer codecs in your lab or your production environment, we are giving away up to $100 of HEVC and H264 in coding every month. That’s each and every month. Just go to https://beamer.com/free and get started immediately.