With numerous advantages, AV1 is now supported on about 60% of devices and all major web browsers. To accelerate its adoption – Beamr has introduced an easy, automated upgrade to the codec that is in the forefront of today’s video technology
Four years ago we explored the different video codecs, analyzing their strengths and weaknesses, and took a look at current and predicted market share. While it is gratifying to see that many of our predictions were pretty accurate, that is accompanied by some degree of disappointment: while AV1 strengths are well known in the industry, significant change in adoption of new codecs has yet to materialize.
The bottom line of the 2020 post was: “Only time will tell which will have the highest market share in 5 years’ time, but one easy assessment is that with AVC current market share estimated at around 70%, this one is not going to disappear anytime soon. AV1 is definitely gaining momentum, and with the giants backing we expect to see it used a fair bit in online streaming. “
Indeed we are living in a multi-codec reality, where AVC still accounts for, by far, the largest percentage of video content, but adoption of AV1 is starting to increase with large players such as Netflix and YouTube incorporating it into their workflows, and many others using it for specific high value use cases.
Thus, we are faced with a mixture of the still dominant AVC, HEVC (serving primarily UHD and HDR use cases), AV1 and some additional codecs such as VP9, VVC which are being used in quite small amounts.
The Untapped Potential of AV1
So while AV1 adoption is increasing, there is still significant untapped potential. One of the causes for slower than hoped rollout of AV1 is the obstacle present for adoption of any new standard – critical mass of decoding support in H/W on edge devices.
While for AVC and HEVC the coverage is very extensive, for AV1 that has only recently become the case, with support across an estimate of 60% of devices and all major web browsers, and complementing the efficient software decoding offered by Dav1d.
Another obstacle AV1 faces involves the practicalities of deployment. While there is extensive knowledge, within the industry and available online, of how best to configure AVC encoding, and what presets and encoding parameters work well for which use cases – there is no such equivalent knowledge for AV1. Thus, in order to deploy it, extensive research is needed by those who intend to use it.
Additionally, AV1 encoding is complicated, resulting in much higher processing power required to perform software encoding. In a world that is constantly trying to cut back costs, and use lower power solutions, this can pose a problem. Even when using software solutions at the fastest settings, the compute required is still significantly slower than AVC encoding at typical speeds. This is a strong motivator to upgrade to AV1 using H/W accelerated solutions (Learn more about Beamr solution to the challenge).
The upcoming codec possibilities are also a deterrent for some. With AV2 in the works, VVC finalized and gaining some traction, and various groups working on AI based encoding solutions, there will always be players waiting for ‘the next big thing’, rather than having to switch out codecs twice.
In a world where JPEG, a 30+ year old standard, is still used in over 70% of websites and is the most popular format on the web for photographic content, it is no surprise that adoption of new video codecs is taking time.
While a multi codec reality is probably going to stay with us, we can at least hope that when we revisit this topic in a blog a few years down the line, the balance between deployed codecs leans more towards the higher efficiency codecs, like AV1, to yield the best bitrate – quality options for the video world.
This year at IBC 2024 in Amsterdam, we are excited to demonstrate Live 4K p60 optimized streaming with our Content-Adaptive Bitrate (CABR) technology on NVIDIA Holoscan for Media, a software-defined, AI-enabled platform that allows live video pipelines to run on the same infrastructure as AI. Using the CABR GStreamer plugin, premiered at the NAB Show earlier this year, we now support live, quality-driven optimized streaming for 4Kp60 video content.
It is no secret that savvy viewers are coming to expect the high-quality experience of 4K Ultra-High-Definition streamed at 60 frames per second for premium events. What started with a drizzle a few years back has become the high end norm for recent events such as the 2024 Olympics, where techies were sharing insights on where it could be accessed.
Given the fact that 4K means a whopping four times the pixels compared to full HD resolution, keeping up with live encoding of 4K at 60 fps can be quite challenging, and can also result in bitrates that are too high to manage.
One possible solution for broadcasters is to encode and transmit at 1080p and rely on the constantly improving upscalers available on TVs to provide the 4K experience, but this of course means they cannot control the user experience. A better solution is to have a platform that is super fast, and can create live 4Kp60 encodes, which combine excellent quality with an optimization process that minimizes the required bitrate for transmission.
Comparison of 4K Live video before and after optimization
Beamr CABR on Holoscan for Media offers exactly that, by combining the fast data buses and easy-to-use architecture of Holoscan for Media with Beamr hardware-accelerated, quality-driven optimized AV1 encoding. Together, it is possible to stream super efficient, 4K, lower bitrate encodes at top notch quality.
Content Adaptive Bitrate encoding, or CABR, is Beamr’s patented and award-winning technology that uses a quality measure to select the best candidate with the lowest bitrate and the same perceptual quality as a reference frame. In other words, users can enjoy 30-50% lower bitrate, faster delivery of files or live video streams and improved user experience – all with exactly the same quality as the original video.
In order to achieve aggressive bitrates which are feasible for broadcast of live events, we configure the system to use AV1 encoding. The advanced AV1 format has been around since 2018. However, its full potential has not been fully realized by many players in the video arena. AV1 is raising the bar significantly in comparison to previous modern codecs, such as AVC (H.264) or HEVC (H.265), in terms of efficiency, performance with GPUs and high quality for real-time video. When combined with CABR – AV1 is offering up even more. According to our tests, AV1 can reduce data by 50% compared to AVC and by 30% compared to HEVC. We also showed that CABR optimized AV1 is beneficial for machine learning tasks.
Putting all three of these technologies together, namely deploying Holoscan for Media with the Beamr CABR solution inside, which in turn is using NVIDIA’s hardware-accelerated AV1 encoder, provides a platform that offers spectacular benefits. With the rise in demand for high-quality live streaming at high resolution, high fps and manageable bitrates, while keeping an eye on the encoding costs – this solution is definitely an interesting prospect for companies looking to boost their streaming workflows.
Beamr’s Content Adaptive Bit Rate solution enables significantly decreasing video file size or bitrates without changing the video resolution or compromising perceptual quality. Since the optimized file is fully standard compliant, it can be used in your workflow seamlessly, whatever your use case, be it video streaming, playback or even part of an AI workflow.
Beamr first launched Beamr cloud earlier this year, and we are now super excited to announce that our valued partnership with Oracle Cloud Infrastructure (OCI) is enabling us to offer to OCI customers more features and better performance.
The performance improvements are due in part to the availability of the powerful NVIDIA L40S GPUs on OCI. In preliminary testing we found that running our video encoding workflows can be up to 30% faster when using these cards, than when running on the cards we currently use in the Beamr Cloud solution.
This was derived from testing AVC and HEVC NVENC driven encodes for a set of nine 1080p classic test clips with eight different configurations, and comparing encoding wall times on an A10G vs. a L40S GPU. Speedup factors of up to 55% were observed, with an average just above 30%. The full test data is available here.
Another exciting feature about these cards is that they support AV1 encoding, which means Beamr Cloud will now offer to turn your videos into optimized AV1 encodes, offering even higher bitrate/file size savings.
What’s the fuss about AV1?
In order to store and transmit video, substantial compression is needed. From the very earliest efforts to standardize video compression in the 90s, there has been a constant effort to create video compression standards offering increasing efficiency – meaning that the same video quality can be achieved with smaller files or lower bitrates.
As shown in the schematic illustration below, AV1 has come a long way in improving over H.264/AVC, the most widely adopted standard today, despite being 20 years old. However, the increased compression efficiency is not free – the computational complexity of newer codecs is also significantly higher, motivating the adoption of hardware accelerated encoding options.
With the demand and need for Video AI workflows continuing to rise, the ability to perform fully automatic, fast, efficient, optimized video encoding is an important enabler.
The Beamr GPU powered video compression and optimization occur within the GPU on OCI, right at the heart of these AI workflows, making them extremely well placed to offer benefits to such workflows. We have previously shown in a number of case studies that there is no negative impact on inference or training results when using the optimized files – making the integration of this optimization process into AI workflows a natural choice for cost savvy developers.
This year at the NAB Show 2024 in Las Vegas, we are excited to demonstrate our Content-Adaptive Bitrate (CABR) technology on the NVIDIA Holoscan for Media platform. By implementing CABR as a GStreamer plugin, we have, for the first time, made bitrate optimization of live video streams easily achievable in the cloud or premise.
Building on the NVIDIA DeepStream software development kit, which can extends GStreamer’s capabilities, significantly reduced the amount of code required to develop the Holoscan for Media based application. Using DeepStream components for real-time video processing and NMOS (Networked Media Open Specifications) signaling, we were able to keep our focus on the CABR technology and video processing.
The NVIDIA DeepStream SDK provides an excellent framework for developers to build and customize dynamic video processing pipelines. DeepStream provides pipeline components that make it very simple to build and deploy live video processing pipelines that utilize the hardware decoders and encoders available on all NVIDIA GPUs.
Beamr CABR dynamically adjusts video bitrate in real-time, optimizing quality and bandwidth use. It reduces data transmission without compromising video quality, making the video streaming more efficient. Recently we released our GPU implementation which uses the NVIDIA NVENC, encoder, providing significantly higher performance compared to previous solutions.
Taking our GPU implementation for CABR to the next level, we have built a GStreamer Plugin. With our GStreamer Plugin, users can now easily and seamlessly incorporate the CABR solution into their existing DeepStream pipelines as a simple drop-in replacement to their current encoder component.
Holoscan For Media
A GStreamer Pipeline Example
To illustrate the simplicity of using CABR, consider a simple DeepStream transcoding pipeline that reads and writes from files.
By simply replacing the nvv4l2av1enc component with our CABR component, the encoding bitrate is adapted in real-time, according to the content, ensuring optimal bitrate usage for each frame, without any loss of perceptual quality.
Similarly, we can replace the encoder component used in a live streaming pipeline with the CABR component to optimize live video streams, dynamically adjusting the output bitrate and offering up to a 50% reduction in data usage without sacrificing video quality.
The Broad Horizons of CABR Integration in Live Media
Beamr CABR, demonstrated using NVIDIA Holoscan for Media at NAB show, marks just the beginning. This technology is an ideal fit for applications running on NVIDIA RTX GPU-powered accelerated computing and sets a new standard for video encoding.
Lowering the video bitrate reduces the required bandwidth when ingesting video to the cloud, creating new possibilities where high resolution or quality were previously costly or not even possible. Similarly, reduced bitrate when encoding on the cloud allows for streaming of higher quality videos at lower cost.
From file-based encoding to streaming services — the potential use cases are diverse, and the integration has never before been so simple. Together, let’s step into the future of media streaming, where quality and efficiency coexist without compromise.
The proliferation of AI-generated visual content is creating a new market for media optimization services, with companies like Beamr well positioned to help businesses optimize their video content for reduced storage, faster delivery, and better user experiences.
We are living in a brave new world, where any image and video content we can imagine is at our fingertips, merely a prompt and AI based content generation engine away. Platforms like Wochit, Synthesia, Wibbitz, and D-ID are using AI technology to automate the video creation process. Using these tools makes it almost trivial for businesses to create engaging video content at scale. These platforms allow users to create tailored content quickly and efficiently, with minimal time and cost.
Wochit, for example, offers a library of pre-made templates that users can customize with their own branding and messaging. The platform’s AI technology can also automatically generate videos from text, images, and video clips, making it easier for businesses to create engaging video content without needing specialized video production skills.
However, as businesses increasingly rely on AI-generated content to reach their audiences, and can create a multitude of ‘perfect fit’ videos, the struggle with storage and bit rates becomes a significant factor in their operations. When dealing with bandwidth gobbling video, companies need to ensure that their videos are optimized for fast delivery, high quality, and optimal user experiences. That’s where Beamr comes in.
Beamr’s technology uses advanced compression algorithms to automatically optimize image and video content for fast delivery over any network or device, without compromising quality. This means that you will get to keep the full look and feel of the content, and maintain standard compatibility, but reduce the file sizes or bitrates – without having to do anything manually. The underlying, patented and Emmy Award winning technology will guarantee that the perceptual quality is preserved while any unnecessary bits and bytes are removed. This allows businesses to deliver high-quality content that engages their audience and drives results, while also minimizing the impact on network resources and reducing storage and delivery costs.
To demonstrate the synergy between AI based video content generation and Beamr’s optimization technology we went to Wochit and created a magnificent video showcasing Ferrari above. We then applied the Beamr optimization technology, and received the reduced size perceptually identical optimized video, with file size down from the original 8.8MB to 5.4MB, offering saving of almost 38%.
For our next experiment we took the title of this blog, went to D-ID, and turned the text into a promotional video, using all the default settings. This resulted in the source video shared below.
With an easy drag & drop into the Beamr optimization utility, a completely equivalent video file – using the same codec, resolution and perceptual quality was obtained, except its size was reduced by 48%.
Image synthesis using AI is also becoming more and more common. Along with the already commonplace AI based image generators such as DALL-E (2), many additional platforms are becoming available including Midjourney, DreamStudio and Images.ai.
Feeling the tug of the Land-Down-Under we headed to https://images.ai/prompt/ and requested an image showing ‘a koala eating ice-cream’. The adorable result is shown below on the left. Then we put it through Beamr optimization software and obtained an image with the exact same quality, but reduced from the original 212 KB JPEG, to a mere 49 KB perceptually identical fully standard compliant JPEG image.
Original versionOptimized version
Beamr is also preparing to launch a new SaaS platform that leverages Nvidia’s accelerated video encoding technology, to further speed up the video optimization process. This will allow businesses to optimize their video content even faster than traditional video encoding services, giving them a competitive edge in the rapidly evolving market for AI-generated video content.
For businesses that use Wochit to create their videos, Beamr’s technology can be integrated into the delivery process, ensuring that the videos are optimized for fast delivery and high quality. This allows businesses to stay ahead of the curve in this rapidly evolving market, and keeps their audiences coming back for more. As the demand for AI-generated video content continues to grow, media optimization services like Beamr will become increasingly important for businesses that want to deliver high-quality image and video content that engages their audience and drives results ensuring that they stay ahead of the curve in this rapidly evolving market.
By now, most of us have seen the data and know that online video consumption is soaring at a rate that is historically unrivaled. It’s no surprise that in the crux of the streaming era, so many companies are looking to innovate and figure out how to make their workflows or customers workflows better, less expensive, and faster.
In Episode 4 of The Video Insiders, we caught up with streaming veteran Tim Siglin to discuss HEVC implementation trends that counter previous assumptions, notable 2018 streaming events, and what’s coming in 2019. Tune in to hear The Video Insiders cover top-of-mind topics:
Want to join the conversation? Reach out to TheVideoInsiders@beamr.com
TRANSCRIPTION (lightly edited to improve readability only)
Mark Donnigan: 00:00 On today’s episode, the Video Insiders sit down with an industry luminary who shares results of a codec implementation study, while discussing notable streaming events that took place in 2018 and what’s on the horizon for 2019. Stay tuned. You don’t want to miss receiving the inside scoop on all this and more.
Announcer: 00:22 The Video Insiders is the show that makes sense of all that is happening in the world of online video, as seen through the eyes of a second generation Kodak nerd and a marketing guy who knows what I frames and macroblocks are. Here are your hosts, Mark Donnigan and Dror Gill.
Mark Donnigan: 00:40 Welcome, everyone. I am Mark Donnigan, and I want to say how honored Dror and I are to have you with us. Before I introduce this very special guest and episode, I want to give a shout of thanks for all of the support that we’re receiving. It’s really been amazing.
Mark Donnigan: 00:59 In the first 48 hours, we received 180 downloads. It’s pretty amazing.
Dror Gill: 01:06 Yeah. Yeah, it is. The industry is not that large, and I think it’s really an amazing number that they’re already listening to the show from the start before the word of mouth starts coming out, and people spread the news and things like that. We really appreciate it. So, if it’s you that is listening, thank you very much.
Mark Donnigan: 01:29 We really do aim for this to be an agenda-free zone. I guess we can put it that way. Obviously, this show is sponsored by Beamr, and we have a certain point of view on things, but the point is, we observed there wasn’t a good place to find out what’s going on in the industry and sort of get unbiased, or maybe it’s better to say unfiltered, information. That’s what we aim to do in every episode.
Mark Donnigan: 01:57 In this one, we’re going to do just that. We have someone who you can definitely trust to know what’s really happening in the streaming video space, and I know he has some juicy insights to share with us. So, without further ado, let’s bring on Tim Siglin.
Tim Siglin: 02:15 Hey, guys. Thank you for having me today and I will definitely try to be either as unfiltered or unbiased as possible.
Mark Donnigan: 02:21 Why don’t you give us a highlight reel, so to speak, of what you’ve done in the industry and, even more specifically, what are you working on today?
Tim Siglin: 02:31 Sure. I have been in streaming now for a little over 20 years. In fact, when Eric Schumacher-Rasmussen came on as the editor at StreamingMedia.com, he said, “You seemed to be one of the few people who were there in the early days.” It’s true. I actually had the honor of writing the 10-year anniversary of Streaming Media articles for the magazine, and then did the 20-year last year.
Tim Siglin: 02:57 My background was Motion Picture production and then I got into video conferencing. As part of video conferencing, we were trying to figure out how to include hundreds of people in a video conference, but not need necessarily have them have two-way feedback. That’s where streaming sort of caught my eye, because, ultimately, for video conferencing we maybe needed 10 subject matter experts who would talk back and forth, and together a hundred, then it went to thousands, and now hundreds of thousands. You can listen in and use something like chat or polling to provide feedback.
Tim Siglin: 03:31 For me, the industry went from the early revolutionary days of “Hey, let’s change everything. Let’s get rid of TV. Let’s do broadcast across IP.” That was the mantra in the early days. Now, of course, where we are is sort of, I would say, two-thirds of the way there, and we can talk a little bit about that later. The reality is that the old mediums are actually morphing to allow themselves to do heap, which is good, to compete with over the top.
Tim Siglin: 04:01 Ultimately, what I think we’ll find, especially when we get to pure IP broadcast with ATSC 3.0 and some other things for over-the-air, is that we will have more mediums to consume on rather than fewer. I remember the early format ways and of course we’re going to talk some in this episode about some of the newer codec like HEVC. Ultimately, it seems like the industry goes through the cycles of player wars, format wars, browser wars, operating system wars, and we hit brief periods of stability which we’ve done with AVC or H.264 over the last probably eight years.
Tim Siglin: 04:46 Then somebody wants to stir the pot, figure out how to either do it better, less expensively, faster. We go back into a cycle of trying to decide what the next big thing will be. In terms of what I’m working on now, because I’ve been in the industry for almost 21 years. Last year, I helped start a not-for-profit called Help Me Stream, which focuses on working with NGOs in emerging economies, trying to help them actually get into the streaming game to get their critical messages out.
Tim Siglin: 05:18 That might be emerging economies like African economies, South America, and just the idea that we in the first world have streaming down cold, but there are a lot of messages that need to get out in emerging economies and emerging markets that they don’t necessarily have the expertise to do. My work is to tie experts here with need there and figure out which technologies and services would be the most appropriate and most cost effective.
Tim Siglin: 05:48 The other thing I’m working on here, just briefly, is we’re getting ready for the Streaming Media Sourcebook, the 2019 sourcebook. I’m having to step back for the next 15 days, take a really wide look at the industry and figure out what the state of affairs are.
Dror Gill: 06:06 That’s wonderful. I think because this is exactly the right point, is one you end and the other one begins, kind of to summarize where we’ve been in 2018, what is the state of the industry and the fact that you’re doing that for the sourcebook, I think, ties in very nicely with our desire to hear from you an overview of what were the major milestones or advancements that were made in the streaming industry in 2018, and then looking into next year.
Dror Gill: 06:39 Obviously, the move to IP, getting stronger and stronger, now the third phase after analog and digital, now we have broadcast over IP. It’s interesting what you said about broadcasters not giving up the first with the pure OTT content providers. They have a huge business. They need to keep their subscribers and lower their churn and keep people from cutting the cord, so to speak.
Dror Gill: 07:04 The telcos and the cable companies still need to provide the infrastructure for Internet on top of which the over-the-top providers and their content, but they still need to have more offering and television and VLD content in order to keep their subscribers. It’s very interesting to hear how they’re doing it and how they are upgrading themselves to the era of IP.
Tim Siglin: 07:30 I think, Dror, you hit a really major point, which is we, the heavy lift … I just finished an article in ATSC 3.0 where I talk about using 2019 to prepare for 2020 when that will go live in the U.S. The heavy lift was the analog to digital conversion. The slightly easier lift is the conversion from digital to IP, but it still requires significant infrastructure upgrade and even transmission equipment to be able to do it correctly for the over-the-year broadcasters and cable.
Dror Gill: 08:07 That’s right. I think on the other hand, there is one big advantage to broadcast, even broadcast over-the-air. That is the ability to actually broadcast, the ability to reach millions, tens of millions, hundreds of millions of people over a single channel that everybody is receiving. Whereas, because of historic reasons and legacy reasons in IP, we are limited, still, when you broadcast to the end user to doing everything over unicast. When you do this, it creates a tremendous load on your network. You need to manage your CDNs.
Dror Gill: 08:46 I think we’ve witnessed in 2018 on one hand very large events being streamed to our record audience. But, on the other hand, some of them really failed in terms of user experience. It wasn’t what they expected because of the high volume of users, and because more and more people have discovered the ability to stream things over IP to their televisions and mobile devices. Can you share with us some of the experience that you have, some of the things that you’re hearing about in terms of these big events where they had failures and what were the reasons for those failures?
Tim Siglin: 09:30 I want to reiterate the point you made on the OTA broadcast. It’s almost as if you have read the advanced copy of my article, which I know you haven’t because it’s only gone to the editor.
Dror Gill: 09:42 I don’t have any inside information. I have to say, even though we are the Video Insiders.
Mark Donnigan: 09:47 We are the Video Insiders. That’s right.
Mark Donnigan: 09:49 But no inside information here.
Dror Gill: 09:51 No inside information. I did not steal that copy.
Tim Siglin: 09:55 What I point out in that article, Dror, I think which will come out in January shortly after CES is basically this. We have done a good job in the streaming industry, the OTT space of pushing the traditional mediums to upgrade themselves. One of the things as you say with OTA, that ability to do essentially a multicast from a tower wirelessly is a really, really good thing, because to get us to scale, and I think about things like the World Cup, the Olympics and even the presidential funeral that’s happened here in December, there are large-scale events that we in the OTT space just can’t handle, if you’re talking about having to build the capacity.
Tim Siglin: 10:39 The irony is, one good ATSC transmission tower could hit as many people as we could handle essentially globally with the unicast (OTT) model. If you look at things like that and then you look at things like EMBMS in the mobile world, where there is that attempt to do essentially a multicast, and it goes to points like the World Cup. I think one of the horror stories in the World Cup was in Australia. There was a mobile provider named Optus who won the rights to actually do half of the World Cup preliminary games. In the first several days, they were so overwhelmed by the number of users who wanted to watch and were watching, as you say, in a unicast model that they ended up having to go back to the company they had bid against who had the other half of the preliminaries and ask them to carry those on traditional television.
Tim Siglin: 11:41 The CEO admitted that it was such a spectacular failure that it damaged the brand of the mobile provider. Instead of the name Optus being used, everybody was referring to it as “Floptus.” You don’t want your brand being known as the butt of jokes for an event that only happens once every four years that you have a number of devotees in your market. And heaven forbid, it had been the World Cup for cricket, there would have been riots in the street in Sydney and Melbourne. Thank goodness it was Australia with soccer as opposed to Australia with cricket.
Tim Siglin: 12:18 It brings home the point that we talk about scale, but it’s really hard to get to scale in a unicast environment. The other event, this one happened, I believe, in late 2017, was the Mayweather fight that was a large pay-per-view event that was streamed. It turned out the problem there wasn’t as much the streams as it was the authentication servers were overwhelmed in the first five minutes of the fight. So, with authentication gone, it took down the ability to actually watch the stream.
Tim Siglin: 12:53 For us, it’s not just about the video portion of it, it’s actually about the total ecosystem and who you’re delivering to, whether you’re going to force caps into place because you know you can’t go beyond a certain capacity, or whether you’re going to have to partner up with traditional media like cable service providers or over-the-air broadcasters.
Mark Donnigan: 13:14 It’s a really good point, Tim. In the World Cup, the coverage that I saw, it was more of, I’d almost say or use the phrase, dashed expectations. Consumers, they were able to watch it. In most cases, I think it played smoothly. In other words, the video was there, but HDR signaling didn’t work or didn’t work right. Then it looked odd on some televisions or …
Tim Siglin: 13:43 20 frames a second instead of 60 frames a second.
Mark Donnigan: 13:48 Exactly. What’s interesting to me is that, what I see is, the consumer, they’re not of course walking around thinking as we are, like frame rate and color space and resolution. They are getting increasingly sensitive to where they can look at video now and say, “That’s good video,” or “That doesn’t look right to me.” I know we were talking before we started recording about this latest Tom Cruise public service announcement, which is just super fascinating, because it …
Tim Siglin: 14:24 To hear him say motion interpolation.
Mark Donnigan: 14:26 Yeah. Maybe we should tell the audience, for those, since it literally just came out I think today, even. But you want to tell the audience what Tom Cruise is saying?
Tim Siglin: 14:38 Essentially, Tom Cruise was on the set of Top Gun, as they’re shooting Top Gun. Another gentleman did a brief PSA for about a minute asking people to turn off motion interpolation on their televisions, which motion interpolation essentially takes a 24-frame per second and converts it to 30 frames per second by adding phantom frames in the middle. Because Mission Impossible: Fallout is just being released for streaming, Cruise was concerned and obviously others were concerned that some of the scenes would not look nearly as good with motion interpolation turned on.
Tim Siglin: 15:17 I think, Mark, we ought to go to a PSA model, asking for very particular things like, “How do you turn HDR on? How do you …” Those types of things, because those get attention in a way that you and I or a video engineer can’t get that attention.
Dror Gill: 15:33 How do you know if what you’re getting is actually 4K or interpolate HD, for example?
Tim Siglin: 15:38 Especially in our part of the industry, because we will call something OTT 4K streaming. That may mean that it fits in a 4K frame, but it doesn’t necessarily mean it’s that number of pixels being delivered.
Dror Gill: 15:52 It can also mean that the top layer in your adaptive bit rate stream is 4K, but then if you don’t have enough bandwidth, you’re actually getting the HD layer or even lower.
Dror Gill: 16:02 Even though it is a 4K broadcast and it is 4K content. Sometimes, you can be disappointed by that fact as well.
Mark Donnigan: 16:11 I have to give a very, very funny story directly related, and this happened probably, I don’t know, maybe, at least 18 months ago, maybe two years ago. I’m sitting on an airplane next to this guy. It’s the usual five-minute, get acquainted before we both turn on our computers. Anyway, when someone asks, “What do you do?” I generally just say, “I work for a video software company,” because how do you explain digital encoding? Most people just sort of stop at that, and don’t really ask more.
Mark Donnigan: 16:44 But this guy is like, “Oh, really?” He said, “So, I just bought a 4K TV and I love it.” He was raving about his new Samsung TV. Of course, he figured I’m a video guy. I would appreciate that. I said, “Hey.” “So, you must subscribe to Netflix.” “Yes. Yes, of course,” he says. I said, “What do you think of the Netflix quality? It looks great, doesn’t it?”
Mark Donnigan: 17:10 He sort of hem and hawed. He’s like, “Well, it really … I mean, yeah. Yeah, it looks great, but it’s not quite … I’m just not sure.” Then, I said, “I’m going to ask you two questions. First of all, are you subscribed to the 4K plan?” He was. Then I said, “How fast is your Internet at home.” He’s like, “I just have the minimum. I don’t know. I think it’s the 20 megabit package,” or whatever it was. I don’t remember the numbers.
Mark Donnigan: 17:38 I said, “There’s this thing.” And I gave him like a 30-second primer on adaptive bit rate, and I said, “It is possible, I have no idea of your situation, that you might be watching the HD version.” Anyway, he’s like, “Hah, that’s interesting.” I connect with the guy on LinkedIn. Three days later, I get this message. He says, “I just upgraded my Internet. I now have 4K on my TV. It looks awesome.”
Mark Donnigan: 18:04 On one hand, the whole situation was not surprising and, yet, how many thousands, tens of thousands, maybe millions of people are in the exact same boat? They’ve got this beautiful TV. It could be because they’re running some low-end router in the house. It could be they truly have a low end bandwidth package. There could be a lot of reasons why they’re not getting the bandwidth. They’re so excited about their 4K TV. They’re paying Netflix to get the top layer, the best quality, and they’re not even seeing it. It’s such a pity.
Tim Siglin: 18:37 I had a TSA agent asked me that same question, Mark, when I came through customs. I’m like, “Sure. I’ll stand here and answer that question for you.” The router was actually what I suggested that he upgrade, because he said his router was like this (old unit).
Mark Donnigan: 18:53 In a lot of homes, it’s a router that’s 15 years old and it just isn’t (up to the task).
Tim Siglin: 18:58 But it brings out the point that even as we’re talking about newer codecs and better quality, even if we get a lower sweet spot in terms of 4K content (streaming bandwidth), or as we found in the survey that we worked on together, that using HEVC for 1080p or 720p, if the routers, if the software in the chain is not updated, the delivery quality will suffer in a way that people who have a tuned television and seen the consistent quality aren’t certain what to do to fix when they use an over-the-top service.
Tim Siglin: 19:34 I think this is a key for 2019. As we prepare for ATSC 3.0 on over-the-air broadcast where people will be able to see pristine 4K, it will actually force those of us in the OTT space to up our game to make sure that we’re figuring out how to deliver across these multiple steps in a process that we don’t break.
Dror Gill: 19:54 You really see ATSC 3.0 as a game-changer in 2019?
Tim Siglin: 19:59 What I see it as is the response from the broadcast industry to, A) say that they’re still relevant, which I think is a good political move. And, B) it provides the scale you were talking about, Dror. See, I think what it does is it at least puts us in the OTT space on notice that there will be in certain first world countries a really decent quality delivery free of charge with commercials over the air.
Tim Siglin: 20:31 It takes me back to the early days of video compression when, if you had a good class-one engineer and an analog NTSC transmission system, they could give you really good quality if your TV was tuned correctly. It only meant having to tune your TV. It didn’t mean having to tune your router or having to tune your cable modem, having to tune your settings on your TV. I think that’s where the game-changer may be, is that those tuner cards, which will send HDR signaling and things like that with the actual transmission, are going to make it much easier for the consumer to consume quality in a free scenario. I think that part of it is a potential game-changer.
Mark Donnigan: 21:19 That’s interesting. Tim, we worked together earlier this year on a survey, an industry survey that I think it would be really, really interesting to listeners to talk about. Shall we pivot into that? Maybe you can share some of the findings there.
Tim Siglin: 21:38 Why don’t you take the lead on why Beamr wanted to do that? Then I’ll follow up with some of the points that we got out of it.
Mark Donnigan: 21:46 Obviously, we are a codec developer. It’s important for us to always be addressing the market the way that the market wants to be addressed, meaning that we’re developing technologies and solutions and standards that’s going to be adopted. Clearly, there has been, especially if we rewind a year ago or even 18 months ago, AV1 was just recently launched. There were still questions about VP9.
Mark Donnigan: 22:19 Obviously, H264 AVC is the standard, used everywhere. We felt, “Let’s go out to the industry. Let’s really find out what the attitudes are, what the thinking is, what’s going on ‘behind closed doors’ and find out what are people doing.” Are they building workflows for these new advanced codecs? How are they going to build those workflows? That was the impetus, if you will, for it.
Mark Donnigan: 22:49 We are very happy, Tim, to work with you on that and of course Streaming Media assisted us with promoting it. That was the reason we did it. I know there were some findings that were pretty predictable, shall we say, no surprises, but there were some things that I think were maybe a little more surprising. So, maybe if you like to share some of those.
Tim Siglin: 23:12 Yeah. I’ll hit the highlights on that. Let me say too that one of the things that I really like about this particular survey, there was another survey that had gone on right around that time that essentially was, “Are you going to adopt HEVC?” What we took the approach on with this survey was to say, “Okay. Those of you who’ve already adopted HEVC, what are the lessons that we can learn from that?”
Tim Siglin: 23:36 We didn’t exclude those who were looking at AV1 or some of the other codes, even VP9, but we wanted to know those people who used HEVC. Were they using it in pilot projects? Were they thinking about it? Were they using it in actual production? What we found in the survey is that AVC, or H.264, was still clearly dominant in the industry, but that the ramp-up to HEVC was moving along much faster than at least I … I believed. Mark, I told you when we started the survey question creation, which was about a year ago and then launched it in early 2018, I expected we wouldn’t see a whole lot of people using HEVC in production.
Tim Siglin: 24:23 I was pleasantly surprised to say that I was wrong. In fact, I think you mentioned in our recent Streaming Media West interview that there was a statistic you gave about the number of households that could consume HEVC. Was it north of 50%?
Mark Donnigan: 24:40 Yeah, it’s more than 50%. What’s interesting about that number is that that actually came from a very large MSO. Of course, they have a very good understanding of what devices are on their network. They found that there was at least one device in at least 50% of their homes that could receive and decode, playback, HEVC. That’s about as real world as you can get.
Tim Siglin: 25:06 What was fascinating to me too in this study was, we asked open-ended questions, which is what I’ve done in the research projects for the last 25 years both the video conferencing and streaming. One of the questions we asked was, “Do you see HEVC as only a 4K solution or do you see it as an option for lower resolutions?” It turned out overwhelmingly, people said, “We not only see it for 4K. We see it for high-frame rate (HFR) 1080p, standard frame rate 1080p, with some HDR.”
Tim Siglin: 25:40 Not a majority, but a large number of respondents said they would even see it as a benefit at 720p. What that tells me is, because we had a large number of engineers, video engineers, and we also have people in business development who answer these questions, what it tells me is that companies know as we scale because of the unicast problem that Dror pointed out in the beginning that scaling with a codec that consumes more bandwidth is a good way to lose money, kind of like the joke that the way a rich man can lose money really fast is to invest in an airline.
Tim Siglin: 26:19 If indeed you get scale with AVC, you could find yourself with a really large bill. That look at HEVC is being not just for 4K, HDR, or high frame rate in the future, but also for 1080p with some HDR and high frame rate. It tells me that the codec itself or the promise of the codec itself was actually really good. What was even more fascinating to me was the number of companies that had AVC pipelines that were actually looking to integrate HEVC into those same production pipe.
Tim Siglin: 26:55 It was much easier from a process standpoint to integrate HEVC into an AVC pipeline, so in other words, H265 into H264 pipeline than it was to go out of house and look at something like AV1 or VP9, because the work that was done on HEVC builds on the benefits that were already in place in AVC. Of course, you got Apple who has HLS, HTTP Live Streaming, and a huge ecosystem in terms of iPhones and iPads, laptops and desktops supporting HEVC not just as a standard for video delivery, but also with the HEIC or HEIF image format, now having all of their devices shoot images using HEVC instead of JPEG. That in and of itself drives forward adoption of HEVC. I think you told me since that survey came out, probably now seven months ago, you all have continued to see the model of all-in HEVC adoption.
Dror Gill: 28:03 This is what we promote all the time. It’s kind of a movement. Are you all in HEVC or are you doing it just for 4K, just where you have to do it? We really believe in all-in HEVC. Actually, this week, I had an interesting discussion with one of our customers who is using our optimization product for VOD content, to reduce bit-rate of H.264 (streams). He said, “I want to have a product. I want to have a solution for reducing bit-rates on our live channels.”
Dror Gill: 28:32 So, I asked them, “Okay. Why don’t you just switch your codec to HEVC?” He said, “No, I can’t do that.” I said, “Why not?” He said, “You know compatibility and things like that.” I asked, “Okay. What are you using? What are you delivering to?” He said, “We have our own set-top boxes (STB), IP set-top boxes which we give out to our customers. Well, these are pretty new.” So, they support HEVC. I’m okay there. “Then we have an Apple TV app.” “Okay, Apple TV has a 4K version. So, it supports HEVC. All of the latest Apple TV devices have HEVC. That’s fine.” “Then we have smartphone apps, smart TV apps for Android TV and for the LG platform.”
Dror Gill: 29:15 Obviously, TV’s support 4K. So, I’m okay there. With delivering to mobile devices, all the high-end devices already support HEVC. He was making this estimate that around 50 to 60% of his viewers are using devices that are HEVC capable. Suddenly, he’s thinking, “Yeah, I can do that. I can go all in HEVC. I will continue, of course, to support H.264 for all of the devices that don’t support HEVC. But if I can save 50% of the bandwidth to 50 to 60% of my customers, that’s a very big savings.”
Mark Donnigan: 29:48 What’s interesting about this conversation, Dror, is first of all I’m pretty certain that the operator you’re talking with is different than the operator that I shared, found the exact same thing. This is a consistent theme, is that pretty much in developed parts of the world, it really is true that 50% or more of the users can today receive HEVC. This number is only growing. It’s not like it’s static It is just growing. Next year, I don’t know if that number will be 60% or 70%, but it’s going to be even bigger.
Mark Donnigan: 30:27 What’s fascinating is that, again, we’ve said earlier, that the consumer is getting just more aware of quality, and they’re getting more aware of when they’re being underserved. For operators who are serving to lowest common denominator, which is to say, AVC works across all my devices, and it’s true. AVC works on all the high-end devices equally well, but you’re under-serving a large and growing number of your users.
Mark Donnigan: 31:01 If your competitors are doing the same, then I guess you could say … well, “Who are they going to switch to?” But there are some fast-moving leaders in the space who are either planning or they’re shortly going to be offering better quality. They’re going to be extending HEVC into lower bit rates or lower resolutions, that is, and therefore lower bit rates, and the consumers are going to begin to see like, “Well, wait a second. This service over here that my friend has or we have another subscription in the household, how come the video looks better?” They just begin to migrate there. I think it’s really important when we have these sorts of conversations to connect to this idea that don’t underserve your consumer in an effort to be something to everybody.
Tim Siglin: 31:57 I would add two other quick things to that, Mark. One is, we’ve always had this conversation in the industry about the three-legged stool of speed, quality and bandwidth in terms of the encoding.
Tim Siglin: 32:09 Two of those are part of the consumer equation, which is quality and bandwidth. Then, oftentimes, we’ve had to make the decision between quality and bandwidth. If the argument is ostensibly that HEVC as it stands right now, had a couple years of optimization, can get us to about, let’s say, 40%. Let’s not even say 50%. For equivalent quality, it can get us to 40% bandwidth reduction. Why wouldn’t you switch over to something like that?
Tim Siglin: 32:39 Then the second part, and I have to put a plugin for what Eric Schumacher-Rasmussen and the Streaming Media team did at Streaming Media West by having Roger Pantos come and speak, Roger Pantos being of course the inventor of HLS, and I’m not a huge fan of HLS, just because of the latency issues. But he pointed out in his presentation, his tutorial around HLS that you can put two different codecs in a manifest file. There is absolutely no reason that an OTT provider could not provide both HEVC and AVC within the same manifest file and then allow the consumer device to choose.
Tim Siglin: 33:22 When Dror mentioned the company who has the OTT boxes that they give away, they could easily set a flag in those boxes to say, “If you’re presented with a manifest file that has AVC and HEVC, go with HEVC to lower the bandwidth, overall.” The beauty is it’s a technical issue at this point and it’s a technical implementation issue, not a ‘can we make it work?’ Because we know that it works based around the HLS.
Mark Donnigan: 33:54 This is excellent. Tim, let’s wrap this up, as I knew it would be. It has just been an awesome conversation. Thank you for sharing all your years of collective experience to give some insight into what’s happening in the industry. Let’s look at 2019. I know we’ve been talking a little bit about … you’ve made references to ATSC 3.0. Some of our listeners will be going to CES. Maybe there’s some things that they should be looking at or keeping their eyes opened for. What can you tell us about 2019?
Tim Siglin: 34:35 Here’s what I think 2019 is bringing. We have moved in the cloud computing space and you all are part of this conversation at Beamr. We’ve moved from having cloud-based solutions that were not at parity with on-premise solutions to actually in 2018 reaching parity between what you could do in an on-premise solution versus the cloud. Now, I think in 2019, what we’re going to start seeing is a number of features in cloud-based services, whether it’s machine learning, which the popular nomenclature is AI, but I really like machine learning as a much better descriptor, whether it’s machine learning, whether it’s real-time transcoding of live content, whether it’s the ability to simultaneously spit out AVC and HEVC like we’ve been talking about here that the cloud-based solutions will move beyond parity with the on-premise solutions.
Tim Siglin: 35:35 There always will be needs for the on-premise parts from a security standpoint in sort of the industries, but I don’t think that will inhibit cloud-based in 2019. If people are going to CES, one of the things to look at there, for instance, is a big leap in power consumption savings for mobile devices. I’m not necessarily talking about smartphones, because the research I’ve done says the moment you turn GPS on, you lose 25% of battery. Tablets have the potential to make a resurgence in a number of areas for consumers and I think we’ll see some advances in battery (capacity).
Tim Siglin: 36:19 Part of that goes to HEVC, which as we know is a much harder codec to decode. I think the consumer companies are being forced into thinking about power consumption as HEVC becomes more mainstream. That’s something I think people should pay attention to as well. Then, finally, HDR and surround sound solutions, especially object placement like Dolby Atmos and some of these others, will become much more mainstream as a way to sell flat panels and surround sound systems.
Tim Siglin: 36:56 We sort of languished in that space. 4K prices have dropped dramatically in the last two years, but we’re not yet ready for 8K. But I think we’ll see a trend toward fixing some of the audio problems. In the streaming space, to fix those audio problems, we need to be able to encode and encapsulate into sort of the standard surround sound model. Those are three areas that I would suggest people pay attention.
Mark Donnigan: 37:25 Well, thank you for joining us, Tim. It’s really great to have you on. We’ll definitely do this again. We want to thank you, the listener, for supporting the Video Insiders. Until the next episode. Happy encoding!
Announcer: 37:39 Thank you for listening to the Video Insiders Podcast, a production of Beamr Imaging Limited. To begin using Beamr’s codecs today, go to Beamr.com/free to receive up to 100 hours of no cost HEVC and H.264 transcoding every month.
The news of Beamr building a transcoder is spreading fast. Many are applauding the move and have expressed interest in adopting – but some may be wondering why a pioneering codec technology company feels the need to build a transcoder? Well, here’s the story.
Beamr was founded on a simple but technically complex idea that there are opportunities to deliver massive value to the video streaming and distribution ecosystem by guaranteeing the smallest video file and stream size possible without compromising video quality.
With the introduction of Beamr Video (now Beamr Optimizer), we proved that we could deliver an additional bitrate savings of up to 50% at the same quality. An incredible feat acknowledged by well-known streaming services and tier-one studios in Hollywood who adopted the technology for use in workflows including Blu-ray disc authoring which represents the highest pinnacle of direct-to-consumer quality.
By incorporating our perceptually driven optimization process (content-adaptive) into the codec, we enable MSO’s and streaming video delivery services to meet their objective of not delivering a single bit more than needed to represent the absolute quality they desire to transmit. This led us to acquire Vanguard Video in April 2016, a company providing codec engineering and software development services for many of the best-known video encoder OEM’s and streaming services. Now with our best-in-class codec, combined with our highly patented and well established perceptual optimization technology, we have the most advanced HEVC video codec in the market: Beamr 5x.
As a video codec does not function standalone outside of a lab environment, all Beamr Codec SDKs work inside the framework of FFmpeg through our plug-in interface. Through the development of our FFmpeg plug-in and as we surveyed the needs of our Telecom and MSO customers, it became clear that FFmpeg could not meet their requirements for industry-standard integration points such as SCTE-35 ad insertion.
What modern video services need in a transcoder.
To deliver uncompromising performance and system integration that meets our customer’s expectations, we built a transcoder that can enable new video workflow architectures with a value-added and performance-oriented engineering philosophy. The Beamr Transcoder is designed to work seamlessly in virtualized video encoding workflows and provides a full transcoding solution which includes Beamr codec SDKs: Beamr 4, Beamr 5, and Beamr 5x.
The first thing we recognized after talking with customers about what they were looking for in a transcoder is that the days of the monolithic black box “workflow solution” that includes content management, transcoding, packaging, DRM, etc. were over.
Customers were unanimous in reporting that they are tired of being held hostage by vendors.
Everyone said that they need a flexible media processing framework to fulfill the requirements of their video encoding service today and in the future.
We took this feedback as inspiration and adopted a critical design philosophy. The transcoder must satisfy the functional framework requirements to work in today’s containerized virtual computing and cloud environments while connecting to complementary solutions from any vendor.
Why FFmpeg isn’t enough.
A major performance advantage with Beamr codec SDKs is our multi-bitrate encoding capability. With multi-bitrate encoding, the SDK can share analysis information from the highest quality tier to the lowest bitrate tiers of the same picture size (resolution). This feature offers significant CPU and performance efficiencies, which allows the quality level to be dialed up for a given job, in a specified turnaround (processing) time. For anyone generating ABR profile stacks, Beamr’s multi-bitrate functionality is a game changer.
Today’s transcoder is a video processing framework that supports video codecs and the associated processing required for audio, metadata, etc. The performance advantages of Beamr’s H.264 and HEVC codec SDKs which ship inside the Beamr Transcoder start with much better parallelism (tiled encoding). Additionally, Beamr codec SDKs enable highly efficient CPU utilization on a many-core server and include better overall thread pool management.
Over the years as OTT workflows developed, many video distributors found FFmpeg as a media processing framework a good solution, but in time and as technical requirements and compatibility needs increased it has become less viable. FFmpeg is flexible, with many libraries and filters supporting container files and audio/video codecs. It also works with any video transformation during the transcoding process (decode – transform – encode): scaling function, frame rate conversion, color space conversion, bit depth conversion, filtering (noise reduction), etc.
Because the libraries and filters were developed separately, to make everything work in every combination needed, each filter must be run sequentially, passing the resulting uncompressed video frame to the next filter after it is ready. When the first processing step is complete, and the pixels or data needed for the next step are in the memory cache of the processor, it is more efficient to perform the following processing step immediately, as a tightly synchronized operation, all without saving the intermediate results to system memory, or disk. FFmpeg was developed asynchronously by hundreds and thousands of contributors, meaning this operational structure was not possible to build outside of a closely coordinated product roadmap, which by definition is lacking in open source projects.
Beamr Transcoder was built using a single thread pool that runs all processing tasks easily while outperforming FFmpeg as a media processing framework that runs multiple libraries with their own separate thread pool’s.
How we approached development of the Beamr Transcoder.
The Beamr Transcoder is lightweight, flexible, and not loaded down with legacy bloat. It may not meet a long laundry list of supported technologies, but, as service providers and OTT streaming services are shedding their legacy requirements in favor of lightweight, agile solutions that run 100% in software spanning virtualized environments, this is an advantage to the product. Designed for today’s video workflows, the Beamr Transcoder can scale across on-prem, public, private, or hybrid cloud infrastructures.
The heart of this highly efficient and flexible transcoder is an engine that was developed in native C++ to run on Linux while being highly scalable across public clouds such as AWS using Docker containers.
Beamr Transcoder VOD 1.0 features overview.
Beamr Transcoder is finely tuned and automatically allocates the optimal number of threads to the video encoding function, reserving the minimum needed for decoding, pipeline management, and network function control. Following is a summary of the critical capabilities of the first version of Beamr Transcoder VOD. (Beamr Transcoder Live will ship later in 2018)
– Supports high resolution, high frame rate (HFR) up to 8Kp120
– SCTE-35 pass-through
– Encoding Boundary Points (EBP) support
– EIA 608/708 closed captioning
– Logo insertion
– Support for high dynamic range (HDR): HDR-10/HLG
Where we are going.
With Beamr Transcoder, Beamr’s HEVC & H.264 codec SDKs are now full video encoding solutions that can power your service to deliver the highest quality at the lowest bitrate possible. Like all Beamr solutions, the Beamr Transcoder has a fully committed road-map with the industries most reliable video technology engineering team executing it.
By leveraging Beamr’s video transcoder and the included encoding solutions, video encoding operations will operate as much as three times faster creating files that are up to 50% smaller.
To experience the performance of Beamr Transcoder which comes with an innovative business model that includes a free and premium version along with our highest rated HEVC and H.264 codec SDKs, send us an email to transcoder@beamr.com.
Visit beamr.com/transcoder to download the Beamr Transcoder VOD product information sheet or CLICK HERE for the product page.