Netflix’s Adaptive Streaming Algorithm Will Make Videos Sound Better

The opening scene of season two of Netflix’s Stranger Things, a popular horror series that appeals to pretty much everyone who grew up in the 1980s, would sound great to the average ear. A little loud, maybe, since it’s a cacophony of car chase sounds. But shortly after season two was released in October of 2017, Netflix’s internal sound technicians noticed it didn’t actually sound great. There was an issue with the mix, says Scott Kramer, Netflix’s manager of sound technology. “A lot of it was mushy,” he says.

Words like “mushy” and “smeared” are ones that Kramer and his team use often when describing audio that just isn’t quite as crisp as it should be. It’s these kinds of observations, ones barely discernible to the average ear, that led Netflix to spend the past several months working on a solution that would make audio across all of its programming—both original shows and licensed content—sound better.

Doing this required more than just bumping up the bit rate on its audio streams. Instead, the company borrowed some tricks from its adaptive video streaming process, which uses machine learning to recognize network conditions (read: the strength of your internet connection) and adjust the video quality up or down based on that. The changes to Netflix’s audio start rolling out to users today with the straightforward name “high-quality audio.”

The sound upgrade won’t technically be applied to everyone. It’s optimized for people with streaming boxes, speakers, and soundbars that support Dolby 5.1 and Dolby Atmos sound. And when I asked Netflix’s sound engineers to cite a minimum bandwidth requirement for this improved audio experience, the answer was a little murky (mushy?). You won’t need to upgrade your internet package to tack an extra 10 megabits per second onto your connection, says Sean Sharma, one of Netflix’s engineering directors. At the high end, the new audio bit rate requires roughly half a megabit per second of additional bandwidth.

That’s probably a useful tidbit for people who monitor their internet download speeds very closely, but the point of the new feature is that it’s supposed to adjust automatically. As part of the engineering process, Netflix first established a kind of ladder for audio bit rates. This ranged from the minimum audio quality Netflix normally offers—192 Kbps using a Dolby audio codec—scaling all the way up to 640 Kbps, the bit rate the engineers thought was the highest they needed to achieve what’s known as “perceptual transparency.” Beyond 640 Kbps, there was no audible benefit, they determined.

Turn It Up

As part of a demo of the improved sound quality, Netflix sat me in a meeting room it calls Full House. Giant images of DJ and Kimmy Gibbler plastered on walls stared at me during the demo. I was surrounded by at least 10 ATC (read: very expensive) speakers, while a large monitor in front of me offered a visual representation of the audio upgrade I was about to experience. A Netflix engineer played a simple, 20-second track of studio applause and toggled between the near-perfect studio master track, a track encoded at the old bit rate of 192 Kbps, and a track encoded at the new optimal bit rate of 640 Kbps.

I’m not an audio engineer or even much of an audiophile, but the applause track streamed at 640 Kbps sounded uncannily close to the studio master track and offered a crispness that the 192 Kbps stream couldn’t match.

Then Netflix’s engineers showed off an even more extreme example. They streamed the scene from Stranger Things in which the character Will first discovers the Upside Down. In that clip, an insect buzzes in the background, a small noise that adds volumes to the eeriness of the scene. When the audio was streamed at 192 Kbps, the insect was muffled, almost muted. On the higher-quality streams, it was much more obvious. And sure, the highly controlled demo I got was done on an optimal audio setup, but that bug sounded like it was buzzing right by my ear.

Audio Active

Technically speaking, there was nothing preventing Netflix from rolling out this higher-quality audio before today. Kramer points out that the streaming video service’s sound quality was good before—“industry standard”—and that these audio changes will be subtle in some cases.

Some Netflix subscribers might disagree with the assertion that the sound quality was totally fine before, but Netflix has now at least assembled a team that’s dedicated to improving it. Another audio demo I experienced included scenes from Beyoncé’s musical film Homecoming, a blend of exhilarating performance footage and behind-the-scenes clips that underscore just how hard Beyoncé works at being Beyoncé. Netflix’s engineers joked that they were simply waiting for Homecoming to come out before upgrading the audio, which seemed about as good of a reason as any.