Home
Internet
Internet News
YouTube's New Automatic Caption Tool Can Now Describe Sound Effects

YouTube's New Automatic Caption Tool Can Now Describe Sound Effects

By Ketan Pratap | Updated: 27 March 2017 16:32 IST

Click Here to Add Gadgets360 As A Trusted Source

Highlights

YouTube to soon automatically detect sound effects
The company is using machine learning
Initially, sound effects like applause, music, laughter will show up

YouTube introduced captions in its videos back in 2007, and made automated captions for speech available a few years later. The company will soon also start describing sound effects in videos through machine learning. YouTube has developed a sound effect captioning system for its video platform collaborating with Sound Understanding and Accessibility teams. The automatic sound effect captioning system will identify and label sounds in the video without manual input.

With machine learning, YouTube will be able to automatically detect the existence of sound effects in a video and transcribe it to appropriate classes or sound labels. YouTube will soon start showing sound effects like [APPLAUSE], [MUSIC], and [LAUGHTER]. The company explains that "these were among the most frequent manually captioned sounds, and they can add meaningful context for viewers who are deaf and hard of hearing."

YouTube stresses that the new changes will help the 360 million people around the world who have problems in hearing. The company has so far made several changes to cater to these users, and claims that the number of videos with automatic captions now exceeds 1 billion while adding that people watch videos with automatic captions more than 15 million times per day.

"We started this project by taking on a wide variety of challenges, such as how to best design the sound effect recognition system and what sounds to prioritise. At the heart of the work was utilising thousands of hours of videos to train a deep neural network model to achieve high quality recognition results," said Noah Wang, Software Engineer in a blog post.

The company adds that its new captioning tech is still in the early stages of recognising sound effects automatically. YouTube lists some more challenges that will make video watching experience even better for the targeted users. "Future challenges might include adding other common sound classes like ringing, barking and knocking, which present particular problems -- for example, with ringing we need to be able to decipher if this is an alarm clock, a door or a phone."

Comments

Get your daily dose of tech news, reviews, and insights, in under 80 characters on Gadgets 360 Turbo. Connect with fellow tech lovers on our Forum. Follow us on X, Facebook, WhatsApp, Threads and Google News for instant updates. Catch all the action on our YouTube channel.

Further reading: YouTube, Google

Ketan Pratap

Email Ketan

Ketan Pratap is the Editor at Gadgets 360. His primary role - debugging the tech hype, benchmarking the future, and compiling it all into precise news, features or reviews. He has spent over 14 years calling out vaporware and identifying the best tech. He’s your guide for everything from chipsets to smart-home meltdowns. When his own batteries are low, he heads for the hills—literally. He’s someone who prefers a remote mountain pass, appreciating a 12,000-foot view that no VR ...More