Home
Internet
Internet News
YouTube's New Automatic Caption Tool Can Now Describe Sound Effects

YouTube's New Automatic Caption Tool Can Now Describe Sound Effects

By Ketan Pratap | Updated: 27 March 2017 16:32 IST

Highlights

YouTube to soon automatically detect sound effects
The company is using machine learning
Initially, sound effects like applause, music, laughter will show up

YouTube introduced captions in its videos back in 2007, and made automated captions for speech available a few years later. The company will soon also start describing sound effects in videos through machine learning. YouTube has developed a sound effect captioning system for its video platform collaborating with Sound Understanding and Accessibility teams. The automatic sound effect captioning system will identify and label sounds in the video without manual input.

With machine learning, YouTube will be able to automatically detect the existence of sound effects in a video and transcribe it to appropriate classes or sound labels. YouTube will soon start showing sound effects like [APPLAUSE], [MUSIC], and [LAUGHTER]. The company explains that "these were among the most frequent manually captioned sounds, and they can add meaningful context for viewers who are deaf and hard of hearing."

YouTube stresses that the new changes will help the 360 million people around the world who have problems in hearing. The company has so far made several changes to cater to these users, and claims that the number of videos with automatic captions now exceeds 1 billion while adding that people watch videos with automatic captions more than 15 million times per day.

"We started this project by taking on a wide variety of challenges, such as how to best design the sound effect recognition system and what sounds to prioritise. At the heart of the work was utilising thousands of hours of videos to train a deep neural network model to achieve high quality recognition results," said Noah Wang, Software Engineer in a blog post.

The company adds that its new captioning tech is still in the early stages of recognising sound effects automatically. YouTube lists some more challenges that will make video watching experience even better for the targeted users. "Future challenges might include adding other common sound classes like ringing, barking and knocking, which present particular problems -- for example, with ringing we need to be able to decipher if this is an alarm clock, a door or a phone."

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: YouTube, Google

Ketan Pratap

Email Ketan

Ketan Pratap is the editor at Gadgets 360 - with over 12 years of experience covering the technology domain. With a breadth and depth of knowledge in the field, he's done extensive work across news, features, reviews, and opinion pieces. But what's truly inspiring about Ketan is how he spends his free time. He's often found gazing at snow-capped mountains from over 20,000 feet while sitting on the hood of his car, taking in the breathtaking beauty of nature. His passion for the great ...More