Google's Latest Experiment Allows Your Paint Brush to Produce Musical Notes

Paint with Music tool's canvas, machine learning, and special sound effects come together to make your painting produce music.

By Edited by Gadgets 360 Newsdesk | Updated: 1 July 2021 12:39 IST

Google's Latest Experiment Allows Your Paint Brush to Produce Musical Notes

Photo Credit: Paint with Music/ Experiments with Google

Paint with Music offers a variety of tools that are easy to learn and use

Highlights

Paint with Music tool was launched in June
It is powered by Magenta's DDSP library
Machine learning plays a huge role in the Paint with Music tool

Google has introduced a new tool — Paint with Music — that allows you to have a one of a kind experience. Launched in June this year, the platform offers an interactive experience, connecting two completely different but major art forms — painting and music composition. How does it work, you ask?

Powered by Magenta's DDSP library (Differentiable Digital Signal Processing) and assisted by machine learning, the Paint with Music tool transforms your brush strokes into musical notes. Your paintbrush is essentially the tool that helps you create music. And yes, you can play it on any instrument from an elaborate list available. So from flute to saxophone, you have the option of painting your notes on a variety of sensorial canvases. These include "in the sky," "underwater," "on the street," and "on paper."

The canvas, machine learning (ML), and special sound effects on the platform come together to make your painting talk or produce music. Having said that, a little bit of understanding of musical notes will come in handy. The tool is fairly easy to use. On the Paint with Music website or the app, once you launch the experiment, Google gives you four canvas options to choose from.

Once you have selected your favourite canvas, the next thing to factor in is the musical instrument you want your paintbrush to sound like every time you draw something. On the right bottom of your screen, you have the option to undo a note that you have drawn or even delete the entire painting and start afresh.

This is the second Google experiment that involves painting as the mainstay of the tool. In November 2020, Google AI's Chimera Painter enabled users to create a fleshed-out creature based on a rough sketch. The tool added features and textures online and this machine learning model was created with Unreal Engine and the GAN-based (generative adversarial network) ML algorithm. Even though the tool primarily focussed on the needs of game developers, anyone could use it and turn a rough doodle into a monster that looked real.

“What if artists had a paintbrush that acted less like a tool and more like an assistant? A machine learning model acting as a paintbrush could reduce the amount of time necessary to create high-quality art without sacrificing artistic choices, perhaps even enhancing creativity,” the Google AI team wrote back then in a blog post, explaining the Chimera Painter demo.

We discuss the return of PUBG Mobile, sorry, Battlegrounds Mobile India on Orbital, the Gadgets 360 podcast. Orbital is available on Apple Podcasts, Google Podcasts, Spotify, Amazon Music and wherever you get your podcasts.

Affiliate links may be automatically generated - see our ethics statement for details.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.