Here's Why Microsoft Built a Tool to Guess Your Emotions

By Abby Ohlheiser, The Washington Post | Updated: 14 November 2015 18:37 IST

Months after Microsoft released a tool designed to guess the age of someone based on a photograph, the company has unleashed another round of fun tools designed to show off its machine learning and AI capabilities. The most appealing tool in the new batch promises to detect what emotion someone is feeling in a particular, photographed, moment, based on cues in their facial expression.

These APIs (you can see them at https://www.projectoxford.ai/ ) are designed to show off Microsoft's growing AI and machine learning capabilities for developers who might be interested in incorporating them into their own projects. The age guesser, and this month's Movember-themed facial hair analyzer, are supposed to show how developers might be able to get creative in using these tools, and the range that's capable here.

And, for users to opt in, Microsoft says that some of the tools can improve themselves using your input.

Machine learning tools like these generally get better as they're exposed to more data, and there's been ongoing concerns about how Microsoft might use the photos uploaded to these tools in the future - prompted by this section of the company's blanket terms of use, linked to by the Movember and How Old Am I? tools:

"Microsoft does not claim ownership of the materials you provide to Microsoft (including feedback and suggestions) or post, upload, input or submit to any Services or its associated services for review by the general public, or by the members of any public or private community, (each a "Submission" and collectively "Submissions").

"However, by posting, uploading, inputting, providing or submitting ("Posting") your Submission you are granting Microsoft, its affiliated companies and necessary sublicensees permission to use your Submission in connection with the operation of their Internet businesses (including, without limitation, all Microsoft Services), including, without limitation, the license rights to: copy, distribute, transmit, publicly display, publicly perform, reproduce, edit, translate and reformat your Submission; to publish your name in connection with your Submission; and the right to sublicense such rights to any supplier of the Services."

To try and cool some of those concerns, Microsoft's outward-facing machine learning sites - like How Old - now contain a note stating that Microsoft doesn't keep the photos you upload, unless you check a box and give them permission to do so - which they're calling an opportunity to "donate" to science.

Although programs like these learn by processing more and more data, Microsoft says that not every photo uploaded to one of these tools is used for that purpose. "The APIs don't get better just from people using them. However, if people opt in to letting us keep their photos, we would like to use those photos to help us improve," Ryan Galgon, senior program manager for Project Oxford, said in an email to The Washington Post. Otherwise, Galgon said, the company doesn't keep the photos you upload, or use them to improve the service.

That will work a little differently when it comes to independent applications that developers build using these APIs, a spokeswoman for Microsoft said, because what they choose to keep from users of their own apps will be mainly up to them.

"We have a Code of Conduct to help developers understand our expectations about protecting people whose data they collect and legal terms to help us enforce good behavior if necessary," spokeswoman Melissa Hovis said in an e-mail, "Our legal terms also require developers to get consent."

Those independent apps could very well keep the data they collect on you for any number of purposes, but Microsoft said it's taken steps to avoid privacy violations on their end. "Microsoft has processes in place that help de-identify user generated content in audio, video, images and other data when it is processed by Project Oxford APIs," Hovis added. "For example, any images that are provided by an application to the Face API are stripped of any metadata, session data or IP addresses to protect the privacy of people in those images."

When more developers start using these APIs, though, Microsoft might have a bigger pool of user-submitted information to learn from. Microsoft "may use the data" sent from an independent app to the API to "perform and improve our services," the company said.

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.