Home
Apps
Apps News
Google Assistant Defeats Siri, Alexa, Cortana in IQ Test by Loup Ventures

Google Assistant Defeats Siri, Alexa, Cortana in IQ Test by Loup Ventures

By Jagmeet Singh | Updated: 26 July 2018 13:55 IST

Highlights

Google Assistant was able to answer 86 percent of questions
Siri answered 79 percent questions correctly
Cortana was found to be the "laggard" in the race

Amazon, Apple, Google, and Microsoft are all working on artificial intelligence (AI) to enhance their virtual assistants. But a new report shows that Google's Assistant has come first in an IQ test that saw it compete with Amazon's Alexa, Apple's Siri, and Microsoft's Cortana. It appears that the recent developments by the search giant have improved the Assistant quite significantly and made it a strong contender against Siri, which has got the second position in the test. All four voice-based virtual assistants were asked 800 questions.

Research-driven venture capital firm Loup Ventures on Thursday published the results of its annual digital assistant IQ test that made the Google Assistant a clear winner against Siri, Cortana, and Alexa. Of the 800 questions that were asked to each digital assistant, Google Assistant was able to answer 86 percent of them correctly, while Siri answered 79 percent correctly, Alexa answered 61 percent correctly, and Cortana answered 52 percent correctly, the firm claimed. The questions were spread across five categories, including local, commerce, navigation, information, and command. Further, the questions were graded on two metrics; one was to see whether the assistant understood what was being asked and the other one was to observe whether it was able to deliver a correct response.

Loup Ventures says that Google Assistant was able to understand all the 800 questions during the test, though it was able to give correct answers to 86 percent of them. Siri, on the other hand, was close behind the Google offering by correctly answering 79 percent of the given questions and misunderstanding only 11 questions. After Siri, it was Alexa that correctly answered 61 percent of questions and misunderstood 13, while Cortana was found to be the "laggard" in the race by correctly answering 52 percent and misunderstanding as many as 19 questions. "Both the voice recognition and natural language processing of digital assistants across the board has improved to the point where, within reason, they will understand everything you say to them," the firm said while releasing the IQ test results.

Photo Credit: Loup Ventures

Against the test that it had conducted last year, Loup Ventures has modified the questions for this year's version to understand the improvements of the assistants over time. It was also noticed that the issues that had emerged in the last year examination have been erased this time with improvements to natural language processing and inter-device connectivity.

Google Assistant won four out of the five categories, lagging behind Siri just on the "command" front that tests the execution of a specific feature. Nevertheless, the "information" part of the Assistant is praised to be better than other digital assistants. "Google Assistant's outperformance stems largely from the search function 'featured snippets.' This is the box at the top of the search results in your browser with bold text, a link to learn more, and, often times, exactly what you're looking for," Loup Ventures said.

Contrary to Google Assistant, Siri was found to have an edge when it comes to music-related queries. The Apple assistant was also capable of responding more to "flexible language" and versatile enough to control phones, smart home devices, and music. Alexa and Cortana, on the other side, lagged behind Google Assistant and Siri. Alexa, which was originally expected to at least answer the commerce questions properly - considering its parent's legacy, wasn't able to give a broader listing of products over "Amazon's choice" each time when asked about a product. "When Alexa detects a query related to a product, it will almost always present "Amazon's Choice" for whatever it may be," the firm stated.

In contrast, Cortana surfaced as the most inferior offering. The Microsoft assistant performed quite poorly in the "commerce" category as it responded correctly only 20 percent of queries.

Loup Ventures has observed that Google Assistant and Apple's Siri are taking on the competition when it comes to the performance in the "command" category - thanks to their integration within phones. This is the key area where Alexa and Cortana lag as they come in the form of an app rather baked directly into the device. The firm also highlighted that over a 15-month period, Google Assistant improved by 11 percent, Siri 13 percent, and Cortana only 3 percent in terms of questions answered correctly. The data of Alexa hasn't been provided, though.

"With scores nearing 80-90 percent, it begs the question, will these assistants eventually be able to answer everything you ask? The answer is probably not, but continued improvement will come from allowing more and more functions to be controlled by your voice. This often means more inter-device connectivity (e.g., controlling your TV or smart home devices) along with more versatile control of functions like email, messaging, or calendars," the firm said.

The IQ test was conducted using Siri on iOS 11.4, Google Assistant on Pixel XL, and Alexa as well as Cortana via their iOS apps. It also included connected devices, namely Wemo mini plug, TP-Link Kasa plug, Phillips Hue Lights, and Wemo Dimmer Switch.

Loup Ventures estimates that over time, payments and ride-hailing are likely to be the two use cases that will become a standard capability of the leading voice assistants. Amazon, Apple, and Google are also working hard to make the voices of their assistants more natural to enhance their experiences.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.