Google Reportedly Using Anthropic’s Claude to Improve Gemini’s Outputs

Google reportedly asked contractors working on Gemini to evaluate its answers against Claude’s.

Written by Akash Dutta, Edited by Siddharth Suvarna | Updated: 26 December 2024 16:47 IST

Google Reportedly Using Anthropic’s Claude to Improve Gemini’s Outputs

Google is a major investor in Anthropic

Highlights

Anthropic forbids users from using Claude to build competing products
Gemini contractors evaluate its responses on multiple parameters
One contractor reportedly found Claude’s reference in Gemini’s responses

Google is reportedly comparing Gemini's responses against those generated by Anthropic's Claude artificial intelligence (AI) models. As per the report, the Mountain View-based tech giant asked third-party contractors who work on evaluating the quality of Gemini's responses to also rate whether Claude's responses are better. While it is not an uncommon practice, using third-party AI models for this purpose requires obtaining permission from the company. The report could not confirm whether the tech giant had received any permission from Anthropic.

Gemini Reportedly Being Improved Using Claude

According to a TechCrunch report, contractors working on Gemini are comparing its outputs against those created by Claude, a direct competitor of the former. The publication claimed to have seen internal correspondence between Google and the contractors, where they were told to follow this practice.

The report also claimed that the contractors are given up to 30 minutes per prompt where they are tasked to determine if they prefer Gemini or Claude's response. Notably, these contractors are typically subject matter experts and they evaluate the chatbot's responses on niche and technical topics across several parameters such as truthfulness, precision, and verbosity.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

After the practice began, some contractors reportedly began noticing references to Anthropic and Claude in Gemini's responses. The publication claimed that one Gemini output directly stated “I am Claude, created by Anthropic.” Typically, this should not happen if Gemini's responses are simply being compared against Claude's. This raises the concern of whether developers are feeding Claude's output to Gemini in instances where the former's responses are better.

In its commercial terms of conditions, Anthropic states that those accessing Claude are forbidden from building “competing product or service” or training “competing AI models” without obtaining approval from Anthropic.

Speaking with TechCrunch, Google DeepMind's spokesperson Shira McNamara said that while the division does compare model outputs for evaluation purposes, Gemini is not trained on Anthropic's AI models.

Last week, a report claimed that the contractors assigned to evaluate Gemini's responses were being told to rate the chatbot's outputs even when they were outside of their expertise. While earlier, contractors could skip such questions, the option to do so was reportedly being removed.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: Google, Gemini, Anthropic, Claude, AI, Artificial Intelligence

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More