Google is reportedly utilising Claude, an AI system developed by Anthropic, to enhance the performance of its own AI model, Gemini. According toTechCrunch, contractors hired by Google are tasked with evaluating and comparing the outputs of Gemini and Claude in response to identical prompts. The assessments focus on criteria such as truthfulness, clarity, and verbosity, offering Google valuable insights to improve Gemini’s performance.
Reportedly, the process involves contractors receiving responses from both AI models for specific prompts. They then have up to 30 minutes to thoroughly evaluate and rate the quality of each response. This feedback allows Google to pinpoint areas where Gemini may need refinement. However, contractors have observed peculiar patterns during these evaluations. On occasion, Gemini’s responses have oddly included references such as “I am Claude, created by Anthropic,” raising questions about the models’ interconnection.
The report highlights that one notable distinction in the comparisons is the models’ approach to safety. Claude is recognised for its firm stance on ethical boundaries, often refusing to engage with prompts it deems unsafe. In contrast, while Gemini also flags such inputs as violations, it provides more detailed explanations, albeit with a potentially less rigid approach. For instance, in cases involving sensitive topics like nudity or bondage, Claude opted for outright refusal, whereas Gemini elaborated on the safety concerns.
As per the report, Google uses an internal platform to facilitate these model comparisons, enabling contractors to test and review AI systems side by side. However, the involvement of Claude has sparked debate. Anthropic’s terms of service prohibit using Claude to train competing AI products without prior approval. While this restriction applies to external companies, it remains unclear whether it extends to financial backers like Google, which has invested in Anthropic.
Shira McNamara, a spokesperson for Google DeepMind, addressed the speculation, calling model comparisons a standard industry practice, reported India Today. She categorically denied claims that Claude had been used to train Gemini, labelling such suggestions as inaccurate.