Researchers at Oxford University’s Internet Institute have discovered that large language models specifically tuned to exhibit empathy and warmth are more likely to make factual errors.
According to a report by Ars Technica, the study—published this week in the journal Nature—found that these 'war/er' models mimic human tendencies to 'soften difficult truths' in an effort to preserve social bonds and avoid conflict.
To conduct the study, researchers used supervised fine-tuning techniques to modify several models, including Llama-3.1, Mistral-Small, Qwen-2.5, and OpenAI's GPT-4o. The goal was to increase 'expressions of empathy, inclusive pronouns, informal register and validating language.'
While the tuning instructions explicitly commanded the models to 'preserve the exact meaning, content, and factual accuracy of the original message,' the results showed a breakdown in performance. The Ars Technica report notes that these warmer models were more likely to validate a user's incorrect beliefs, particularly when the user expressed feelings of sadness.
The cost of warmth
The researchers defined 'warmness' as the degree to which an AI's output leads users to infer positive intent, such as friendliness and trustworthiness. They used the SocioT score and double-blind human ratings to confirm that the fine-tuned models were perceived as warmer than their original counterparts.
However, this increased sociability came at a cost to accuracy. The study found that the models trained to be more empathetic showed a higher error rate across various tasks.
This tendency mirrors human social behavior, where the desire to be polite or empathetic often conflicts with the need to be truthful. The researchers found that the models essentially prioritized the user's emotional state over the preservation of factual truth.
