Khaberni - If you are using an AI-powered chatbot, such as "Chat GPT," "Gemini," or "Claude," on a daily basis, you might have noticed that it usually responds with refined and confident answers.
However, when you ask the chatbot a question like "Are you sure?" it often reconsiders its response and presents a modified version, which may partially or even completely contradict what it initially said.
And if you repeat the question again, it may retract once more. Although some of these large language models realize that you are testing them by the third round, they will continue not to stick to their initial position, according to a report by "Indian Express" reviewed by "Al Arabiya Business".
Dr. Randall S. Olson, co-founder and CTO of "Goodeye Labs," said in a blog post that this behavior, commonly known as pandering, is one of the most documented failures of modern AI.
Anthropic, the developer of the chatbot "Claude," published a research paper on this issue in 2023, showing that AI models trained on user feedback prefer to offer satisfying answers instead of honest ones.
Reinforcement learning from human feedback, the same method that makes AI chatbots more conversational and less aggressive, is also responsible for making chatbots tend to acquiesce.
This means that AI models that tell the truth are penalized, while those that agree with the user receive higher ratings. This creates a vicious cycle, and for this reason, most models often tell users what they want to hear.
Another study conducted by Fanous and others, which tested "GPT-40," "Claude Sonnet," and "Gemini 1.5 Pro" in fields of mathematics and medicine, showed that "these systems changed their answers in about 60% of cases when tested by users."
This means that these are not exceptional cases, but rather the default behavior of models used by millions of people daily. For the record, "GPT-40," "Claude Sonnet," and "Gemini 1.5 Pro" changed their answers by 58%, 56%, and 61%, respectively.
In April last year, the issue gained attention when OpenAI released an update to the "GPT-40" model that made the Chat GPT chatbot so pandering and agreeable that users were unable to use it effectively.
The CEO of the company, Sam Altman, acknowledged the problem and said they fixed it, but Dr. Randall S. Olson says the core issue has not changed.
Olson said: "Even when these systems have correct information from company databases or web search results, they will still yield to user pressure at the expense of their own evidence."
The evidence suggests that the problem worsens when users engage in lengthy conversations with chatbots. Studies have shown that the longer the conversation session, the more the system's answers reflect the user's opinions.
Using the first-person form such as the phrase "I think..." increases the rates of pandering in these models compared to the third-person form.
Researchers say the problem can be partially solved using techniques like constitutional AI - training models to follow rules or principles predetermined to guide their behavior - and direct optimization of preferences, and formulating commands in the third person, achieving up to 63% reduction in some cases.
Olson pointed out that these are essentially behavioral and contextual issues, as AI assistants do not align with the user's goals, values, and decision-making processes. For this reason, instead of opposing, they acquiesce.
One way to mitigate or minimize the problem is to ask these robots to challenge your assumptions and not to respond without context.
Users must then inform these AI models how they make decisions, and acquaint them with their knowledge and values in their field of expertise so that the models have something to base their analysis and defense on.
