Each time chatbot makes something wrong, the first instinct of most users is to ask what happened to why who made a mistake. But with language models, unlike real people, this will not help. They simply cannot analyze their own mistakes. Information port arstechnica.com details speakWhy.
The first problem is the concept. When you talk to a chatbot, you do not communicate with some people or the permanent nature. Yes, chatgpt, Claude, Grok or Reprit technically named, but they are not worthy of knowledgeable – this is just an illusion created by the dialogue interface. In fact, you control the statistics of the text, providing results depending on the requirements of the user.
In other words, there is no chatgpt or Grok. A person interacts with a system that creates a complete audio text based on the samples found in the database (usually outdated). This system does not have its own self -understanding or system knowledge, as well as its ability to remember itself.
After studying a language model, it takes a lot of time and resources, its basic knowledge about the world is sealed in a nerve network and is rarely modified. Any external information comes from BOT requirements, user tools or software to use to find data in real time.
For example, if you take Grok, the main source of information for the answer may be contrary to each other from recent posts on social networks. Not knowledge, as in the case of everyone. And if this information is not enough, the model will simply give something due to the ability to predict text.
The number two – who cannot understand their own abilities for a number of reasons. As a rule, they do not have data on how they research, the surrounding system architecture is not available for them, they cannot determine the boundaries of their performance. If you ask chatbot what he can do and what is not, then he will give feedback on the basis of what they say about the limitations of the previous language models. In fact, he will simply answer with a conjecture, and not practical information about himself.
Research in 2024 proved this nuance with the help of an experiment. Although models II can be trained so they can predict their own behavior in simple tasks, they constantly fail in more complex tasks. In the same way, research on the recursive introduction of the Viking shows that there is no response from the outside, who tries to correct their mistakes leading to a reduction in the productivity of the model. That is, AI's self -esteem has worse, not better.
Sometimes this leads to paradoxical situations. The model can confidently say that it cannot complete the tasks but in fact, on her shoulder and vice versa, assure users that she can do what is impossible. We must not forget that, asking anyone about where he has made a mistake, users will receive another part of the text created and not a real analysis of the error.
The downside of the language models is that they do not have a stable and affordable knowledge basis that you can always contact. Their knowledge only expresses themselves to meet specific requirements; Each user's request plays the role of an address type that refers to the different parts of the database he is trained. This is exactly what leads to contradictory answers.
But even if II has an ideal knowledge about its own mechanisms, other layers of the BOT chat of applications may not be able to penetrate. Therefore, modern AISSTROS, such as TATGPT, is not a single model, but large systems including different models working together. Each of them, most, does not know about the existence of others. Therefore, asking anyone about errors like asking a part of a large company about what is happening in another part they have never contacted.
Finally, the most important point is that the user almost always affects the answer of anyone – does not matter whether they know or not. If you ask chatbot in horror, whether he has destroyed the records or code of the program, then he will tend to answer that yes, because this option is suitable for the emotional context of the request.
