ChatGPT, the chat bot that OpenAI launched in November 2022, managed to go viral all over the world with what it could do. ChatGPT has started to be used in many mobile applications today, especially as an actor in customer representation.
While we assume that ChatGPT improves with each update, a study published on July 18 revealed interesting results. Scientists from Stanford and Universities of California explained that ChatGPT started to give worse answers with each new language model.
Higher failure rate on latest GPT models:
The research team members Lingjiao Chen, Matei Zaharia, and James Zou put two different versions and two versions of GPT, the language model that created ChatGPT, to the test: GPT-3.5 March/June and GPT-4 March/June.
In the test, the language model was asked several math problems and sensitive questions. Asked to write code and reason about input. Especially the difference between the versions was quite surprising.
The March release of GPT-4 was able to know with 97.6% accuracy whether a number presented to it was a prime number. In the June version of the same version, this rate was limited to only 2.4%.
In prime numbers, the success of GPT-3.5 was even better than GPT-4 March. GPT-3.5 March release knew prime numbers with higher accuracy in the same time period. Similar decreases experienced as it developed showed itself in coding.
When it came to sensitive questions, a strange picture emerged. In March releases, the bot detailed the user why it couldn’t answer a question. In the June versions, he just apologizes to the user and says he can’t answer the question.
The research team was also unable to determine the reason behind the decline in ChatGPT’s capabilities.
44163
