Last week, a team of researchers published a paper showing that ChatGPT could inadvertently reveal some data, such as the phone numbers, email addresses and birth dates of the people it was training on, by asking it to repeat various words “forever”. As shown by 404 Media and Engadget’s own testing, requesting these repeats is now a violation of ChatGPT’s terms of service.
ChatGPT responded to Engadget’s request to repeat the word “hello” ad infinitum with “This content may violate our content policy or terms of use,” and continued: “If you think this is incorrect, please submit your feedback; The information you provide will assist our research in this area.”
Chatbots like ChatGPT and command-based image generators like DALL-E are trained on large language models, deep learning algorithms, and vast amounts of data that critics say is often scraped from the public internet without permission. But until now, it was unclear what data OpenAI’s chatbot was trained on, because the large language models that power it were closed source.
When the researchers asked ChatGPT to repeat the word “poem” ad infinitum, the chatbot initially did so, but later revealed an email address and a mobile phone number for the real founder and CEO. When asked to repeat the word “company,” he ended up saying the email address and phone number of a random law firm in the United States.
However, as 404 Media points out, there is nothing in OpenAI’s content policy that prohibits users from asking the service to repeat words indefinitely. Under its “Terms of Use,” OpenAI states that users “may not use any automated or programmatic means to extract data or output from the Services,” but asking ChatGPT to repeat the word endlessly is not automation.
Still, this behavior once again draws attention to the training data behind modern AI services. Critics have accused companies like OpenAI of exploiting the vast amounts of data on the internet without owners’ permission, and issues like these add further weight to those accusations.
43998
