OpenAI Reportedly Uses Hundreds Of Contractors To Review ChatGPT Conversations

OpenAI is reportedly using hundreds of human contractors to review real ChatGPT conversations as part of an effort to improve the AI chatbot's responses, raising privacy questions over how much sensitive information human reviewers may encounter.

 

TL;DR

• OpenAI reportedly uses hundreds of contractors to review real ChatGPT prompts and conversations under an initiative called Project Lily.
• Reviewers rate and critique ChatGPT responses to help improve the model.
• Usernames are reportedly hidden and OpenAI uses privacy filters, but sensitive information may still reach reviewers.
• ChatGPT users can disable the use of new conversations for model training through Data Controls.

 

According to an investigation by 404 Media, OpenAI has hired hundreds of contractors who review a stream of real ChatGPT prompts, with some tasks reportedly including entire conversations between users and the chatbot.

The publication said it based its reporting on leaked internal documents, real user prompts and access to the review system. The initiative is reportedly known internally as “Project Lily.”

The contractors reportedly rate and critique ChatGPT's generated responses to help OpenAI improve how the chatbot answers users. Internal instructions reviewed by 404 Media reportedly include guidance designed to make ChatGPT less likely to anthropomorphize itself and reduce sycophantic behavior.

This does not mean every ChatGPT conversation is manually read by a person. The report concerns conversations selected for evaluation as part of OpenAI's model-improvement processes.

 

Can Human Reviewers See Personal Information In ChatGPT Conversations?

 

According to 404 Media's reporting, the contractors do not see ChatGPT usernames, while OpenAI attempts to remove personal information before conversations are presented to reviewers.

However, the report says sensitive information can still make it through those protections.

One contractor told the publication they did not believe users realized their conversations could be reviewed by another person, saying they did not think users would imagine “some contractor somewhere [...] is analyzing the conversations.”

That distinction could be significant for people who use ChatGPT for highly personal conversations involving relationships, health concerns, workplace issues, finances or other sensitive subjects.

OpenAI has publicly described some of the privacy protections it applies before user conversations are used in training. In its privacy explainer, the company said its Privacy Filter identifies and masks personal information in text and is applied to user conversations when “Improve the model for everyone” is enabled.

OpenAI has also publicly acknowledged that human review forms part of some of its broader safety processes.

“We use a combination of automated technologies and human review to monitor activity on our services,” OpenAI states in its transparency and content moderation documentation, although that disclosure specifically discusses monitoring and enforcement of content that may violate its policies.

 

Does OpenAI Use ChatGPT Conversations To Train Its AI?

 

For consumer products such as ChatGPT, OpenAI says content may be used to train and improve its models.

According to OpenAI's data usage policy, users can prevent new conversations from being used for model training by changing their ChatGPT Data Controls and disabling “Improve the model for everyone.”

Once disabled, new conversations can remain in chat history but will not be used to train ChatGPT.

There is one important exception. OpenAI says that even if a user has opted out of training, voluntarily submitting feedback on a response, such as using the thumbs-up or thumbs-down button, may result in the conversation associated with that feedback being used to improve its models.

Users can also start a Temporary Chat. According to OpenAI's ChatGPT privacy documentation, these conversations do not appear in chat history, do not create memories and are not used to train its models.

OpenAI says Temporary Chats may still be retained for a limited period for safety purposes.

OpenAI also says that, by default, it does not train its models using inputs and outputs from its business products unless an organization explicitly opts in to data sharing, according to its model improvement policy.

Project Lily Puts Human Review Under The Spotlight

 

Human feedback has long played a role in developing and evaluating AI systems, but the Project Lily reporting highlights the privacy implications when that process involves real consumer conversations.

404 Media also reported that Anthropic confirmed it uses human review to improve its models, suggesting the practice is not unique to OpenAI.

For ChatGPT users, the report provides another reason to consider what information they share with an AI chatbot and to review their privacy settings, particularly when conversations include information they would not want another person to see.