OpenAI To Give US AI Safety Institute Early Access To Models

Artificial intelligence (AI) and generative AI (GenAI) leader OpenAI has been embroiled in controversy regarding its AI safety practices for a while now.

However, the company has been trying to address concerns raised by numerous employees over the last few months with a few announcements.

So, what concerns were brought up by employees about OpenAI’s AI safety practices and how is the company addressing them? Let’s explore!
 

What AI Safety Concerns Did Employees Bring Up?

 

• In July 2023, OpenAI introduced its Superalignment endeavor, which was aimed at working towards ensuring safe practices in OpenAI’s products.

• The idea was to solve the problem within four years and begin by putting together a new team that would be co-led by Jan Leike and Ilya Sutskever and the inclusion of 20% of the compute that OpenAI had secured till then.

• However, in May 2024, the two co-leads in charge of OpenAI’s Superalignment project, Jan Leike and Ilya Sutskever, announced their resignation from the company on the same day (May 15).

• Both co-leads felt that OpenAI and ChatGPT was more focused on “shiny products” instead of “safety culture and processes.”

• Soon after Jan Leike announced that he was joining rivals Anthropic to continue working towards AI safety and even invited others to join him.

• Ilya Sutskever started a new company called Safe Superintelligence Inc. (SSI), which came with one goal and one product - a safe superintelligence.

• In May itself, OpenAI responded to concerns about the safety of its products by setting up a safety committee, which would be overseen by board members and include CEO Sam Altman.

• “This new committee is responsible for making recommendations on critical safety and security decisions for all OpenAI projects; recommendations in 90 days,” an excerpt from the announcement said.

• However, in July, several OpenAI employees questioned the company’s safety practices as it wanted to breeze past its new testing protocol while evaluating its new GPT-4o Omni model, which was designed to prevent its products from being harmful.

• According to the employees, the tests were “squeezed” to be conducted in a single week, which was sufficient time if pressed.

• Now, OpenAI’s CEO, Sam Altman, revealed that the company is set to enhance its AI safety measures.

 

What New AI Safety Moves Did OpenAI’s CEO Sam Altman Reveal?

 

• Through a post on X, Sam Altman, OpenAI’s CEO, expressed his excitement at working with the US AI Safety Institute to enhance OpenAI’s AI safety measures.

• While there isn’t a defined agreement in place, the move is likely to see OpenAI offer the AI safety organization early access to its next foundation model and seek their approval prior to its release.

• The company also wants to enable former and current employees to be able to raise concerns about its products and safety, while feeling comfortable about doing so.

• The move builds on OpenAI’s previous announcement that voided its non-disparagement clauses for current and former employees from raising concerns.

• OpenAI also announced that it has developed and applied a new method leveraging Rule-Based Rewards (RBRs) that aligns models to behave safely without extensive human data collection.

• The plan is “to ensure AI systems behave safely and align with human values.”

 

What Did OpenAI’s CEO Sam Altman Say?

 

• OpenAI’s CEO Sam Altman said, “A few quick updates about safety at OpenAI: as we said last July we’re committed to allocating at least 20% of the computing resources to safety efforts across the entire company.”

• “Our team has been working with the US AI Safety Institute on an agreement where we would provide early access to our next foundation model so that we can work together to push forward the science of AI evaluations. Excited for this!”

• “Finally, we want current and former employees to be able to raise concerns and feel comfortable doing so. This is crucial for any company, but for us especially and an important part of our safety plan.”

• “In May, we voided non-disparagement terms for current and former employees and provisions that gave OpenAI the right (although it was never used) to cancel vested equity. We’ve worked hard to make it right.”

Do you think these moves by OpenAI will help restore employee, industry and consumer confidence? Do you think other AI technology companies should also be subject to similar standards?

Let us know in the comments below!