Two AI Safety Executives From OpenAI Quit, With One Joining Rival Anthropic

As the popularity and use cases of artificial intelligence (AI) and generative artificial intelligence (GenAI) grow, AI companies are looking to innovate with their offerings and its capabilities.

This led to OpenAI introducing its Superalignment endeavor, which was aimed at creating scientific and technical breakthroughs to steer and control AI systems much smarter than humans.

The idea was to solve the problem within four years and began by putting together a new team that would be co-led by Jan Leike and Ilya Sutskever and the inclusion of 20% of the compute that OpenAI secured to date.

However, less than a year into the effort, OpenAI faced major setbacks resulting from moves made by the projects’ co-leads.

So, what did the co-leads have to say? Let’s explore!
 

What Did Jan Leike And Ilya Sutskever Announce?

 

• Both co-leads of OpenAI’s Superalignment project, Jan Leike and Ilya Sutskever, announced their resignation from the company on the same day (May 15) through separate posts on X.

• While Sutskever’s post spoke highly of the company and didn’t mention details of the move (which he said would be shared “in due time”), Leike’s resignation simply read “I resigned”.

• However, Leike published a more detailed follow up post on May 17, which spoke about OpenAI not taking AI safety as important as it should have.

• As per Leike, ChatGPT was more focused on “shiny products” instead of “safety culture and processes.”

• This defeated the project’s purpose of ensuring superintelligence doesn't go rogue.

• Following these moves, the team tasked with this purpose was dissolved, with its members joining the core research team.

• On May 28, Jan Leike posted on X again, this time announcing he had joined OpenAI’s rivals, Anthropic, where he will continue working towards AI safety and even invited others to join him.

• Leike’s post read, “I'm excited to join Anthropic to continue the superalignment mission! My new team will work on scalable oversight, weak-to-strong generalization, and automated alignment research. If you're interested in joining, my DMs are open.”

• The move, which opened questions about OpenAI’s AI safety efforts, was addressed by the company through an announcement saying it had set up a safety committee, which would be overseen by board members and includes CEO Sam Altman.

• “This new committee is responsible for making recommendations on critical safety and security decisions for all OpenAI projects; recommendations in 90 days.” 

 

What Did Jan Leike Say?

 

• On May 15, 2024, through a post on X, Jan Leike announced his resignation, saying, “I resigned.”

• This was followed by a series of posts on X (May 17), which read, “Yesterday was my last day as head of alignment, superalignment lead, and executive at OpenAI.”

• “I joined because I thought OpenAI would be the best place in the world to do this research. However, I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until we finally reached a breaking point.”

• “I believe much more of our bandwidth should be spent getting ready for the next generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact, and related topics.”

• “These problems are quite hard to get right, and I am concerned we aren't on a trajectory to get there.”

• “Over the past few months my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.”

• “But over the past years, safety culture and processes have taken a backseat to shiny products.”

 

What Did Ilya Sutskever Say?

 

• On May 15, 2024, through a post on X, Ilya Sutskever announced his resignation, saying, “After almost a decade, I have made the decision to leave OpenAI.”

• [Contd.] “The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama (Sam Altman), @gdb (Greg Brockman), @miramurati (Mira Murati) and now, under the excellent research leadership of @merettm (Jakub Pachocki).”

• [Contd.] “It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.”

Do you think OpenAI needs to make moves soon to instill people’s confidence in its AI safety efforts? Do you think governments need to intervene with AI governance to provide better control for safety across AI technology and AI companies?

Let us know in the comments below!