What’s Known About The Powerful And Mysterious Chatbot That Left Users In Awe?

As artificial intelligence (AI) and generative artificial intelligence (GenAI) continues to impress the public as well as businesses with technology that can boost productivity, streamline processes and enhance workforce efficiency, AI companies are striving to build and release newer models with improved capabilities.

The marvelous and revolutionary technology that has brought in chatbots enabling users to simply ask questions to get answers, has gathered the interest of the biggest technology companies.

This includes names such as Google, Apple, Amazon, Oracle and Microsoft, which is a primary backer of ChatGPT, the world’s most popular chatbot.

Each iteration developed and deployed by these tech giants aims to outshine not just the previous one developed by the company but also those of rivals.

In the last week, one such chatbot suddenly popped up without anyone knowing whose it was.

However, users that tested the chatbot seemed impressed with its responses and even suggested it was better than anything else out there. That was until the chatbot left just as abruptly as it had come.

So, what was said about the chatbot and its capabilities?
 

What Happened With GPT2-chatbot?

 

• Over the weekend, users discovered a mysterious new AI chatbot called “gpt2-chatbot” on the popular large language model benchmarking site, LMSYS Org.

• As per reports about the chatbot, no one knows who developed it or where it came from.

• Users of the site and others speculate that the chatbot might be a secret test version of OpenAI’s upcoming GPT-4.5 or GPT-5 chatbot or a supercharged GPT-2, while some suggested it might even be a Google or Anthropic product.

• Through a post on X, Ethan Mollick, a professor researching artificial intelligence at the Wharton School of the University of Pennsylvania, said, “There is a mysterious new model called gpt2-chatbot accessible from a major LLM benchmarking site.”

• [Contd.] “No one knows who made it or what it is, but I have been playing with it a little and it appears to be in the same rough ability level as GPT-4. A mysterious GPT-4 class model? Neat!”

• The chatbot also came with a rate limit of eight queries per day, which limited detailed testing of the tool.

• While there was no confirmation or response from OpenAI about the chatbot, Sam Altman, the CEO of OpenAI, posted on X, saying, “I do have a soft spot for gpt2,” which could also be a reference to OpenAI’s previous version, GPT-2.

• Steven Heidel, a Member of Technical Staff at OpenAI, simply posted, “when gpt-2".

• On the other hand, Matt Shumer, the CEO of HyperWrite AI and Otherside AI, didn’t seem very impressed by the theory of it being the next iteration of OpenAI’s GPT-4.5 or GPT-5, posting on X, “gpt2-chatbot is good. Really good. But if this is gpt-4.5, I’m disappointed.”

• However, on Tuesday, the chatbot disappeared, with the LLM benchmarking site confirming that it was offline through a post on X, which read, ”Due to unexpectedly high traffic & capacity limit, we have to temporarily take gpt2-chatbot offline. Please stay-tuned for its broader releases.”

 

What Did Users Say About GPT2-chatbot?

 

• Florian S., a coder currently focusing on AI, asked the chatbot, “Today I have three apples. Yesterday I ate one. How many apples do I have?”

• While this comes across as a simple question, chatbots powered by GPT-4 Turbo, Claude-3 Opus, Gemini Ultra and Llama3 all provided incorrect answers.

• However, gpt2-chatbot provided a correct response, replying, “You have three apples today. The fact that you ate one apple yesterday does not affect the number of apples you have today. You currently have three apples.”

• “The mysterious gpt2-chat model really seems good. It got this question right,” posted Florian, along with a screenshot of the questions asked. This was followed by screenshots of other models providing incorrect answers to the same question.

 

• Pietro Schirano, the Founder of EverArt AI, said, “A mysterious new model called "gpt2-chatbot" has appeared on lmsys and it's really good. Not only does it seem to show incredible reasoning, but it also gets notoriously challenging AI questions right with a much more impressive tone. Judge for yourself.”

• Ahead of this, one user posted, “GPT2-Chatbot nearly built a flappy bird clone in one shot. It messed up initializing movement and didn't give actual assets,” another one said, “gpt2-chatbot just solved an International Math Olympiad (IMO) problem in one-shot.”

Which AI company do you think was behind the chatbot that got the online AI communities buzzing? Do you think this was a marketing tactic by one of the top players or do you think it was a genuine attempt to test the product?

Let us know in the comments below!