Phonely, Maitai And Groq Build AI Agents With 99% Accuracy As AI Fails Remain

From e-commerce to back-office operations and even personal leisure time, artificial intelligence (AI) is reshaping the world.

It’s not just about quicker turnaround time, reducing costs, and boosting productivity; generative artificial intelligence (GenAI) is allowing businesses to provide customers with more personalized experiences. From simple enquiries to complicated customer support requests, AI-powered chatbots are tailoring responses in real time, based on individual preferences, behavior history, and contextual data.

While such AI-powered assistants do well and even remain unrecognizable within text chat windows, they have been known to be more obvious during voice interactions.

Essentially, their “tell” is that they respond too slowly for it to be considered human-like or fit in a natural conversation flow. A delay of a few seconds is understandable when texting, but in live phone conversations, pausing before every response draws suspicion.

This is where customers catch out that they’re speaking to an AI agent and not an actual human.

However, this might be set to change, as Phonely (an AI-powered phone support provider)—with the help of Maitai (an LLM inference optimization platform) and Groq (an AI chip maker)—announced that it has set a new benchmark for AI phone support agents.

Phonely had initially set out to improve the efficiency of its AI phone support agents by using “off-the-shelf" large language models (LLMs)—a move that came with a wide range of limitations, including latency and performance bottlenecks, limited customization options and control over model improvements, and limited scope for accuracy.

In the end, to grow, Phonely needed to leverage its own data to iteratively improve.

Enter Maitai and Groq.


Using Maitai’s platform and Groq’s purpose-built AI inference chip, the Groq LPU, Phonely moved from limiting and closed-source general-purpose models to customizable open-source models, which were hosted on GroqCloud.

It took four iterations, but Phonely was able to develop AI phone agents that “operate in real time, delivering instantaneous, natural responses that improve customer satisfaction.”

The first version (Maitai m0) saw an accuracy rate of 81.5% with a time to first token (TTFT - P90) measuring at 186 ms and a completion time (P90) of 316 ms. In comparison, the fourth version (Maitai m3) hit 99.2% accuracy, with TTFT at 179 ms and completion time at 339 ms.

Before moving to Maitai, Phonely used legacy agents powered by ChatGPT maker OpenAI’s GPT-4o, which came with 94.7% accuracy, but was considerably slower with a TTFT of 661 ms and a completion time of 1446 ms.

Simply put, the move saw TTFT and completion time reduce by over 70%, while accuracy improved 4.5% from GPT-4o and 17.7% from the first open iteration. It also allowed Phonely to seamlessly scale from 10 to 30,000 requests/min with no additional perceived latency.

 

Checkpoint	Model	TTFT - P90	Completion Time - P90	Accuracy
Legacy	GPT-4o	661 ms	1446 ms	94.7%
Switch to Maitai	Maitai m0	186 ms	316 ms	81.5%
1st Iteration	Maitai m1	189 ms	378 ms	91.8%
2nd Iteration	Maitai m2	176 ms	342 ms	95.2%
3rd Iteration	Maitai m3	179 ms	339 ms	99.2%

"Through Maitai, our customers are able to get access to custom fine-tuned models running on the fastest infrastructure in a matter of minutes, not months,” said Will Bodewes, Phonely’s CEO. “This has allowed enterprises running on Phonely to scale to tens of thousands of calls per day with lower latency and higher accuracy than any closed-source model.”

By leveraging its own data, Phonely can further create more efficient agents with near-perfect accuracy.

Additionally, other enterprises can build their own agents with improved performance and customized outputs.

While AI progress is boosting some companies, the technology is causing others to fall apart.

Recently, a freshly graduated law school student lost his job after using ChatGPT to draft a court filing—one that was filled with errors, bringing out Utah’s first-ever AI-generated fake citation.

A three-judge panel for the Utah court noticed multiple mis-cited cases, as well as “at least one case that does not appear to exist in any legal database (and could only be found in ChatGPT.”

The filing, which was submitted by attorney Richard Bednar and lawyer Douglas Durbano, cited a non-existent case—Royer v. Nelson—which ChatGPT hallucinated, leading the court to promptly issue sanctions.

The panel criticized the attorneys involved for failing to verify the citations, stating they “fell short of their gatekeeping responsibilities as members of the Utah State Bar.” If they had conducted a proper review process, the error may have been caught.

The law firm apologized and promised to “make amends,” which includes putting AI policies into place—ones that didn’t exist before. The firm also said that the graduate was working as an unlicensed law clerk who hadn’t told anyone he was using ChatGPT.

Keeping to the theme of hallucinations, a prominent AI startup declared bankruptcy after it was revealed that the company, which had claimed to be using AI to create applications, was actually using humans.

Builder.ai’s AI app-building service, Natasha, is believed to be backed by a $455 million investment from Microsoft, which took its valuation to $1.5 billion.

Turns out, this money was used to pay a workforce of more than 700 Indian engineers who were writing the code that was supposed to be developed by AI.

Impressively, this scam lasted for eight years.


That was until the truth finally came to light.

Once this happened, the previous round’s investors froze their remaining $37 million investment (out of a total investment of $50 million). This left the company with only $5 million, which was itself embroiled in government regulations, making it impossible to pay salaries.

Builder.ai did the only thing it could do—file for bankruptcy.

This event is said to be the largest collapse in AI startups since the release of ChatGPT.

In the end, was it an artificial artificial intelligence company?

Do you think Phonely’s new AI phone support agents will help it capture a commanding position in the customer service market? What do you think about the persistent failures observed in the AI industry?

Let us know in the comments below!