TechDogs-"Neural Magic Delivers An The Promise Of Software-Delivered AI With Latest MLPerf Inference v3.0 Results"

Software Development

Neural Magic Delivers An The Promise Of Software-Delivered AI With Latest MLPerf Inference v3.0 Results

By Business Wire

Business Wire
Overall Rating

1,000X increase in CPU performance means lower computational costs for more affordable deep learning

BOSTON--(BUSINESS WIRE)--#AI--Neural Magic, AI leader in sparse inferencing technology, today announced a 6X improvement over their groundbreaking results from their last round submission with open engineering consortium, MLCommons®. Neural Magic’s latest results in MLPerf™ Inference v3.0, validate that customers can achieve high-performance AI using just x86 CPU architectures.

“We are so excited about our latest performance results, validated by MLPerf Inference v3.0,” said Brian Stevens, Chief Executive Officer, Neural Magic. “By applying our specialized compound sparsity algorithms with our patented DeepSparse inference runtime, we were able to achieve a boost in CPU performance by 1,000X while reducing power consumption by 92% over other inference solutions. The numbers prove users can have GPU speeds to support AI projects with just software and off-the-shelf processors.”

This year, Neural Magic used 4th Gen AMD EPYC™ processors for their benchmark testing. Neural Magic’s software stack takes advantage of continued innovations in AMD EPYC processors, such as AVX-512 and VNNI instructions as well as advanced features like highly performant DDR5 memory and a core count up to 96 cores, to unlock new possibilities for delivering better than GPU speeds on x86 CPUs.

“Neural Magic’s MLPerf™ Inference v3.0 results, benchmarked using our latest 4th Gen AMD EPYC processors, prove customers can achieve outstanding levels of AI inference performance for deep learning projects on x86 based CPUs, ” said Kumaran Siva, Corporate Vice President, Strategic Business Development, AMD.

With Neural Magic, data scientists can achieve breakthrough performance with their deep learning models, while lowering computational expenses and simplifying operations. Read this blog for more details on Neural Magic’s MLPerf Inference v3.0 results.

About Neural Magic

Neural Magic is an AI company, born out of the Massachusetts Institute of Technology (MIT), on a mission to help customers innovate with machine learning, without added complexity or cost. While pursuing research at MIT, founders Nir Shavit and Alexander Matveev launched Neural Magic, a software-delivered AI solution, to address their frustration with the constraints of GPUs and existing hardware. Using Neural Magic’s DeepSparse Inference Runtime, customers can easily deploy deep learning models on commodity CPUs with GPU-class performance. For more information, including all of Neural Magic’s offerings, visit or follow @neuralmagic on Twitter, LinkedIn, and YouTube.

AMD, the AMD Arrow logo, EPYC, and combinations thereof, are trademarks of Advanced Micro Devices, Inc. Other names are for informational purposes only and may be trademarks of their respective owners.


Nicole Kim
Head of Product Marketing and Communications
(415) 902-8732

First published on Thu, Apr 6, 2023

Liked what you read? That’s only the tip of the tech iceberg!

Explore our vast collection of tech articles including introductory guides, product reviews, trends and more, stay up to date with the latest news, relish thought-provoking interviews and the hottest AI blogs, and tickle your funny bone with hilarious tech memes!

Plus, get access to branded insights from industry-leading global brands through informative white papers, engaging case studies, in-depth reports, enlightening videos and exciting events and webinars.

Dive into TechDogs' treasure trove today and Know Your World of technology like never before!

Disclaimer - Reference to any specific product, software or entity does not constitute an endorsement or recommendation by TechDogs nor should any data or content published be relied upon. The views expressed by TechDogs’ members and guests are their own and their appearance on our site does not imply an endorsement of them or any entity they represent. Views and opinions expressed by TechDogs’ Authors are those of the Authors and do not necessarily reflect the view of TechDogs or any of its officials. All information / content found on TechDogs’ site may not necessarily be reviewed by individuals with the expertise to validate its completeness, accuracy and reliability.


Neural Magic Massachusetts Institute Of Technology (MIT) software-delivered AI solution MLPerf™ Inference v3.0

Join The Discussion

- Promoted By TechDogs -

Are You Ready To Accelerate Your Cloud Migration And Data Modernization?