Meta Unveils Llama 4 Herd With Native Multimodal AI Models

Around a month ago, Meta’s open-source AI model, Llama, hit a major milestone of one billion downloads. It was more than just a magic number; it highlighted the increasing impact of open-source access in the ever-growing field of artificial intelligence (AI).

By making powerful models freely available, Meta sparked innovation across industries and inspired a global community of tech giants, from Spotify to nimble startups like Fynopsis.

Now, with the release of Llama 4 Herd, Meta is building on its momentum and pushing the boundaries further than ever.

What’s New In The Llama 4 Lineup?

The initial offerings within this Llama 4 family are Llama 4 Scout and Llama 4 Maverick. These models are designed to enable the creation of more personalized and intuitive digital experiences.


Llama 4 Scout, a model with 17 billion active parameters and a structure incorporating 16 "experts," is being positioned as a leading multimodal model in its size category. Notably, it is engineered to operate efficiently on a single high-performance NVIDIA H100 graphics processing unit. Furthermore, Llama 4 Scout boasts an exceptionally large "context window" of 10 million tokens.

This capability allows the model to process and understand significantly longer sequences of information, reportedly leading to better performance than previous Llama models and outperforming other contemporary models like Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 across a range of standard evaluations.

The second model released is Llama 4 Maverick, which also features 17 billion active parameters but with a more extensive network of 128 experts. This model is also being presented as a top-tier multimodal model in its class, purportedly surpassing the performance of GPT-4o and Gemini 2.0 Flash on various widely used benchmarks.

Interestingly, Llama 4 Maverick is said to achieve comparable results to the newer and larger DeepSeek v3 on tasks requiring reasoning and coding abilities while utilizing less than half the active parameters. This suggests a notable improvement in performance and efficiency. An experimental chat version of Llama 4 Maverick has also achieved a high score of 1417 on the LMArena evaluation platform, indicating strong conversational capabilities.

The advancements in Llama 4 Scout and Maverick are attributed to a process called "distillation" from an even larger and more powerful model, Llama 4 Behemoth. This "teacher" model has 288 billion active parameters and 16 experts, making it one of the most sophisticated language models developed so far.

Even in its ongoing training phase, Llama 4 Behemoth has reportedly demonstrated superior performance compared to models like GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on specific science and mathematics benchmarks. The insights gained from training Behemoth have been used to enhance the capabilities of the smaller Llama 4 models.

What Makes Llama 4 So Smart?

A standout architectural innovation in the Llama 4 series is the use of a "mixture-of-experts" (MoE) design. In MoE models, only a subset of the total parameters is activated for each task when processing information. This approach is computationally efficient for both training and inference. For instance, Llama 4 Maverick includes a huge 400 billion parameters, but only 17 billion are in use at any given time during operation.

Additionally, the Llama 4 models are built with "native multimodality," which allows them to understand and process both text and visual information from the outset. This ability is gained through a technique called "early fusion," which lets the models get trained on large datasets containing text, images, and videos at the same time.

How Did Meta Train These Models To Be Better?

The development process of Llama models included advanced training techniques, involving "MetaP" for setting crucial model parameters and the use of FP8 precision for efficient computation. The models were trained on a vast dataset of over 30 trillion tokens spanning 200 languages, marking a substantial expansion in multilingual capabilities compared to Llama 3.

Post-training filtering was a prime element to enhancing the performance of Llama 4 Scout and Maverick for a wide range of applications. For Llama 4 Maverick, this included a precisely designed sequence of supervised fine-tuning and reinforcement learning techniques to balance its understanding of different types of data, reasoning abilities, and conversational skills.


Llama 4 Scout, on the other hand, was specifically trained to handle very long sequences of information, achieving a 10 million token context window through innovations in its attention mechanisms.

The developers emphasize their commitment to open-source principles, making Llama 4 Scout and Llama 4 Maverick available for download on platforms like llama.com and Hugging Face. They also highlight the integration of these models into their own products, such as Meta AI on WhatsApp, Messenger, Instagram Direct, and the Meta.AI website.

The launch of the Llama 4 herd marks a major step forward in creating more intelligent and adaptable AI systems. Meta’s developers believe that these models will empower developers and businesses to build a variety of innovative applications. They also emphasize their continuous efforts in research and development while inviting the community to learn more about their vision at the upcoming LlamaCon event.

What About AI Safety And Responsibility?

In conjunction with the model releases, the developers have also outlined their approach to safety and responsible AI development. This includes data filtering during training, post-training techniques to align models with desired behaviors, and the open-sourcing of tools like Llama Guard and Prompt Guard to help developers identify and mitigate potential risks.

They also detailed their efforts in evaluating and addressing bias in the models, reporting improvements in reducing refusals on sensitive topics and achieving a more balanced response across different viewpoints.

The launch of the Llama 4 herd represents a notable advancement in open-source AI, offering powerful multimodal capabilities and extended context understanding. As these models become readily available, the AI community will be watching closely to see the innovative applications and experiences they enable.

Do you think Llama 4 will change the way we work with AI?

Share your thoughts in the comments below.