Meta launches Llama 3.3, shrinking powerful 405B open model

Meta’s VP of generative AI, Ahmad Al-Dahle took to rival social network X today to announce the release of Llama 3.3, the latest open-source multilingual large language model (LLM) from the parent company of Facebook, Instagram, WhatsApp and Quest VR.

As he wrote: “Llama 3.3 improves core performance at a significantly lower cost, making it even more accessible to the entire open-source community.”

With 70 billion parameters — or settings governing the model’s behavior — Llama 3.3 delivers results on par with Meta’s 405B parameter model from the Llama 3.1 from the summer, but at a fraction of the cost and computational overhead — e.g., the GPU capacity needed to run the model in an inference.

It’s designed to offer top-tier performance and accessibility yet in a smaller package than prior foundation models.

Meta’s Llama 3.3 is offered under the Llama 3.3 Community License Agreement, which grants a non-exclusive, royalty-free license for use, reproduction, distribution, and modification of the model and its outputs. Developers integrating Llama 3.3 into products or services must include appropriate attribution, such as “Built with Llama,” and adhere to an Acceptable Use Policy that prohibits activities like generating harmful content, violating laws, or enabling cyberattacks. While the license is generally free, organizations with over 700 million monthly active users must obtain a commercial license directly from Meta.

A statement from the AI at Meta team underscores this vision: “Llama 3.3 delivers leading performance and quality across text-based use cases at a fraction of the inference cost.”

How much savings are we talkin’ about, really? Some back-of-the-envelope math:

Llama 3.1-405B requires between 243 GB and 1944 GB of GPU memory, according to the Substratus blog (for the open source cross cloud substrate). Meanwhile, the older Llama 2-70B requires between 42-168 GB of GPU memory, according to the same blog, though same have claimed as low as 4 GB, or as Exo Labs has shown, a few Mac computers with M4 chips and no discrete GPUs.

Therefore, if the GPU savings for lower-parameter models holds up in this case, those looking to deploy Meta’s most powerful open source Llama models can expect to save up to nearly 1940 GB worth of GPU memory, or potentially, 24 times reduced GPU load for a standard 80 GB Nvidia H100 GPU.

At an estimated $25,000 per H100 GPU, that’s up to $600,000 in up-front GPU cost savings, potentially — not to mention the continuous power costs.

A highly performant model in a small form factor

According to Meta AI on X, the Llama 3.3 model handedly outperforms the identically sized Llama 3.1-70B as well as Amazon’s new Nova Pro model in several benchmarks such as multilingual dialogue, reasoning, and other advanced natural language processing (NLP) tasks (Nova outperforms it in HumanEval coding tasks).

Llama 3.3 has been pretrained on 15 trillion tokens from “publicly available” data and fine-tuned on over 25 million synthetically generated examples, according to the information Meta provided in the “model card” posted on its website.

Leveraging 39.3 million GPU hours on H100-80GB hardware, the model’s development underscores Meta’s commitment to energy efficiency and sustainability.

Llama 3.3 leads in multilingual reasoning tasks with a 91.1% accuracy rate on MGSM, demonstrating its effectiveness in supporting languages such as German, French, Italian, Hindi, Portuguese, Spanish, and Thai, in addition to English.

Cost-effective and environmentally conscious

Llama 3.3 is specifically optimized for cost-effective inference, with token generation costs as low as $0.01 per million tokens.

This makes the model highly competitive against industry counterparts like GPT-4 and Claude 3.5, with greater affordability for developers seeking to deploy sophisticated AI solutions.

Meta has also emphasized the environmental responsibility of this release. Despite its intensive training process, the company leveraged renewable energy to offset greenhouse gas emissions, resulting in net-zero emissions for the training phase. Location-based emissions totaled 11,390 tons of CO2-equivalent, but Meta’s renewable energy initiatives ensured sustainability.

Advanced features and deployment options

The model introduces several enhancements, including a longer context window of 128k tokens (comparable to GPT-4o, about 400 pages of book text), making it suitable for long-form content generation and other advanced use cases.

Its architecture incorporates Grouped Query Attention (GQA), improving scalability and performance during inference.

Designed to align with user preferences for safety and helpfulness, Llama 3.3 uses reinforcement learning with human feedback (RLHF) and supervised fine-tuning (SFT). This alignment ensures robust refusals to inappropriate prompts and an assistant-like behavior optimized for real-world applications.

Llama 3.3 is already available for download through Meta, Hugging Face, GitHub, and other platforms, with integration options for researchers and developers. Meta is also offering resources like Llama Guard 3 and Prompt Guard to help users deploy the model safely and responsibly.