Mark Zuckerberg Goes to Bat for Open Source AI, Releases the Massive Llama 3.1 405B

Facebook and Meta founder and chief executive officer Mark Zuckerberg has come out in support of open source approaches to developing and releasing artificial intelligence (AI) systems, and to put his money where his mouth is has what he calls "the first frontier-level open source AI model" — though the company is hedging its bets by using a license that gives it the ability to nix deployments by entities exceeding more than a certain number of monthly active users.

"Today, several tech companies are developing leading closed models. But open source is quickly closing the gap," Zuckerberg claims, in a post which likens open source AI development to the development of the Linux kernel. "Last year, Llama 2 was only comparable to an older generation of models behind the frontier. This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry. But even before that, Llama is already leading on openness, modifiability, and cost efficiency."

Now, Zuckerberg and Meta's AI division have announced the release of Llama 3.1 405B, a 405-billion parameter "frontier-level" model — along with improved versions of the earlier, smaller Llama 3.1 70B and 8B models. "In addition to having significantly better cost/performance relative to closed models," the Facebook founder claims, "the fact that the 405B model is open will make it the best choice for fine-tuning and distilling smaller models."

This latter statement points to a shift in just how "open" an "open source" model is. Many supposedly open models include license terms that preclude their use to train or improve other AI models — but Zuckerberg is directly positioning Llama 3.1 405B, which requires a beefy system to run, as ideal for improving the capabilities of smaller, less resource-intensive models. "You'll be able to take the most advanced Llama models, continue training them with your own data and then distill them down to a model of your optimal size — without us or anyone else seeing your data," Zuckerberg promises.

"With past Llama models, Meta developed them for ourselves and then released them, but didn't focus much on building a broader ecosystem," Zuckerberg admits. "We're taking a different approach with this release. We're building teams internally to enable as many developers and partners as possible to use Llama, and we're actively building partnerships so that more companies in the ecosystem can offer unique functionality to their customers as well. I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source, and I expect that approach to only grow from here. I hope you'll join us on this journey to bring the benefits of AI to everyone in the world."

The new Llama models are available on the company's dedicated site, under the Llama 3.1 Community License Agreement license. While this allows for use, reproduction, distribution, modification, and derivation, it also includes a clause that applies only to commercial deployments by entities boasting more than 700 million monthly active users in the month preceding Llama 3.1's release — demanding that they "request a license from Meta, which Meta may grant to you in its sole discretion," giving the company a kill-switch for any service that could rise to challenge its own offerings.

Zuckerberg's full post, which does not mention this caveat, is available in the Meta newsroom.