Meta’s latest AI model wants to live on your PC

Most modern, capable AI models rely on cloud computing to give you fast and reliable responses — your prompt and associated data has to leave your device, head to a server, get processed by the model in use, and then make its way back to you as a response. But we’ve also seen models taking an on-device approach, one that minimizes concerns related to cloud-based AI, but at the same time introducing their own limitations. A new model from Meta takes the latter approach, and it does so in a way that tackles some of on-device AI’s main constraints head on.

With cloud-based AI, one of the biggest limitations is that models cannot run without an active internet connection. Models like Meta’s new Muse Glimmer that run totally on-device don’t face this limitation.

Then there’s the privacy concern with queries going to the cloud. You’re simply trusting the company behind the AI tool with your data every time you send in a request. This isn’t a concern when you opt for the on-device approach.

To be clear, Muse Glimmer isn’t the first model to break away from the cloud server approach. Google’s Gemini Nano and Gemma 4, Microsoft’s Phi-4-mini, and even Meta’s own Llama 3 can run locally. However, said models are extremely lightweight (at least when compared to Meta’s new model) and focus on simpler tasks. Muse Glimmer, in comparison, focuses specifically on agentic AI. That’s what makes its on-device existence so special.

Muse Glimmer itself isn’t necessarily lightweight. It is a 30-billion-parameter model. Gemini Nano 1, for comparison, has 1.8 billion parameters, while Nano 2 has 3.25 billion parameters. Nano 3 and Nano 4 go up to roughly 4 billion parameters.

The tech giant says that a model like Muse Glimmer would normally require over 55GB of memory. Meta gets around this memory barrier using 4-bit quantization, bringing the model down to under 20GB. “It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation,” wrote the company.

Here’s everything that the model can do

Muse Glimmer can write and debug code, resolve multi-turn commands from start to finish, and work through long tasks without losing track of the task or context. In case something goes wrong, the model is capable enough to “diagnose the error and retry rather than halt.”

Additionally, the model is trained on data from over 100 languages, and it can toggle between different reasoning strengths to hit a balance between quality and speed.

Oh, and it’s fast.

For those unfamiliar, regular AI models generate their responses one token at a time. This can result in a delayed response when the model has to perform a complicated task. With Muse Glimmer, Meta is taking a slightly different approach.

Using a workaround called speculative decoding, Muse Glimmer is able to leverage a lightweight “drafter” model based on DFlash, which is a technique used to speed up AI chatbots’ response by using prediction. The drafter model’s only job is to quickly predict what Muse Glimmer is likely to generate next.

Essentially, the drafter is able to propose several contextual tokens at once, which Muse Glimmer can then accept or reject. This results in a faster overall response time without compromising output quality.

Meta’s Muse Glimmer is available now on Hugging Face. Support for Ollama, LM Studio, Unsloth, llama.cpp, ExecuTorch, and MLX is on the way.