Run LFM2.5-24B-A2B Locally on Windows 11, Mac and Kali Linux: Memory Needs, llama.cpp and Ollama Setup, Tool Calling and AWS

Logeshwaran
—

LFM2.5-24B-A2B is Liquid AI's new open model for agents that run on your own machine, released on October 13, 2026. You can run it locally on Windows 11, a Mac or Kali Linux today with llama.cpp, Ollama, LM Studio or, on a Mac, MLX. Here's the thing to understand before you download it: it has 24 billion parameters but uses only about 2.45 billion for each word it writes, so it answers at the speed of a small model, while it still needs the memory of a big one. The common 4-bit file is 14.53 GB, so plan on a machine with 24 GB of memory, or use a smaller file on 16 GB. Mac users have one trap to dodge first: mlx-lm versions before 0.32 load the model without complaint but route its experts incorrectly, so update before you judge it. This page covers the memory math, the exact commands for each system, the right settings, tool calling, the license, and renting a GPU on AWS for it.

Ethan, a developer, wants a small assistant for his friend Jake's phone repair shop: something that can read the repair tickets, look up parts and answer customers' "is my phone ready?" messages, all without sending the shop's data to anyone. Jake's laptop has 32 GB of memory and no big graphics card, which used to rule out anything this capable.

⚡ Quick Answer

• Will it run on my machine? → 24 GB of memory for the 14.53 GB 4-bit file; 16 GB with a 3-bit or 2-bit file. The memory table.

• Fastest setup: install llama.cpp, then llama serve -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M. Steps for Windows 11, Mac and Kali.

• Ollama: ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M, on Ollama 0.30 or newer. Details.

• Settings: temperature 0.1, top-k 50, repeat penalty 1.05. Why so low.

Free for commercial use below $10 million in annual revenue. Built for tools and agents, not for trivia.

New to running AI on your own machine? Our plain-English series on running AI locally explains models, parameters, GGUF files and quantization in about ten minutes. Every step on this page starts from zero, so you can follow it without that, too.

What LFM2.5-24B-A2B is, in plain English

Liquid AI is the company behind the LFM family, short for Liquid Foundation Models, which are built to run on phones, laptops and other small devices. LFM2.5-24B-A2B is the largest member of the LFM2.5 generation, and its name tells you its shape: 24B parameters in total, A2B for about 2 billion active per token (2.45 billion, to be exact).

It's a mixture of experts: inside each block are 64 small "expert" networks, and for every word only the 4 best-suited ones run. That's why it's fast. The other unusual part is how it pays attention. Only 10 of its 40 layers use classic attention; the other 30 are short convolution blocks, which is Liquid's way of keeping it quick on modest hardware.

Compared with the earlier LFM2-24B-A2B, this version:

  • reads up to 131,072 tokens of context, up from 32,768;
  • has a vocabulary of 128,000 tokens, up from 65,000, which handles non-Latin scripts better;
  • was pretrained on 33 trillion tokens instead of 17 trillion, then trained further with reinforcement learning inside agent tools;
  • works in 16 languages, including English, Hindi, Arabic, Chinese, Japanese, Korean, Thai and Indonesian;
  • always reasons before it answers. It writes its thinking first, between <think> tags, then the reply.

Jake: "So it's a big model that only uses a small part of itself at a time?"

Ethan: "Exactly. Like a shop with 64 technicians where each job only goes to the 4 who know that phone. Fast, but you still need room for all 64."

Will it run on my machine? The memory math

Here's the catch hiding in that speed. Every expert has to sit in memory, ready to be picked, even though only four of them work on each word. So the file size, not the active size, decides whether your machine can run it. These are the files you can download, with their real sizes:

File (GGUF)SizeWhere it fits
Liquid Q8_025.49 GB32 GB or more of memory, or a 32 GB graphics card
Liquid Q6_K19.69 GB32 GB machines with room to spare
Liquid Q5_K_M17.03 GB24 to 32 GB of memory
Liquid Q4_K_M14.53 GB24 GB of memory; the usual choice
Liquid QAD-Q4_013.58 GB24 GB; slightly smaller, trained to make up for 4-bit losses
Unsloth UD-Q3_K_XL10.95 GB16 GB of memory
Unsloth UD-IQ2_M7.54 GB16 GB with lots of room, or 12 GB graphics cards

Add memory for the conversation itself, the "context". This is where the convolution layers pay off: only the 10 attention layers keep a running record of the text, so the cost per token is small. The arithmetic, at the usual 16-bit setting: 10 layers × 8 key-value heads × 64 values × 2 (keys and values) × 2 bytes = about 20 KB per token. That's roughly 0.67 GB for a 32,000-token conversation and about 2.7 GB for the full 131,072, a quarter of what the same model would need if all 40 layers used attention.

So the Q4_K_M file plus a 32,000-token context needs a little over 15 GB, before your operating system and browser take their share. That's why 24 GB is the comfortable minimum for it, and why a 16 GB laptop should use a 3-bit or 2-bit file. Smaller files fit in less memory but trade some quality for it; start with the largest one that fits. For what each memory size buys you in general, see our honest guide to laptops for local AI.

How to run LFM2.5-24B-A2B on Windows 11

On Windows 11, llama.cpp is the most direct route, and it now has a one-line installer. Open PowerShell (not as administrator) and follow these steps:

  1. Install llama.cpp with irm https://llama.app/install.ps1 | iex, or with winget install llama.cpp if you prefer winget. Close and reopen PowerShell afterward so the new command is found.
  2. Start the model with the recommended settings. The first run downloads the 14.53 GB file from Hugging Face, so give it time:
    llama serve -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M --temp 0.1 --top-k 50 --repeat-penalty 1.05
  3. Open http://localhost:9931 in your browser for the built-in chat page; llama serve uses port 9931 by default, not the 8080 that older guides mention. The same address serves an OpenAI-compatible API at http://localhost:9931/v1 for apps and agents.
  4. For a quick test in the terminal instead, use llama cli in place of llama serve.

If you have an NVIDIA graphics card and the model runs slowly, the GPU may not be in use; our guide to local AI falling back to the CPU covers the usual causes. Prefer clicking to typing? With LM Studio installed, paste lmstudio://open_from_hf?model=LiquidAI/LFM2.5-24B-A2B-GGUF into your browser's address bar, and it opens the model's download page inside LM Studio, where you pick the Q4_K_M file.

How to run LFM2.5-24B-A2B on a Mac (and the MLX trap)

On an Apple Silicon Mac you have two good routes. llama.cpp works the same as on Windows: install it with brew install llama.cpp or the installer script, then run the same llama serve command. The Mac's unified memory is shared between the processor and the graphics, so a 24 GB Mac counts as 24 GB for the table above.

The other route is MLX, Apple's own machine-learning framework, which Liquid publishes ready-made versions for, from 4-bit (13.6 GB) to full precision. This is where the trap is. You need mlx-lm 0.32.0 or later: earlier versions load the model but send each word to the wrong experts. Nothing crashes; the answers are just worse than they should be. So upgrade first, then chat:

pip install -U "mlx-lm>=0.32.0"
mlx_lm.chat --model LiquidAI/LFM2.5-24B-A2B-MLX-4bit

For an API your apps can call, use mlx_lm.server --model LiquidAI/LFM2.5-24B-A2B-MLX-4bit instead. On speed, an Apple M5 Max reaches about 184 tokens per second with llama.cpp and the Q4_K_M file; smaller Macs are slower, but this is still one of the quickest models of its size you can run on one.

Jake: "So on a Mac the risky part is an old version, not the model?"

Ethan: "Right. An old version that fails loudly would be fine. This one fails politely, and you'd blame the model."

How to install LFM2.5-24B-A2B on Kali Linux

On Kali Linux, building llama.cpp from source takes a few minutes and gives you the newest version, which matters for brand-new models. These are the steps, on a current Kali Rolling:

sudo apt update
sudo apt install -y git cmake build-essential libssl-dev
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli

Then start the model, which downloads it on the first run:

./build/bin/llama-server -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M --temp 0.1 --top-k 50 --repeat-penalty 1.05

The libssl-dev package gives llama.cpp the secure connection it needs for -hf to download from Hugging Face; without it, you'd have to download the file yourself and pass it with -m. If you'd rather not compile, the installer script also works on Kali: curl -LsSf https://llama.app/install.sh | sh, then llama serve as on Windows. Our Gemma 4 on Kali guide covers running a model fully offline, if your Kali machine has no internet.

Running it in Ollama or LM Studio

Ollama's own library has the smaller LFM2.5 8B, not the 24B yet. You can still run the 24B in Ollama by pulling the file straight from Hugging Face:

ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M

You need Ollama 0.30 or newer, the minimum that Ollama's own LFM2.5 models, built on the same design, require; check yours with ollama --version. To make Liquid's settings the default instead of typing them each time, save these four lines as a file named Modelfile:

FROM hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M
PARAMETER temperature 0.1
PARAMETER top_k 50
PARAMETER repeat_penalty 1.05

Then run ollama create lfm25-24b -f Modelfile once, and ollama run lfm25-24b from then on. If you're choosing between the runners, our Ollama vs LM Studio vs llama.cpp comparison lays out the trade-offs.

The right settings, and why the temperature is so low

Most chat models are run at a temperature around 0.7. This one runs best at 0.1, with top-k 50 and a repeat penalty of 1.05. A low temperature makes the model pick its most likely next word almost every time, which is what you want from an agent: the same tool call for the same request, not a creative variation of it.

SettingValuellama.cppOllama Modelfile
Temperature0.1--temp 0.1PARAMETER temperature 0.1
Top-k50--top-k 50PARAMETER top_k 50
Repeat penalty1.05--repeat-penalty 1.05PARAMETER repeat_penalty 1.05

Two more things to expect. The model always thinks before answering, so every reply starts with a reasoning section; give it room by not setting a tiny maximum reply length. And most of the published scores were measured at a temperature of 0.6, so if you're checking it against them, that's the setting to use; for daily work, stick with 0.1.

Tool calling: what it's built for

Tool calling is the headline skill here. You describe the tools, such as "look up a repair ticket" or "check stock", in the system message as a JSON list, or with the tools= option of your chat template, and the model decides when to call one. By default it writes the call as a short Python-style function call between two special markers:

<|tool_call_start|>[get_ticket_status(ticket_id="R-1042")]<|tool_call_end|>

Your program runs the tool, sends the result back in a message with the tool role, and the model writes the answer in plain text. If your software expects JSON tool calls instead, ask for JSON in the system message; the model can produce either. When you run it with llama serve or Ollama, agent tools that speak the OpenAI API can drive it directly through http://localhost:9931/v1 (or port 8080 if you run llama-server from a source build); our guide to OpenCode with a local model shows one such setup.

Jake: "So it doesn't know my stock. It asks the stock list?"

Ethan: "That's the whole idea. It's good at asking the right tool the right question. It's not meant to remember facts on its own."

LFM2.5-24B-A2B benchmarks: where it wins and where it doesn't

Here it is against three other mixture-of-experts models of similar size. The pattern is clear, and honest: it leads on tool use and on following instructions, and it trails on knowledge and on hard coding.

BenchmarkLFM2.5-24B-A2BQwen3.6-35B-A3BGemma 4 26B-A4B
BFCLv4 (function calling)65.0165.0057.09
ToolSandbox80.7175.0272.09
IFEval (instruction following)96.7593.0692.00
IFBench74.6964.2147.44
MATH-50096.0496.7395.66
GPQA Diamond (science knowledge)74.4283.6974.90
SWE-Bench Verified (coding)51.0066.2025.80

Larger models keep an edge on knowledge-heavy tests, and this one is designed to be paired with retrieval or web search for that kind of work. Take that seriously: use it to run tools and follow instructions, and give it your documents or a search tool when the answer depends on facts. For a coding assistant on the same hardware, Qwen3.6-35B-A3B scores higher, at the cost of more memory. For a decision-making model from the same lab, see our page on Liquid's d1.

The license: free below $10 million in revenue

The weights come under the LFM Open License v1.0, not a standard open-source license. In plain terms:

  • Personal use, research and commercial use are allowed, as long as you or your company make less than $10 million a year in revenue.
  • Above that, commercial use isn't covered by this license at all.
  • Qualified non-profits aren't bound by the revenue limit for non-commercial and research use.
  • If you share the model or a modified version, include the license and mark the files you changed.

For Jake's shop, that means free use. If you'd rather not host it at all, Liquid also sells access through its own API, at $0.10 per million input tokens and $1.00 per million output tokens.

LFM2.5-24B-A2B on AWS: renting a GPU for it

Sometimes the right machine is one you rent for an afternoon. Liquid's models aren't in Amazon Bedrock's catalog, so on AWS this is a self-hosting job; our Bedrock vs SageMaker AI explainer sorts out the services if the names are new. The simplest setup is a single-GPU EC2 instance running llama.cpp exactly as on your laptop, switched off when you're done.

On-demand Linux prices in US East (N. Virginia), from the EC2 price list of October 1, 2026, for single-GPU instances that fit the Q4_K_M file with a long context:

InstanceGPUPer hour
g6.xlarge1x NVIDIA L4, 24 GB$0.8048
g5.xlarge1x NVIDIA A10G, 24 GB$1.006
g6e.xlarge1x NVIDIA L40S, 48 GB (room for Q8_0)$1.861

An evening of testing on a g6.xlarge costs about the price of a coffee; a month of round-the-clock uptime is about $590 plus storage, more than a used graphics card. Rent to test and for bursts; buy for an assistant that runs all day. llama.cpp's server listens only on the machine itself by default, which is what you want: reach it through an SSH tunnel or from inside your VPC, and don't open its port to the internet, because it has no sign-in unless you set an API key. Our guide to Hugging Face models on AWS walks through the instance setup.

Before you change anything, here's the order that gets most people running on the first try:

  1. Check your memory against the table, and pick the largest file that fits.
  2. Install or update the runner: llama.cpp from the installer or source, Ollama 0.30 or newer, mlx-lm 0.32 or newer.
  3. Start it with temperature 0.1, top-k 50 and repeat penalty 1.05.
  4. Test a plain question, then a tool call, before you connect an agent to it.

LFM2.5-24B-A2B not working? Troubleshooting by symptom

Answers are oddly bad on a Mac with MLX

Your mlx-lm is older than 0.32.0, which routes the experts wrongly. Run pip install -U "mlx-lm>=0.32.0".

Ollama can't load the model

Update Ollama to 0.30 or newer; older versions don't know this model family.

The download starts every time, or -hf isn't recognized

Your llama.cpp is old or was built without download support. Run llama update if you used the installer, or rebuild with libssl-dev installed.

It's very slow on a PC with a graphics card

The model may be running on the CPU. Check that your llama.cpp build or Ollama uses the GPU.

Out of memory while loading

The file plus context doesn't fit. Use a smaller file from the table, or a shorter context with -c.

Every reply starts with a long reasoning section

That's by design: it always thinks first. Hide the think section in your app; don't cut the reply length short.

LFM2.5-24B-A2B: frequently asked questions

What is LFM2.5-24B-A2B?

Liquid AI's open mixture-of-experts model with 24B parameters and about 2.45B active per token, released October 13, 2026.

How do I run LFM2.5-24B-A2B locally?

Install llama.cpp and run llama serve -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M, or use Ollama, LM Studio or MLX.

How much RAM does LFM2.5-24B-A2B need?

About 24 GB for the 14.53 GB 4-bit file with room for context. 16 GB machines should use a 3-bit or 2-bit file.

Can I run LFM2.5-24B-A2B on Windows 11?

Yes. Install llama.cpp with the PowerShell installer or winget, then run llama serve with the GGUF.

Can I run LFM2.5-24B-A2B on a Mac?

Yes, with llama.cpp or MLX. For MLX, use mlx-lm 0.32.0 or later.

How do I install LFM2.5-24B-A2B on Kali Linux?

Build llama.cpp from source with git, cmake, build-essential and libssl-dev, then run llama-server -hf.

Is LFM2.5-24B-A2B in Ollama?

Not in Ollama's library yet. Run ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M on Ollama 0.30 or newer.

What settings should I use for LFM2.5-24B-A2B?

Temperature 0.1, top-k 50 and repeat penalty 1.05.

What context length does LFM2.5-24B-A2B support?

131,072 tokens, up from 32,768 in LFM2-24B-A2B.

Is LFM2.5-24B-A2B free for commercial use?

Yes, under the LFM Open License v1.0, if your annual revenue is below $10 million.

What is the QAD Q4_0 file?

A 13.58 GB 4-bit file from a checkpoint trained to recover the accuracy that Q4_0 quantization loses.

Is LFM2.5-24B-A2B good for coding?

It's decent, with 51.0 on SWE-Bench Verified, but Qwen3.6-35B-A3B scores 66.2. It's strongest at tool use.

Does LFM2.5-24B-A2B support tool calling?

Yes. It writes Python-style calls between tool-call markers by default, and JSON when asked.

Why does LFM2.5-24B-A2B always think before answering?

It's a reasoning-only model: every reply starts with its reasoning, then the answer.

Is LFM2.5-24B-A2B on Amazon Bedrock?

No. On AWS, run it yourself on an EC2 GPU instance such as a g6.xlarge with llama.cpp.

What languages does LFM2.5-24B-A2B support?

16, including English, Hindi, Arabic, Chinese, Japanese, Korean, Spanish, French and German.

What's the difference between LFM2.5-24B-A2B and LFM2-24B-A2B?

A 4x longer context, double the vocabulary, about twice the pretraining data, and agent-focused reinforcement learning.

For readers preparing for an AWS certification

We have been covering AWS certifications alongside our fixes, so here is a short extra for readers studying for one. If you only came for the fix, you already have it; feel free to skip this part.

Exams that test this topic: AI Practitioner (AIF-C01).

Here are the exam-style questions this topic tends to raise, with the reasoning you need to answer them:

1. What does a mixture-of-experts model do differently?

It activates only some expert sub-networks per token, so it runs faster than its total size suggests.

2. Why does a model with few active parameters still need lots of memory?

All weights must be loaded, even though only some are used per token.

3. What is the difference between training and inference cost?

Training builds the model once at great expense; inference cost grows with every request.

4. Which AWS chip is designed for low-cost inference?

AWS Inferentia.

5. What is tokenization?

Splitting text into tokens, the units models read and are billed by.

6. Why might a company run small models on edge devices?

For privacy, offline use and low latency.

Exam codes and dates change often. Our AWS certification changes page keeps track of what is current.

Ethan put the Q4_K_M file on Jake's 32 GB laptop with llama serve, gave it two tools, one for the ticket list and one for the parts sheet, and asked it whether the phone in ticket R-1042 was ready. It looked the ticket up, checked the part, and answered in one go. Jake's verdict: "So it's quick because it only wakes up four of its technicians." Ethan said that was the best summary of a mixture of experts he'd heard.

📌 If you keep one line from this page

LFM2.5-24B-A2B runs at the speed of a 2B model but needs the memory of a 24B one: 24 GB for the 4-bit file, temperature 0.1, and mlx-lm 0.32 or newer on a Mac.

llama serve -hf LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M · Ollama 0.30+ · temp 0.1, top-k 50, repeat 1.05 · free below $10M revenue.

Revision note. Written October 14, 2026, a day after LFM2.5-24B-A2B's release, for llama.cpp, Ollama 0.30 and later, LM Studio and mlx-lm 0.32 and later. "Many hands make light work," says the old proverb; here, four hands at a time make it fast.

Related