LFM3-1.7B Locally on Windows 11 and Kali Linux: Ollama vs llama.cpp vs Transformers, Real Download Sizes, Tool Calling, and a Local OpenAI-Compatible Server

Jake runs a phone repair shop, and his week is a wall of tiny writing tasks: repair estimates sent over text, replies to "will my photos survive the reset," a summary of what walked in the door. He wanted an assistant that could handle that without his customer conversations leaving the building, and without depending on the shop's Wi-Fi, which folds whenever the mall gets busy. So he did what he always does: he called Ethan, the friend who spends his days architecting cloud systems, and asked which API to subscribe to.

Ethan, for once, told him to buy nothing. Late October 2026, Liquid AI released LFM3-1.7B, a text model the company built the opposite way from the giants: small enough to live on a laptop, trained specifically to drive local agents and tool calling rather than to win trivia contests. Ethan's pitch was blunt: a 1.73B-parameter model that carries a 256K-token context window, runs in about a gigabyte of memory at 4-bit precision, and downloads in roughly the space of a phone app update. Jake's counter PC runs Windows 11; the battered laptop on the back bench, the one used for device diagnostics, runs Kali Linux. Same model, two very different machines.

This tutorial is the full walkthrough of that local install on both systems, three different ways: Ollama for the fastest start, llama.cpp for a local web UI and an OpenAI-compatible server, and Python with Transformers for tool calling and customization. It is part of the run-AI-locally guide series, and every download size, file name, and command in it is the real thing, so you can plan your disk space before you touch a terminal.

Here is the part nobody believes until the numbers show up: this 1.7B model outscores models five times its size on the agent benchmarks that matter. On LiveClaw it scores 45.60 against 35.77 for Qwen3.5-9B. On WebArena-Verified it scores 38.42 against 28.94. Liquid AI's own line on it: LFM3-1.7B can power agentic applications that used to need 9B parameters. Jake stared at the 1.1 GB file size and said the last system update on his demo phone was bigger than the model.

By the end of this guide you will have LFM3-1.7B chatting in a terminal on both machines, a browser-based chat UI served from your own hardware, and a Python setup that can hand the model real tools to call. Along the way you will see the exact size of every file you can download, the RAM picture at each quantization level, and the five errors people actually hit on Windows 11 and Kali, including the famous externally-managed-environment wall that Kali throws at pip.

No GPU is required for the Ollama and llama.cpp paths; the GGUF checkpoint is built for CPUs and GPUs alike. The Transformers path is happiest with a GPU, since the main repository ships BF16 weights aimed at GPU inference, but it will still teach you the model's tool-calling interface either way. If you have never run a language model on your own hardware before, this is about the friendliest one to start with, and the same series covered speech-to-text in Run Whisper locally on Windows 11, Mac and Kali Linux if you want a second project afterward.

What LFM3-1.7B actually is, in plain English

LFM3-1.7B is a text model for local agents and specialized applications. That one sentence carries three design decisions worth unpacking. It is a text model, part of an LFM3 family that also ships base, text, vision, audio, omni, and decision models. It is aimed at local agents: it was post-trained for agent harnesses such as Hermes Agent, Pi, and OpenClaw, which means it is shaped around following instructions, calling tools, and producing structured output rather than reciting encyclopedic facts. And it is built for specialized applications: Liquid AI expects you to fine-tune it for your own domain, format, or toolset, and has sized it so that a quick single-GPU fine-tuning run is realistic for normal people.

For a shop like Jake's, that adds up to a model that can draft an estimate, hold a conversation about a repair, hand off to a tool when a lookup is needed, and do all of it from a machine behind the counter. Ethan put it in his usual way: "Most small models are cut-down versions of something bigger. This one was raised on purpose." The LFM3 family went public in late October 2026, and the 1.7B text model is the one tuned for exactly this job. If you followed our earlier piece on running Liquid AI's LFM2.5-24B A2B locally, think of this as the opposite end of the same family: where that model traded disk space for capability, this one trades capability for the ability to run anywhere.

LFM3-1.7B at a glance.

Property	Specification
Parameters	1.73B, stored as BF16 weights (2B params on the safetensors badge)
Architecture	24 layers: 13 gated short convolutions, 5 sliding-window attention, 6 global attention
Context length	256K tokens (input + output)
Vocabulary	128,000 tokens
Languages	16, including English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish
License	LFM Open License v1.0
Built for	Local agents, tool calling, structured outputs, domain fine-tuning
Memory at 4-bit	About 1 GB, small enough to fit on any phone or laptop

Two honest caveats belong right next to that table, because they save people disappointment later. Liquid AI says plainly to use retrieval or search for knowledge-heavy tasks, meaning the model is not a walking encyclopedia and should be handed documents or a search tool when accuracy matters. And it is not recommended for agentic coding or complex multi-step reasoning due to its small size. Neither caveat matters much for Jake's estimates and customer replies; both matter if you were hoping to retire your coding assistant.

The tri-hybrid trick: 13 of 24 layers never read the whole sentence

Almost every language model you have heard of is a stack of attention layers, where every token gets to stare at every other token. That full stare is what makes big models smart, and it is also what makes them slow and hungry: the work grows with the square of the text length. LFM3-1.7B's tri-hybrid architecture takes a different route. Of its 24 layers, only 6 are global attention layers that see the entire input. Five are sliding-window attention layers that read the text in local chunks. And 13, more than half the model, are gated short convolutions, the same family of cheap, local pattern-matching operation that image filters use.

Jake's translation, after Ethan tried the technical one twice: most of the model is a bouncer checking IDs at the door, and only a few layers upstairs ever look at the whole room. The payoff shows up in the edge benchmark numbers. At 32K context, running as Q4_0 GGUF files in llama.cpp, LFM3-1.7B prefills at least 2.5 times faster and decodes at least 1.8 times faster than every compared model on all three devices Liquid AI tested, while using the least memory. The test hardware ranged from an AMD Ryzen AI Max+ 395 with its Radeon 8060S integrated GPU on Windows, to an Apple M5 Max, to an iPhone 17 Pro. In other words: the architecture choice is why a 256K-context model can still feel snappy on a laptop with no discrete GPU.

It is also why the model behaves a little differently from a pure-attention model on very long inputs: long-range understanding is carried by fewer, more precious layers. For chats, summaries, and tool-driven workflows, that trade works. For deep reasoning over a whole codebase, it is part of why the makers steer you away from agentic coding with this size. If you want the GGUF side of this story in depth, our GGUF quantization types explained piece walks the format itself.

Size required: the download, the total size, and the RAM you need

The size required for a local install of LFM3-1.7B comes down to one question: which checkpoint do you download? The model lives on Hugging Face in two flavors. The main repository, LiquidAI/LFM3-1.7B, holds the BF16 safetensors weights that Transformers, vLLM, and SGLang load. The quantization repository, LiquidAI/LFM3-1.7B-GGUF, holds ready-made GGUF files for llama.cpp, Ollama, LM Studio, and any compatible app. Every number in the two tables below is the listed size of the actual file, so you can budget disk space with confidence.

The GGUF repo: every file, exact size. These are what Ollama and llama.cpp download.

File	Size	Notes
LFM3-1.7B-QAD-Q4_0.gguf	1.05 GB	Smallest 4-bit download
LFM3-1.7B-Q4_K_M.gguf	1.1 GB	The sweet spot; every command in this tutorial pulls this one
LFM3-1.7B-Q5_K_M.gguf	1.26 GB	A little more fidelity
LFM3-1.7B-Q6_K.gguf	1.43 GB	Quality-leaning middle ground
LFM3-1.7B-Q8_0.gguf	1.85 GB	Near full precision
LFM3-1.7B-F16.gguf	3.47 GB	Full 16-bit weights
LFM3-1.7B-BF16.gguf	3.47 GB	bfloat16 weights

On the RAM side, the number Liquid AI puts on the tin is about 1 GB of memory at 4-bit precision, which is why the Q4_K_M and QAD-Q4_0 rows are the ones people actually install. That figure is the model itself; the context you feed it rides on top, and a 256K-token window holds far more text than a 32K one, so long-context work costs memory the same way it does for any model. For ordinary chat and agent work, an 8 GB machine is a comfortable home for the 4-bit files. If you push context lengths toward the ceiling or load the 16-bit files, plan for correspondingly more.

The main repo: what the Transformers route downloads. Total size lands at about 3.5 GB.

File	Size
model.safetensors	3.46 GB
tokenizer.json	17.9 MB
modeling_lfm3.py	42.4 kB
chat_template.jinja	12.1 kB
Everything else (config, tokenizer config, generation config, and friends)	A few kB each

So the practical budget looks like this. Ollama or llama.cpp with Q4_K_M: about 1.1 GB on disk, and nothing else to speak of. The Transformers route: about 3.5 GB for the repository, plus whatever room your Python environment and its libraries take, which is why a fresh venv and a few spare gigabytes make life calmer. As a rule of thumb, keep 5 GB free before starting any path in this guide and you will never watch a progress bar die at 97 percent. And if disk pressure is a recurring theme on your machine, the smaller quantizations exist precisely for you. People who run lots of local models tend to hit memory ceilings eventually, and when Ollama starts complaining about system memory, this fix for Ollama's memory complaints covers the usual suspects.

Pick your local install path: Ollama, llama.cpp, or Transformers

There is no wrong door here, but the doors lead to different rooms. Ollama is the ten-minute path: an installer, one command, and a chat prompt, with the model fetched from the GGUF repo automatically. llama.cpp is the power path on both Windows 11 and Kali Linux: one small program that either chats in the terminal or serves a web UI and an OpenAI-compatible API from your own hardware. Transformers is the builder's path: a Python environment that loads the full BF16 checkpoint and gives you the model's native tool-calling and chat-template interfaces, which is also the on-ramp if you later want to fine-tune it. LM Studio rounds out the supported apps if you prefer clicking to typing, and our LM Studio download and install guide covers that route.

The three local install paths compared.

Path	What you install	Model download	Runs on CPU?	Best for
Ollama	One installer, no Python	1.1 GB (Q4_K_M GGUF)	Yes	Chatting within ten minutes
llama.cpp	winget package on Windows, one-line script on Kali	1.1 GB (Q4_K_M GGUF)	Yes	Local web UI plus an OpenAI-compatible server
Transformers	Python 3 plus a venv	3.46 GB safetensors, BF16	Aimed at GPU inference (device_map="auto")	Tool calling, custom pipelines, fine-tuning later

Ethan's advice for Jake, and for anyone who wants the short version: start with Ollama on both machines today, because it proves the whole idea in ten minutes. Move to llama.cpp when you want the browser UI or the API for other shop tools. Reach for Transformers when you are ready to wire up tool calling in Python or bend the model toward your own documents. The three installs coexist happily, and only the model files overlap in size.

Local install on Windows 11, path 1: Ollama

This is the path Jake used on the counter PC, and it is deliberately the shortest section of this tutorial. Ollama is a single program that manages model downloads and serves them to you in a terminal chat, and the LFM3-1.7B GGUF it fetches is the 1.1 GB Q4_K_M file from the table above.

1. Install Ollama. Open PowerShell and run the one-line installer:

   irm https://ollama.com/install.ps1 | iex

   If you prefer a click-through installer, the download page at ollama.com offers a manual download as well.
2. Open a fresh terminal. A new PowerShell window, so the newly installed command is found.
3. Pull and chat in one step. Run:

   ollama run hf.co/LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

   The first run downloads the 1.1 GB file, then drops you straight into a chat prompt. Every later run starts instantly, because the model now lives on your disk.
4. Talk to it. Try a shop-flavored prompt like "Write a friendly text message telling a customer their screen replacement is done and the total is $89." The model answers in the terminal, conversation style.
5. Leave whenever you like. Ctrl+C or closing the window ends the session; nothing is lost, and the same command brings the chat back.

One expectation to set: large models are slow on computers without a strong GPU, so on a thin-and-light laptop the replies arrive at a walking pace, and that is normal. If Ollama seems to ignore your GPU entirely, that is a known fixable situation, and our guide to Ollama not using the GPU walks through it. To keep things healthy over time, updating Ollama on Windows, Mac, and Linux is a two-minute habit worth building, especially while LFM3 support is this fresh.

Local install on Windows 11, path 2: llama.cpp and its web UI

llama.cpp is the engine underneath half the local AI world: one lean C++ program that runs GGUF files on CPUs and GPUs. Two things make it the natural second step on Windows 11. It installs through winget, and its serve mode starts a local OpenAI-compatible server with a web UI, which means Jake's shop could point other software at a plain HTTP endpoint later without any cloud in the loop.

1. Install llama.cpp. In PowerShell:

   winget install llama.cpp

2. Open a new terminal window so the fresh install is on your path.
3. Serve the model with its web UI. Run:

   llama serve -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

   The first launch pulls the 1.1 GB Q4_K_M file, then starts a local server with a browser chat interface and an OpenAI-compatible API.
4. Or chat in the terminal instead. For a no-browser session:

   llama cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

A note on freshness: LFM3 is a new architecture, and it runs on llama.cpp builds with LFM3 support. The winget package you install today is current by definition, but if you are ever working with an older llama.cpp binary and LFM3 refuses to load, the answer is to update the binary, not to fight the file. And if winget is not available on a particular machine, the Ollama path above is the smoothest Windows route, with LM Studio as the click-through alternative.

Local install on Windows 11, path 3: Python and Transformers

This is the path for people who want to write code against the model: wire up tool calling, build a custom pipeline, or prep for fine-tuning later. It costs more disk, about 3.5 GB for the repository plus the Python tooling, and it is the route where the model's own Python interface, including its chat template and tool-calling format, is fully exposed. One detail matters early: LFM3-1.7B is a custom-code model, so loading it always includes trust_remote_code=True. The repository literally ships its model implementation, a 42.4 kB modeling_lfm3.py, alongside the weights.

1. Install Python 3 from python.org. During setup, accept the offer to add Python to your PATH; if you skip it, the py command works in place of python everywhere in this guide. Verify in a new PowerShell window:

   python --version

2. Create a virtual environment. From your project folder:

   python -m venv lfm3-env

3. Activate it. In PowerShell:

   lfm3-env\Scripts\Activate.ps1

   In classic cmd, use lfm3-env\Scripts\activate.bat instead.
4. Install the libraries. Inside the activated venv:

   pip install -U transformers accelerate

5. Save a chat script. Call it lfm3_chat.py:

   from transformers import pipeline
   
   generator = pipeline(
       "text-generation",
       model="LiquidAI/LFM3-1.7B",
       trust_remote_code=True,
   )
   messages = [{"role": "user", "content": "What is C. elegans?"}]
   output = generator(
       messages,
       do_sample=True,
       temperature=0.1,
       top_k=50,
       repetition_penalty=1.1,
       max_new_tokens=2048,
   )
   print(output[0]["generated_text"][-1]["content"])

   Those sampling settings, temperature 0.1, top_k 50, repetition_penalty 1.1, are the ones Liquid AI recommends for general chat, tool use, agents, and coding. For math and science work they suggest switching to temperature 1.0, top_p 0.95, top_k 20.
6. Run it. The first launch downloads the full repository, about 3.5 GB, into your home-folder cache, so this is the coffee-break step:

   python lfm3_chat.py

The loading form with automatic device placement, if you prefer building on the model object directly, is AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True, device_map="auto"). A high-level one-liner also exists for direct downloads of specific files, hf download LiquidAI/LFM3-1.7B, once you have run pip install -U "huggingface_hub". Jake's only comment after the first successful reply: "So the model's a file, the program's a file, and neither of them phones home." That is the whole idea.

Local install on Kali Linux, path 1: Ollama

Kali is the machine where people expect local AI to hurt, and the Ollama path refuses to cooperate with that expectation. The whole install is two commands, and it does not touch Python at all, which on Kali is a feature with its own story that we will get to in path 3.

1. Install Ollama. In a terminal:

   curl -fsSL https://ollama.com/install.sh | sh

2. Pull and chat. The same one-liner as on Windows:

   ollama run hf.co/LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

   The 1.1 GB download lands in Ollama's own model storage, and you are chatting in the same terminal.

The same expectations carry over from the Windows section: no strong GPU means a walking pace, and the GPU-ignored situation is fixable. What Kali adds is a security-minded audience, and the question Jake asked Ethan over dinner: "Where do these bytes even come from?" The answer in this path: one installer script from ollama.com, one GGUF pulled from Hugging Face, and from that moment the model answers from the back bench with the network cable out. Ethan, who spends his days connecting things to cloud services, considers that sentence the entire sales pitch. Our earlier roundup of Hugging Face AI models and how they fit into AWS covers the other end of that spectrum, for when you do want a managed deployment.

Local install on Kali Linux, path 2: llama.cpp

The llama.cpp route on Kali is the same two faces as on Windows: a terminal chat and a local server with a web UI. The install differs only in the first line, which pulls the current build through the llama.app installer script:

curl -LsSf https://llama.app/install.sh | sh

# Start a local OpenAI-compatible server with a web UI:
llama serve -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

# Run inference directly in the terminal:
llama cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M

The first serve or cli run fetches the 1.1 GB Q4_K_M file; after that, starts are local and instant. Because LFM3 support in llama.cpp is recent, a current build matters more here than with older models: if an older binary complains about the file, update it. Kali users who keep multiple AI tools around will recognize the pattern from our other run-it-locally walkthroughs, and the same rule of thumb applies everywhere: the GGUF is data, the binary is the program, and mismatches between them are fixed on the program side.

Local install on Kali Linux, path 3: Python, venv, and the pip wall

Every Python tutorial on Kali eventually arrives at the same brick wall, so let us walk straight into it on purpose. If you try to install Transformers system-wide, pip refuses:

error: externally-managed-environment

That error is Kali protecting its system Python. Kali's own tools are installed and managed by APT, and if pip could freely overwrite system-wide Python packages, the two package managers would spend their days breaking each other. So modern Kali marks the system Python as externally managed and tells pip to back off, a convention that arrived through PEP 668 and that Debian-family distributions share. The fix is not to force it. The fix is a virtual environment, which gives this model its own clean Python to live in, and it is the same answer the error message itself suggests.

1. Install venv support. Kali needs the python3-full package for virtual environments:

   sudo apt install python3-full

2. Create and activate a venv. From your project folder:

   python3 -m venv lfm3-env
   source lfm3-env/bin/activate

   Your prompt changes to show the active environment. Inside it, pip works freely, because nothing here is system Python anymore.
3. Install the libraries inside the venv.

   pip install -U transformers accelerate

4. Save and run the chat script. Use the same lfm3_chat.py from the Windows section, unchanged; Python code does not care which OS is underneath. The first run pulls the same roughly 3.5 GB repository, cached under ~/.cache/huggingface in your home folder:

   python lfm3_chat.py

5. Come back later. Each new terminal session needs source lfm3-env/bin/activate before running the script again. The model itself downloads only once.

Two side notes worth knowing. If a Python step anywhere in this guide fails with that externally-managed-environment error, it is always the same fix, and our dedicated guide to the pip fix on Kali goes deeper than this section can. And if your goal is installing a Python application rather than using a library, pipx is the tidier tool for that job; venv is the right home for library work like this. Jake summed up the whole arrangement after the venv clicked: "So the shop tools stay in the shop toolbox, and the model gets its own box." Exactly.

Run LFM3-1.7B as a local OpenAI-compatible server

A chat window is nice; an endpoint is a superpower. Once the model speaks the OpenAI-compatible request format over HTTP, every tool that knows how to call an AI API can call your machine instead, and Jake's estimate generator becomes a local web page rather than a terminal session. You already saw the simplest version in the llama.cpp sections: llama serve -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M starts exactly that, a local OpenAI-compatible server with a web UI, on either operating system.

If you have an NVIDIA GPU and want to serve the full BF16 checkpoint at scale, the heavier inference engines take over. With vLLM:

pip install vllm
vllm serve "LiquidAI/LFM3-1.7B"

And a request against it, in the familiar OpenAI shape:

curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
  "model": "LiquidAI/LFM3-1.7B",
  "messages": [
    {
      "role": "user",
      "content": "What is the capital of France?"
    }
  ]
}'

SGLang is the other engine in that class: pip install sglang, then python3 -m sglang.launch_server --model-path "LiquidAI/LFM3-1.7B" --host 0.0.0.0 --port 30000 brings up the same OpenAI-compatible interface on port 30000, and Docker users can run the model through Docker Model Runner with docker model run hf.co/LiquidAI/LFM3-1.7B. The checkpoint landscape mirrors this split: BF16 weights serve through vLLM, SGLang, and Transformers; FP8 and NVFP4 quantizations target SGLang on GPUs; GGUF covers llama.cpp, Ollama, and LM Studio on CPUs and GPUs; and MLX and ONNX builds cover Apple Silicon and ONNX Runtime respectively. For the shop, one laptop, one llama serve process, and the counter PC's browser is the whole data center.

Tool calling and the thinking mode

This is the feature the model was post-trained for, and the reason "local agents" appears in its job description. In the Transformers route, you describe your tools as function schemas and pass them through tools=... to apply_chat_template(). The template formats the definitions into the prompt automatically, and when the model decides it needs a tool, it answers with an XML-structured call instead of plain text:

<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>

Your application executes the call, appends the parsed assistant call to the history as a tool_calls entry, then appends the result as a tool message, and keeps passing the tool schemas on later requests so the model remembers what it can reach for. One wrinkle worth knowing before it bites: if your tool arguments arrive as JSON strings, decode them with json.loads() first, because the template wants function.arguments as a Python mapping. For harnesses with huge tool catalogs, LFM3 also supports deferred tools: mark a definition with defer_loading=True and let a search tool hand back tool_reference parts, and only the needed definitions get inserted into the prompt.

Underneath all of this runs ChatML, the model's conversation format, which you can generate with tokenizer.apply_chat_template() and add_generation_prompt=True instead of assembling it by hand:

<|startoftext|><|im_start|>system
You are a helpful assistant trained by Liquid AI.<|im_end|>
<|im_start|>user
Explain how a heat pump works.<|im_end|>
<|im_start|>assistant

The other behavior to know about is the thinking mode. Reasoning is enabled by default, so answers to harder questions may open with a visible thinking trace before the real reply. You can pass enable_thinking=False to turn traces off, though the behavior can be inconsistent on reasoning-heavy tasks, and older reasoning is trimmed from the history between turns unless you set preserve_thinking=True. None of this matters for Jake's text message drafts, and all of it matters the day he asks the model to decide whether to check inventory before quoting a price. That day is closer than he thinks: one of Liquid AI's own fine-tuning examples for this model is literally about calling the right tools in a 108-tool phone app.

How fast is LFM3-1.7B, really?

There are two honest answers, one about speed and one about smarts, and they point the same direction. On speed: in Liquid AI's edge benchmarks, every model ran as a Q4_0 GGUF in llama.cpp with 32,768 tokens of context, batch size 1, and GPU offload, across three devices, an AMD Ryzen AI Max+ 395 with Radeon 8060S integrated graphics on Windows, an Apple M5 Max, and an iPhone 17 Pro. LFM3-1.7B prefilled at least 2.5 times faster and decoded at least 1.8 times faster than every compared model on all three devices, while using the least memory. On smarts, the surprise from before the fold, in numbers:

Numbers from Liquid AI's LFM3 release, higher is better. The 1.7B column holds its own against models up to 5x its size, and slips exactly where a small model should.

Benchmark	LFM3-1.7B	LFM2.5-2.6B	Qwen3.5-4B	Qwen3.5-9B
LiveClaw (agents)	45.60	14.70	19.94	35.77
WebArena-Verified (web agents)	38.42	3.02	21.92	28.94
BFCL V4 (tool use)	71.39	55.89	55.12	62.83
IFEval (instruction following)	96.66	95.39	92.52	92.08
LiveCodeBench v6 (coding)	73.46	60.57	61.99	69.86
GPQA Diamond (knowledge)	63.37	57.78	75.76	80.40
Overall average	49.37	31.55	39.27	47.24

Read the table the way Ethan read it to Jake: top rows, the agentic work this model was built for, and it leads every column. Bottom knowledge row, where parameter count buys recall, and the 9B model wins by seventeen points. That is not a contradiction; that is the design brief doing its job. The evaluation setup behind these numbers served every model with vLLM in BF16 at full context, 262,144 tokens for LFM3-1.7B and the Qwen models, and the overall average is the unweighted mean of the full benchmark suite. For the scale ceiling: on server hardware, one NVIDIA B300 serving 1,024 concurrent requests, Liquid AI puts LFM3-1.7B at 3.6 billion tokens per day. Jake will never need that, and it is oddly comforting to know it is there.

What it is great at, and where it taps out

Put it all together and a clear job description falls out. LFM3-1.7B is great at instruction following, with a 96.66 on IFEval and a 93.72 on IFStruct, the discipline that makes small models usable in real workflows. It is great at tool use and agent loops, as the table above shows. It is great on the edge: about 1 GB of memory at 4-bit, fastest prefill and decode in its class, and a 256K-token context window that turns entire chat histories and long documents into single prompts. It speaks 16 languages, from English and Spanish to Hindi, Thai, and Polish, which covers a lot of shop counters. And it is designed to be specialized: through TRL you can run supervised fine-tuning, DPO, GRPO, or continued pre-training, with Liquid AI's own example set including extracting structured JSON from messy documents, answering only from retrieved documents, and calling the right tools in a 108-tool phone app.

Where it taps out is equally clear, and pretending otherwise would be a disservice. Knowledge-heavy questions without retrieval are not its home turf; the GPQA row shows the gap, and the guidance is to hand it search or documents for those tasks. Agentic coding and complex multi-step reasoning are explicitly not recommended for this size. It is a specialist's tool: brilliant at the last mile of language, modest at the encyclopedia. Jake's mental model, which Ethan declared better than most slide decks: "It's the new kid in the shop. Fast, polite, never tired. You just don't send him to the warehouse alone."

Troubleshooting: the errors you will actually hit

1. error: externally-managed-environment (Kali)

The most famous error in this tutorial, covered in depth in the Kali path 3 section above. Short version: it is Kali protecting the system Python that APT manages, the fix is a virtual environment, and forcing your way past it with flags risks breaking tools you did not know depended on those packages. The full pip fix guide has the details.

2. The model refuses to load, or something mentions remote code (any OS)

LFM3-1.7B is a custom-code model: its implementation ships in the repository as Python files, and the loaders expect you to opt in with trust_remote_code=True. If a script omits it, loading stops with an error that mentions remote code. Add the flag to both the pipeline call and any from_pretrained call, exactly as in the script from the Windows section.

3. Ollama is slow, or the GPU sits idle (both OSes)

Large models are slow on computers without a strong GPU, and Ollama defaults can leave a capable GPU unused. If responses crawl and your graphics card shows no activity, our Ollama GPU fix covers the usual causes and how to confirm each one.

4. Out-of-memory complaints (both OSes)

The 4-bit files need about 1 GB for the weights, but context, the runtime, and everything else your machine is doing rides on top. If you hit memory errors, drop to the 1.05 GB QAD-Q4_0 file, shorten your context, and close the browser tabs you are hoarding. For the Ollama-specific version of this problem, this memory fix walkthrough goes deeper.

5. A tool cannot reach your local server

When an app cannot talk to the model, the server is usually not listening where the app expects, or it was stopped without anyone noticing. Confirm the serve command is still running in its own terminal, and check the connection basics in our could-not-connect-to-Ollama fix, which applies the same detective work to any local endpoint. LM Studio users hitting load failures instead should see the LM Studio failed-to-load-model fix; with LFM3 the answer is usually the same freshness rule as llama.cpp, current app, current GGUF.

One last housekeeping note: uninstalling is boring on purpose. The model is files on disk, so when you want the space back, remove the model through whichever tool you installed it with, delete the GGUF you fetched, or drop the venv folder, and the gigabytes return. Nothing hooks into the system beyond the tools themselves.

The LFM3-1.7B local install FAQ: 16 questions

How much disk space do I need for a local install of LFM3-1.7B?

For the Ollama or llama.cpp route, 1.1 GB: that is the exact size of the Q4_K_M GGUF every one-command install downloads. If you want the smallest option, the QAD-Q4_0 file is 1.05 GB, and stepping up in quality costs 1.26 GB at Q5_K_M, 1.43 GB at Q6_K, and 1.85 GB at Q8_0. For the Transformers route, the main repository's total size is about 3.5 GB, dominated by the 3.46 GB model.safetensors file. Keep 5 GB free and every path in this guide fits comfortably, Python tooling included.

How much RAM does LFM3-1.7B need to run locally?

At 4-bit precision the model takes about 1 GB of memory, which is the whole design point: Liquid AI sized it to fit on any phone or laptop. That figure covers the weights themselves; the context window rides on top, so longer conversations and bigger documents add memory like they do for every language model. In practice, an 8 GB machine handles the 4-bit files at everyday context lengths without drama, and you only need to think harder about RAM if you load the 16-bit files, which weigh 3.47 GB on disk, or push toward the full 256K-token context.

Is LFM3-1.7B free to download and use?

The download from Hugging Face is free, and the model ships under the LFM Open License v1.0, with the license file included right in the repository. Running it locally costs nothing beyond electricity: no API keys, no per-token billing, no account. One honest note for business users: open licenses vary in what they permit, and this tutorial is not legal advice, so read the LICENSE file that ships with the model before you build a product on it. For everything in this guide, chatting, serving, and tool calling on your own machines, you are squarely inside the intended use.

Can LFM3-1.7B run on a CPU only, without a GPU?

Yes. The GGUF checkpoint is built for local inference on CPUs and GPUs alike, which is exactly why the Ollama and llama.cpp paths work on the plainest hardware. Expectation setting from the model's own makers: large models are slow on computers without a strong GPU, so a CPU-only laptop gives you walking-pace generation rather than sprinting. The architecture helps here too, since 13 of the model's 24 layers are cheap gated convolutions rather than expensive full attention, which is a big part of why this model in particular stays usable on modest machines.

Ollama vs llama.cpp vs Transformers: which local install should I pick?

Pick Ollama if you want to be chatting in ten minutes and never see a config file; it is two commands on either OS. Pick llama.cpp if you want the local web UI and the OpenAI-compatible server, because llama serve hands you both in one process. Pick Transformers if you are writing Python against the model: tool calling, custom pipelines, and eventually fine-tuning all live on this path, at the cost of a 3.5 GB download and a virtual environment. They are not mutually exclusive; many people end up with all three.

Does LFM3-1.7B work fully offline after the download?

Yes. Once the files are on your machine, inference is arithmetic on your own hardware; nothing about generating a reply requires a network. The only online moments in the whole workflow are the initial download of the GGUF or safetensors, and the one-time fetch of the Python libraries. Jake's favorite proof is pulling the ethernet cable and asking for another estimate. The exceptions are features you would add yourself, like a web search tool handed to the model, which naturally needs the network to fulfill its half of the bargain.

What is the difference between the safetensors repo and the GGUF repo?

The main repository, LiquidAI/LFM3-1.7B, holds the BF16 safetensors weights plus the model's own Python implementation files, and it is what Transformers, vLLM, and SGLang load; the total size lands at about 3.5 GB. The GGUF repository, LiquidAI/LFM3-1.7B-GGUF, holds pre-quantized single-file versions, from 1.05 GB to 3.47 GB, for llama.cpp, Ollama, LM Studio, and compatible apps. Same model, different packaging: safetensors is the full-precision developer route, GGUF is the run-anywhere route, and the quantization choice inside GGUF is the size-versus-fidelity dial.

Why does pip fail with externally-managed-environment on Kali Linux?

Because Kali is protecting its system Python. Kali's own tools arrive through APT, and if pip could freely install into the same system-wide Python, the two package managers would eventually break each other's packages. Following PEP 668, modern Kali marks the system environment as externally managed, so pip refuses system-wide and user-level installs. The fix is a virtual environment: install python3-full, create a venv, activate it, and pip works freely inside. The Kali path 3 section of this guide walks it step by step.

How do I update LFM3-1.7B after installing it locally?

The model is a file, not a service, so there is no auto-update and no subscription; a checkpoint you downloaded keeps working forever. When Liquid AI publishes a newer checkpoint, it lands as a new release on Hugging Face, and the refresh is just another download: re-run the same ollama run or llama serve -hf command to pull the current files. What is worth updating on a schedule is the tooling itself: keep Ollama and llama.cpp current, especially while LFM3 support is this new, so the binary always understands the latest files.

Can LFM3-1.7B really run on a phone?

That is the design target, not a stretch. At 4-bit precision the model takes about 1 GB of memory, which Liquid AI describes as small enough to fit on any phone or laptop, and the phone in the edge benchmark set was an iPhone 17 Pro running an iOS app built on llama.cpp. On that device, along with an AMD Ryzen AI Max+ 395 Windows machine and an Apple M5 Max, it prefilled at least 2.5 times faster and decoded at least 1.8 times faster than every compared model while using the least memory. Android phones with modest RAM are realistic homes for the 1.05 GB QAD-Q4_0 file.

What is the 256K context window, and will it eat my RAM?

The context window is 262,144 tokens of combined input and output, the space the model can consider at once: whole chat histories, long manuals, entire document sets in a single prompt. The memory the model needs grows with how much of that window you actually fill, so the roughly 1 GB figure for 4-bit weights is the floor, not the ceiling, and a fully packed 256K context costs considerably more. The published edge benchmarks ran at 32K context, which is a sensible everyday setting. In short: the window is there when you need it, and it charges you only for what you use.

Does LFM3-1.7B support tool calling?

Yes, and it is a headline feature rather than a bolt-on. You pass function schemas through tools=... to apply_chat_template(), and the model answers with XML-structured calls that your application executes and feeds back as tool messages. It scores 71.39 on BFCL V4 and 80.36 on ToolSandbox, and there is even a deferred-tools mode where a search tool retrieves definitions on demand, so a harness with a huge tool catalog does not have to stuff every schema into every prompt.

How do I turn off the thinking traces?

Reasoning is enabled by default, so harder prompts can produce a visible thinking trace before the answer. Pass enable_thinking=False when you apply the chat template to suppress it, with the caveat that the behavior can be inconsistent on reasoning-heavy tasks. Related: reasoning from earlier turns is removed from the history between requests unless you set preserve_thinking=True, which matters if your workflow relies on the model remembering its own earlier reasoning.

Is LFM3-1.7B good for coding?

For everyday coding questions it is surprisingly capable: 73.46 on LiveCodeBench v6 puts it ahead of much larger models on that benchmark. But the honest boundary from Liquid AI is that it is not recommended for agentic coding or complex multi-step reasoning due to its small size, meaning you should not hand it a repository and a vague goal the way you would a coding agent built on a 100B-class model. Snippets, explanations, small functions, and scripting help: yes. Autonomous multi-file refactors: that is what the bigger LFM3 sizes and other models are for.

Can I fine-tune LFM3-1.7B?

Yes, and specialization is the point of the model: it is described as made to be specialized through quick single-GPU fine-tuning. The supported route runs through TRL, with four flavors depending on your goal: SFT for teaching formats, DPO for answer style and refusal behavior, GRPO for tool-choice accuracy, and continued pre-training for absorbing a new domain from raw text. Liquid AI's own example set includes extracting structured JSON from messy documents and calling the right tools in a 108-tool phone app, both of which map cleanly onto small-business workloads.

Does LFM3-1.7B work with LM Studio?

Yes. LM Studio is one of the three named local apps for the GGUF checkpoint, alongside llama.cpp and Ollama, so you can search for the model from inside the app and pull one of the quantized files from the LiquidAI/LFM3-1.7B-GGUF repository. The same freshness rule applies as everywhere else: use a current version of the app so LFM3's new architecture is supported, and if a download or load fails, an outdated build is the first suspect. Our LM Studio install guide and the failed-to-load-model fix cover the click-side of the process.

The verdict: is LFM3-1.7B worth your disk space?

For the job it applied for, yes, unreservedly. The local install is ten minutes on Windows 11 with Ollama and barely longer on Kali Linux; the download is 1.1 GB; it runs in about a gigabyte of RAM at 4-bit; and it beats models five times its size at the agent and instruction-following work that actual desktop assistants spend their day doing. The 256K context window means you will not be trimming conversations to fit, the tool calling is first-class rather than decorative, and the whole thing works with the network cable pulled out, which is either a security feature or a party trick depending on your audience.

The equally honest counterweight: it is not an encyclopedia, and it is not your next coding agent, and it was never pretending to be either. Treat it as a fast, private, endlessly customizable language engine for the last mile of your daily text, hand it search when facts matter, and it will feel overqualified for its salary. Jake kept the counter PC on Ollama for customer replies and put llama serve on the Kali laptop for the API experiments; Ethan, who builds cloud systems for a living, installed it on his own travel laptop the same week, which is the strongest review a local model can get. If you build something with it, the next stop in this series is waiting, and so are we.