Run Nemotron 3.5 Bolt E4B Locally on Windows 11, Mac and Kali Linux: Why the "4GB" Model Is a 6.2 GB Download, the VRAM and RAM You Really Need, Ollama, LM Studio and llama.cpp Setup

To run Nemotron 3.5 Bolt E4B locally, install or update Ollama and run ollama run nemotron-3.5-bolt:e4b. That pulls NVIDIA's new small model, released October 23, 2026: an 8-billion-parameter model with 4 billion "effective" parameters, post-trained by NVIDIA from Google's Gemma 4 E4B for fast voice assistants, game characters and tool-calling agents. It also runs in LM Studio (lms get nvidia/nemotron-3.5-bolt-e4b), llama.cpp, Unsloth and vLLM, on Windows 11, Mac and Linux, and offline once downloaded. Here is the part the name hides: the "4GB" model is not a 4 GB download. NVIDIA's own file is 6.18 GB, and Ollama's is 7.9 GB. Open the file's tensor table and you find why: 3.7 GB of NVIDIA's file is two lookup tables (the "per-layer embeddings" that make it an E4B), and llama.cpp keeps those in system RAM, so your graphics card only holds about 2.5 GB of working weights. That is why NVIDIA measures 3.1 to 3.5 GB of VRAM. So the 4 GB is a VRAM figure, and you still need 6 to 8 GB of disk and several GB of system RAM. A second catch: the chat template switches thinking off unless your app asks for it. This guide covers setup on every system, which file fits your hardware, thinking and conversational mode, tool calls, and the errors you are likely to see.

Jake's phone shop loses customers in a very specific way. Someone asks whether he has a screen for their model, Jake disappears into the back room to check, and by the time he comes back the queue has thinned. Last Saturday it was three people, about $240 of repairs, walking out while he counted boxes. "I want the counter to answer that question by itself," he told Ethan. Ethan already had a stock script. What he needed was a small model that could listen to a question, call the script, and say the answer out loud on the shop's old office PC, without sending a word to anyone's cloud. "That's exactly what this one was built for," Ethan said, reading NVIDIA's release. "The question is whether 4 GB means what we think it means."

New to running AI on your own computer? A local model is a file you download and run with a free app such as Ollama or LM Studio. After the download, nothing you type leaves your machine: there is no account, no per-token bill, and it works offline. The limits are your computer's memory and speed. Our run AI locally series covers the basics, and our comparison of Ollama, LM Studio, Jan and Unsloth helps you pick an app. Bolt E4B is a conversation and action model. It is built to hold a natural back-and-forth, follow a persona, and call your functions at the right moment, rather than to write long essays or solve olympiad math.

What is Nemotron 3.5 Bolt E4B?

Bolt E4B is the small member of NVIDIA's Nemotron 3.5 Bolt family. Its big sibling, Bolt 74B-A9B, came out in mid-October as a 24 GB model for RTX 5090-class cards. E4B is the one NVIDIA promised for smaller machines, and it arrived on Hugging Face and NVIDIA's API catalog on October 23, 2026. Unlike the 74B model, which NVIDIA built from its own Nemotron 3.5 Super, E4B starts from Google's Gemma 4 E4B instruction model. NVIDIA then trained it further with reinforcement learning for multi-turn tool use, instruction following and staying grounded in what you give it. Here is the model at a glance:

NVIDIA aims it at four kinds of jobs: voice and device assistants (PC, home, car), interactive characters such as game NPCs and companions, robots and industrial screens, and any agent that has to keep a persona while calling tools. Its launch demo is a small Reachy Mini robot on an 8 GB Jetson Orin Nano, where a vision model describes the room and Bolt E4B decides what to say and which animation script to run. "So it is a talker that can push buttons," Jake said. "A talker that knows when to push buttons," Ethan said, "which is the hard part."

Why the "4GB" model is a 6.2 GB file, and 7.9 GB in Ollama

The full name of NVIDIA's GGUF repository is NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, and the file inside it, Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf, is 6,180 MB. Ollama's default tag is bigger again at 7.9 GB. Both numbers are honest. They just measure different things, and the reason is the most interesting part of this model's design.

A GGUF file starts with a table that lists every tensor (block of weights), its shape and its format. Reading that table for NVIDIA's file gives this split:

Now the name makes sense. The "E" in E4B stands for effective, and the 4.03 billion working parameters are exactly that "4B". The other 3.49 billion sit in two lookup tables. Gemma's per-layer embeddings give every layer its own small, 256-number note about each token, which helps a small model punch above its size. But they are read like a dictionary: one row per token, with no heavy maths. llama.cpp treats both tables as input lookups and keeps them in system memory, so the graphics card only needs the 2.47 GB of working weights plus the conversation's memory and some buffers. Add those up and you land on the figures NVIDIA reports: 3.5 GB of VRAM on an RTX 5070 at a 3,000-token prompt and 4.3 GB with 32,000 tokens of cached conversation.

Ollama's 7.9 GB has two extra pieces. Its registry manifest for nemotron-3.5-bolt:e4b lists three files: the same 6.180 GB model NVIDIA publishes, a 0.99 GB image projector (the vision encoder from Gemma 4, even though NVIDIA ships the model as text only), and a 0.74 GB DFlash drafter, the small helper model that speeds up generation by guessing several tokens ahead. So Ollama users get speculative decoding switched on out of the box (its settings file sets 7 draft tokens), at the cost of a bigger download.

"So the 4 GB is real," Jake said, "it just isn't the download." "It is the part that has to be fast," Ethan said. "The rest can wait in ordinary RAM." The practical rules that follow from this:

• Disk: plan on 6.2 GB for NVIDIA's file and 7.9 GB through Ollama, and more if you keep a second version.
• VRAM: about 3 to 5 GB for NVIDIA's file, depending on context. A 6 GB card is comfortable and an 8 GB card has room to spare.
• System RAM: keep at least the size of the lookup tables free on top of your normal use. With no GPU at all, the whole file runs from RAM, and about 6 GB of RAM is a fair floor. LM Studio's minimum is 5 GB.

System requirements: which Bolt E4B file to download

You can choose from NVIDIA's own file, LM Studio's community conversions, ggml-org's, and Unsloth's "dynamic" GGUFs, which shrink the lookup tables as well as the working weights. Sizes below are the Hugging Face file sizes on October 23, 2026. Several repositories were still being re-uploaded that day, so check the size shown before you download:

Unsloth also has UD-IQ3_XXS (3.70 GB), UD-Q2_K_XL (3.74 GB), UD-Q3_K_XL (4.22 GB), UD-Q5_K_XL (5.87 GB), UD-Q6_K_XL (7.11 GB) and UD-Q8_K_XL (8.19 GB), plus separate image projector files (0.56 to 0.99 GB) and a 0.40 GB 8-bit DFlash drafter. If the letters and numbers in those names are a blur, our guide to GGUF quantization types explains what Q4_K_M, IQ2_M and the "UD" prefix mean.

Which should you pick? Start with NVIDIA's Q4_K_M if it fits. It is the version NVIDIA quantized itself, using about a million tokens of the model's own training conversations to decide which weights to keep precise, and it is the version behind every 4-bit benchmark and speed figure NVIDIA publishes. Unsloth's files are smaller mainly because they store the lookup tables at 4 bits instead of 8. How much that costs in quality has not been measured publicly yet, so treat the 4.76 GB file as the space-saver and NVIDIA's as the reference. If you have an 8 GB card and want a little more quality for the same download, LM Studio's Q6_K is the curious one: it trims the lookup tables and spends the space on 6-bit working weights. "Same size, different trade," Ethan said. "Less dictionary, more brain."

GPU and CPU sizing: 4, 6, 8 and 12 GB cards, Jetson and no GPU

Matching those files to common hardware gives these starting points. Each assumes at least 8 GB of system RAM, and 16 GB is more comfortable:

• 4 GB cards (GTX 1650, laptop RTX 3050 4 GB): NVIDIA's file needs 3.1 to 3.5 GB of VRAM at short prompts, and Windows and your screen use some VRAM too, so it is tight. Use Unsloth's UD-Q3_K_XL or UD-IQ2_M, keep the context at 4K to 8K, or let Ollama put part of the model in RAM.
• 6 GB cards (RTX 2060, RTX 3050 6 GB, RTX 4050 laptop): NVIDIA's Q4_K_M fits with a 16K to 32K context. This is the sweet spot the "4GB" name is aiming at.
• 8 GB cards (RTX 3060 Ti, 4060, 5060): Q4_K_M with the full 64K context, Q6_K, or Q8_0 at moderate contexts. You can also run the DFlash drafter on the card for extra speed.
• 12 GB and up: anything up to Q8_0 with the full 64K context and room for a speech model beside it, which matters if you are building a voice assistant.
• No GPU: it runs on the CPU from system RAM. Plan on 8 GB of RAM as a bare minimum and 16 GB to use the computer for anything else. Expect replies to start slowly on long prompts, because reading a 3,000-token prompt is the heavy part on a CPU.

NVIDIA reports these speeds for the Q4_K_M file in llama.cpp with CUDA, one conversation at a time:

Two things stand out. First, a mid-range desktop card is fast enough for real-time speech: 133 tokens a second is many times faster than anyone can listen. Second, the time to first token, not the generation speed, is what a voice assistant feels like. On the small Jetson it takes almost six seconds to read a 3,000-token prompt, so keep the system prompt and tool list lean on small devices, and reuse the cached conversation instead of resending it. NVIDIA also reports that its DFlash drafter adds up to 21 percent more throughput than the drafter Google shipped with Gemma 4. AMD and Intel GPUs work through llama.cpp's Vulkan, ROCm and SYCL builds, and Ollama picks a backend for you; NVIDIA's numbers only cover its own hardware.

Context length vs memory: 64K, 128K and the llama-server trap

The KV cache is the memory a model uses to remember your conversation. Bolt E4B's is unusually small, for two reasons you can read from the file's settings. 35 of its 42 layers use sliding-window attention that only looks back 512 tokens. And the last 18 layers reuse the cache of earlier layers instead of keeping their own, so only 24 layers store anything, and only 4 of those store the full conversation. Working from those settings (2 key/value heads, 512-dimension heads in the full layers), the cache at 16-bit precision costs:

Runtimes add compute buffers on top, which is why NVIDIA measures more than the cache alone. The table explains NVIDIA's numbers well: going from a 3K prompt to 32K of cached conversation added 0.8 GB of VRAM on the RTX 5070. The lesson for you is that context is cheap with this model, so you rarely need to shrink it for memory reasons. The file is a different matter.

Here is the trap. NVIDIA supports up to 64K tokens of input and 64K of output, and its own llama.cpp command sets -c 65536. But the GGUF file itself records a context length of 131,072, and Ollama and LM Studio both list 128K. If you start llama-server without -c, it reads the context from the model, so it reserves memory for 128K and lets you feed in more than NVIDIA ever tested. NVIDIA's longest test runs at 64K, so there is no reason to go further. Always pass -c (8192 is a good start for a chat assistant, 65536 at most), and in Ollama set num_ctx only as high as you need.

Run Bolt E4B on Windows 11 with Ollama

1. Install or update Ollama. Download the Windows installer from ollama.com, or run irm https://ollama.com/install.ps1 | iex in PowerShell. If you already have Ollama, update it first. A model published today can need a newer runner than the one you installed in the spring, and our Ollama update guide shows how.
2. Pull and run the model. Open a terminal and run ollama run nemotron-3.5-bolt:e4b. The first run downloads 7.9 GB, and later runs start from disk in seconds.
3. Try a real prompt. Ask "A customer asks if we have an iPhone 17 screen in stock. Reply in one sentence as the shop assistant." You should get a short, friendly line. Type /bye to leave.
4. Check where it runs. In another terminal, run ollama ps. It shows how much of the model sits on the GPU. On an NVIDIA card it should say 100% GPU, or a GPU/CPU split if your card is small. If it says 100% CPU on a machine with a graphics card, see our Ollama GPU fix.
5. Turn thinking off when you want speed. Start it with ollama run nemotron-3.5-bolt:e4b --think=false, or type /set nothink inside the chat. For a counter assistant answering stock questions, you rarely need reasoning.

Ollama's Bolt E4B tags on October 23, 2026, all with a 128K context listing and text plus image input:

• e4b (the default, 7.2 to 7.9 GB depending on your platform) and e4b-q4_K_M (7.9 GB): NVIDIA's Q4_K_M plus the image projector and DFlash drafter.
• e4b-q8_0 (9.8 GB) and e4b-bf16 (17 GB) for higher precision.
• e4b-nvfp4 (7.2 GB), e4b-mxfp8 (12 GB) and e4b-mlx-bf16 (17 GB): MLX builds for Apple Silicon Macs.

Running a smaller file in Ollama. If 7.9 GB will not fit, Ollama can pull a GGUF straight from Hugging Face: ollama run hf.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF:UD-Q3_K_XL. Use the exact name from the file list. You lose Ollama's bundled drafter, but you can save more than half the download. If loading still fails with a memory error, our memory error guide walks through the options.

Run Bolt E4B on a Mac

This is where E4B differs most from its big sibling. In Ollama, the 74B-A9B model does not run on Apple Silicon yet, while E4B runs on Windows, Linux and Apple Silicon alike. So a MacBook owner who could not run Bolt last week can run this one. You have three good options:

• Ollama: install the Mac app from ollama.com, then run ollama run nemotron-3.5-bolt:e4b. On Apple Silicon you can also choose the MLX builds directly: ollama run nemotron-3.5-bolt:e4b-nvfp4 (7.2 GB) or e4b-mxfp8 (12 GB).
• LM Studio: search for "Nemotron 3.5 Bolt E4B" in the Discover tab, or run lms get nvidia/nemotron-3.5-bolt-e4b. LM Studio lists a 5 GB minimum of system memory. See the LM Studio section for its thinking and conversational toggles.
• llama.cpp: download llama-b11838-bin-macos-arm64.tar.gz (or any newer build) from the llama.cpp releases page, unpack it, and use the commands in the llama.cpp section. Metal acceleration is built in.

On a Mac, the GPU and CPU share one pool of unified memory, so the VRAM-versus-RAM split above does not apply. What matters is the total: an 8 GB MacBook Air can load NVIDIA's 6.18 GB file, but it will be under pressure with a browser open, and a 16 GB Mac is comfortable. On 8 GB, use Unsloth's 3.70 to 4.76 GB files. macOS will tell you when it is short of memory by slowing everything down, rather than with an error message.

Run Bolt E4B on Kali Linux and Ubuntu

Linux gives you the most control. On Kali, Ubuntu or Debian, the quickest route is still Ollama:

1. Install Ollama: curl -fsSL https://ollama.com/install.sh | sh. The script sets up a systemd service and detects an NVIDIA or AMD GPU if the driver is installed.
2. Check the GPU driver first if you have an NVIDIA card: nvidia-smi should list your card. On Kali, install the driver with sudo apt install nvidia-driver nvidia-cuda-toolkit and reboot if it does not.
3. Run the model: ollama run nemotron-3.5-bolt:e4b.
4. Serve it to other programs: the Ollama service listens on http://localhost:11434, with an OpenAI-compatible API at /v1. Keep that port private to your machine; do not expose it to your network without a password in front of it.

For llama.cpp, download llama-b11838-bin-ubuntu-x64.tar.gz (CPU), llama-b11838-bin-ubuntu-vulkan-x64.tar.gz (most GPUs) or llama-b11838-bin-ubuntu-cuda-13.4-x64.tar.gz (NVIDIA) from the releases page. The Ubuntu builds run on Kali as well. Unpack, then use the commands below. Our Gemma 4 on Kali guide covers driver and offline setup in more depth, and everything there applies to Bolt E4B, which is built on the same architecture.

Ubuntu snap. Canonical publishes an inference snap for the Bolt family: sudo snap install nemotron-3-5-bolt, then run nemotron-3-5-bolt. It was at version 2.0.0-beta.17 on October 23 and picks a CPU or NVIDIA GPU engine for your hardware automatically. Its description covers both the 74B and E4B models, so check which one it set up before you rely on it. Kali does not ship snapd by default, so on Kali stay with Ollama or llama.cpp. If you would rather run Ollama in a container with GPU access, our Ollama Docker Compose guide has a ready file.

Run Bolt E4B in LM Studio

LM Studio is the friendliest option if you prefer buttons to a terminal, and its model page for Bolt E4B exposes the model's two special switches as toggles:

1. Update LM Studio and its llama.cpp runtime (in the app's runtime settings) so it recognizes the Gemma 4 architecture and the new chat template.
2. Download the model: search "Nemotron 3.5 Bolt E4B" in Discover, or run lms get nvidia/nemotron-3.5-bolt-e4b. The community files are Q4_K_M (6.18 GB), Q6_K (6.22 GB) and Q8_0 (8.03 GB).
3. Load it with a context of 8K to 32K to start. LM Studio's preset already uses temperature 1.0, top-k 64, top-p 0.95 and no repeat penalty.
4. Use the toggles: Enable Thinking (on by default in LM Studio) and Conversational Mode (off by default). Turn conversational mode on for short, lightly formatted replies meant to be spoken.
5. Serve it: start LM Studio's local server (the Developer tab) to give other apps an OpenAI-compatible endpoint, usually http://localhost:1234/v1.

LM Studio also parses the model's reasoning into a separate collapsible block, using the model's <|channel>thought markers, so you see clean answers. If you get "failed to load model", it is almost always an outdated runtime or not enough memory, and our LM Studio fix covers both.

llama.cpp commands for Bolt E4B (with and without the DFlash drafter)

llama.cpp gives you the most speed and control, and it is what NVIDIA's own recipe uses. Any recent build handles Gemma 4's architecture. NVIDIA's commands pin build b11459, and b11838 was the newest on October 23. Two things changed recently that trip people up: llama.cpp v0.7.0 (October 15) moved llama-server's default port to 9931, so always pass --port, and the model's context default is 128K, so always pass -c.

Simplest start (downloads from Hugging Face and serves an OpenAI-compatible API):

llama-server -hf nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB \
  --jinja -c 8192 -ngl 99 --port 8080

--jinja is required. It makes llama-server use the model's own chat template, which carries the thinking switch, conversational mode and tool-call format. -ngl 99 puts all working layers on the GPU, and the lookup tables stay in RAM by design. To use a smaller Unsloth file, swap the repository: -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF:UD-Q4_K_XL. Unsloth's repository also contains image projector files, which llama.cpp fetches with the model when a repository has one; add --no-mmproj if you only want text and want to save about 1 GB.

NVIDIA's recipe with Docker (an NVIDIA GPU and the NVIDIA Container Toolkit required). First download the model and drafter with the Hugging Face CLI:

hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB \
  Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --local-dir ./bolt-models
hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash \
  Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf --local-dir ./bolt-models

Then serve with the drafter:

docker run --rm --gpus all -p 8000:8000 \
  -v "$PWD/bolt-models:/models:ro" \
  ghcr.io/ggml-org/llama.cpp:server-cuda13-b11459 \
  -m /models/Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf \
  --spec-draft-model /models/Nemotron-3.5-Bolt-E4B-DFlash-BF16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 --spec-draft-ngl 999 \
  --alias nemotron-bolt --host 0.0.0.0 --port 8000 \
  -c 65536 -ngl 999 --jinja

Drop the three --spec-* lines to run without the drafter. The drafter is a 0.74 GB model that proposes seven tokens in one pass, and the main model checks them all at once, so you get the same output faster. It costs extra VRAM, so on a 4 GB card skip it. On Windows, the same flags work with the llama-b11838-bin-win-cuda-13.4-x64.zip release (unpack it next to the matching cudart-llama-bin-win-cuda-13.4-x64.zip) or the CPU build llama-b11838-bin-win-cpu-x64.zip.

Test it from a terminal (on Windows, use curl.exe in PowerShell):

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"nemotron-bolt","messages":[{"role":"user","content":"What is 17 multiplied by 23?"}],
       "max_tokens":512,"chat_template_kwargs":{"enable_thinking":true}}'

Change the port to 8000 if you used the Docker recipe. The reply's reasoning_content holds the thinking and content holds the answer.

vLLM, for data-center GPUs. If you want to serve many users at once from an H100, A100 or GB200, NVIDIA's full-precision recipe uses the vllm/vllm-openai:v0.31.0 image and the BF16 checkpoint:

vllm serve nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 \
  --served-model-name nemotron-bolt --port 8000 \
  --dtype bfloat16 --max-model-len 65536 --max-num-seqs 64 \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 \
  --limit-mm-per-prompt '{"image":0,"video":0,"audio":0}'

The two gemma4 parsers split tool calls and reasoning out of the text, and the last line switches off the image, video and audio inputs the model was not trained for. For speculative decoding in vLLM, NVIDIA pairs the BF16 model with a separate NVFP4 DFlash drafter, nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16-DFlash-NVFP4, passed with --speculative-config and seven speculative tokens. If you prefer a desktop app built around Unsloth's files, Unsloth Desktop runs the model as well and sets these options for you.

Thinking on and off, the thinking budget, and conversational mode

Bolt E4B can reason step by step before it answers, or answer directly. NVIDIA calls thinking the default, and its benchmarks run in reasoning mode for most tests. But read the chat template inside the file and you find the line enable_thinking | default(false): if your app sends nothing, thinking is off. LM Studio switches it on for you, and every official code sample passes it explicitly. In vLLM, Transformers or your own code, set it yourself, every time, so you know which model you are actually testing.

• Ollama: --think=false on the command line, /set think and /set nothink inside a chat, or "think": false in its native API.
• LM Studio: the Enable Thinking toggle.
• llama-server and vLLM: send "chat_template_kwargs": {"enable_thinking": false} in the request body (or true to turn it on).
• Thinking budget in llama-server: start it with --reasoning on --reasoning-budget 300 to cut reasoning off at 300 tokens. The arithmetic works like this: if a voice reply must fit in 500 tokens and answers average 200, give thinking 300 and the total ceiling 500.

Conversational mode is the switch built for voice. Look inside the template and it turns out to be one sentence. When conversational_mode is true, the template adds this to your system prompt: "You are having a spoken conversation with the user. Respond as briefly as possible while still addressing their questions fully." That is useful to know for two reasons. You can see exactly what it changes, and in apps without the toggle you can get the same effect by putting that sentence at the end of your own system prompt (in Ollama, /set system followed by your prompt). It combines with thinking on or off and with tool calls.

For Jake's counter, Ethan settled on thinking off and conversational mode on for everyday questions, with thinking switched on only for the rare "can you book me in for Tuesday and remind me what it costs" requests that need a few steps. "Thinking is for the questions where I would have to think too," Ethan said. "The stock question isn't one of them."

Use Bolt E4B from Python

Every local server here speaks the OpenAI API, so the official openai package works unchanged (pip install openai). This example talks to llama-server or vLLM, where chat_template_kwargs is passed straight to the template:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="local")

reply = client.chat.completions.create(
    model="nemotron-bolt",
    messages=[
        {"role": "system", "content": "You are the counter assistant at a phone repair shop."},
        {"role": "user", "content": "Do you fix cracked back glass?"},
    ],
    max_tokens=1024,
    temperature=1.0,
    top_p=0.95,
    extra_body={"chat_template_kwargs": {"enable_thinking": False,
                                         "conversational_mode": True}},
)
print(reply.choices[0].message.content)

With Ollama, point the same client at http://localhost:11434/v1 and use the model name nemotron-3.5-bolt:e4b. To switch thinking off there, use Ollama's own Python package (pip install ollama), which has a think argument:

from ollama import chat

r = chat(model="nemotron-3.5-bolt:e4b",
         messages=[{"role": "user", "content": "Do you fix cracked back glass?"}],
         think=False)
print(r.message.content)

Keep max_tokens generous when thinking is on. Reasoning counts against it, and if it runs out mid-thought you get an empty answer. The official code samples allow 16,000 tokens, which you can lower for voice once you know your typical answer length.

Tool calls: building a voice assistant or game character

Tool calling is what Bolt E4B was trained hardest on: deciding whether to call a function, when in the conversation to call it, and how to talk to the user while it runs. Send your functions in the standard tools array and read the model's choices from message.tool_calls:

tools = [{
    "type": "function",
    "function": {
        "name": "check_stock",
        "description": "Count how many units of a spare part are on the shelf right now",
        "parameters": {
            "type": "object",
            "properties": {"part": {"type": "string",
                                    "description": "Part name, for example 'iPhone 17 screen'"}},
            "required": ["part"],
        },
    },
}]

reply = client.chat.completions.create(
    model="nemotron-bolt",
    messages=[{"role": "user", "content": "Have you got a screen for an iPhone 17?"}],
    tools=tools,
    max_tokens=2048,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(reply.choices[0].message.tool_calls)

Your code runs check_stock, sends the result back as a tool message straight after the assistant's tool call, and the model turns it into a sentence. Three rules save hours of debugging:

• A tool message must follow its tool call immediately. The template does not accept "stray" tool results that arrive later in the conversation.
• Slow tools need a two-step pattern. If a task finishes later (building a quote, booking a slot), reply at once with a tool message whose content says the result is pending, for example {"status":"pending","task_id":"task_1"}. The model keeps chatting. When the result is ready, send it as a user message that starts with "TOOL UPDATE:" and contains the finished result.
• Describe tools truthfully. A function that drops an item near the player must not be described as putting it in their inventory. The model plans from your descriptions, so a description that overpromises produces confident wrong answers.

The prompting style this model was trained for is worth copying too:

• Put the role, rules and personality in the system prompt under clear headings such as ROLE, PERSONALITY and TASK RULES.
• Write instructions as conditions and actions ("After the stock check, say the count and the price") and use explicit limits ("one or two sentences") instead of "be brief".
• Send changing facts (the shelf count, a game's world state) in a block marked CURRENT STATE, as a separate user or system message, and explain that it is context, not something to read aloud. A state update that depends on a tool goes after the tool result.
• Keep ownership clear. "Unknown" is not the same as "empty", and a part on the supplier's site is not a part on your shelf.
• Never insert fake assistant turns or tool calls into the history. The model treats them as things it really did.

For speech, Bolt E4B is the middle of a three-part chain. A speech-to-text model turns the customer's words into text, Bolt E4B decides what to say and which tools to call, and a text-to-speech voice reads the reply. Because Bolt E4B needs only 3 to 5 GB of VRAM, a 12 GB card has room for the speech models beside it, which is the "headroom" NVIDIA designed for. If your assistant should remember regulars between visits, our guide to local agent memory with Mem0 and Ollama walks through one way to do it. If you script with Transformers' apply_chat_template directly, pass tool-call arguments as a Python dictionary, not a JSON string. The template rejects strings (see errors).

Can Bolt E4B see images? Ollama and NVIDIA disagree

Google's Gemma 4 E4B accepts images and audio. NVIDIA ships Bolt E4B as text in, text out, its vLLM commands switch image, video and audio inputs off, and LM Studio's page says it accepts text input. Ollama, though, tags the model as taking text and image input and bundles a 0.99 GB image projector with it. Unsloth's repository has projector files too.

Both can be true at once. NVIDIA trained and evaluated text only, while the vision encoder inherited from Gemma 4 is still compatible, and the chat template still has the placeholders for images and audio. So an image may "work" in Ollama, but NVIDIA has not tuned or tested what happens. If image understanding matters to your project, use a model trained for it, such as Nemotron 3.5 Super VL or the 74B Bolt, and use E4B for the conversation. That is the same split NVIDIA uses in its robot demo, where a separate vision model feeds Bolt E4B a description of the scene.

Bolt E4B vs Gemma 4 E4B vs Qwen 3.5 4B vs Nemotron 3 Nano 4B

NVIDIA benchmarked Bolt E4B against the model it was built from, against Qwen 3.5 4B, and against its own previous small model, Nemotron 3 Nano 4B. NVIDIA reports these scores from its own test harness at full precision (BF16), with thinking on for most tests:

The pattern is clear. Bolt E4B wins where conversation and character matter: staying in a role, obeying the system prompt over a pushy user, following precise instructions, and not making up tools. Qwen 3.5 4B is stronger at long documents and multi-step agent tasks. Against NVIDIA's previous small model, Nemotron 3 Nano 4B, the jump is large on almost every row (RoleBench goes from 12 to 67), which supports NVIDIA's claim of roughly 40 percent higher average scores. NVIDIA also reports that Bolt E4B writes up to 80 percent fewer output tokens than peers with reasoning on, which matters for speed on small hardware.

One more curiosity: the 4-bit Q4_K_M file scores higher than full precision on 7 of the 21 tests NVIDIA ran (MultiChallenge 55.60 vs 53.10, RoleBench 71.21 vs 67.05, ToolSandbox 74.13 vs 68.77, RelyToolBench 69.29 vs 63.16, plus three smaller gains). Quantization does not make a model smarter. Differences of a few points sit within normal run-to-run noise, and NVIDIA calibrated the 4-bit file on the model's own training conversations. The useful conclusion is that you lose very little by running the small file. "So the cheap version is not the worse version," Jake said. "Not by any number they could find," Ethan said.

If you are choosing between them: pick Bolt E4B for assistants, characters and anything that must keep a persona and call tools; Qwen 3.5 4B for reading long documents or multi-step agent workflows; and Gemma 4 E4B if you need its image and audio input with Google's tuning. Our Gemma 4 guide covers that one.

Is there a Nemotron 3.5 Bolt E4B API?

Yes. Unlike the 74B Bolt, which has no hosted API, E4B launched on NVIDIA's API catalog the same day. The model ID is nvidia/nemotron-3.5-bolt-e4b, and the endpoint is OpenAI-compatible at https://integrate.api.nvidia.com/v1. Create a key on build.nvidia.com (keys start with nvapi-), then change two lines of the Python example above:

client = OpenAI(base_url="https://integrate.api.nvidia.com/v1",
                api_key=os.environ["NVIDIA_API_KEY"])
# model="nvidia/nemotron-3.5-bolt-e4b"

The catalog is the quickest way to try the model before downloading 6 to 8 GB, or to test your prompts from a machine without a GPU. Check the model's catalog page for the current usage limits. For a product, the point of a 4 GB model is to run it yourself, on the device, with no per-request cost and no data leaving the building.

License: is Bolt E4B free for commercial use?

Yes. NVIDIA releases Bolt E4B under the OpenMDW License Agreement, version 1.1, and states that the model is ready for commercial and non-commercial use. Because it is post-trained from Gemma 4, Google's Gemma 4 model itself is covered by the Apache License 2.0. OpenMDW is written for whole model releases (weights, code and data together), and it lets you run, modify, fine-tune and ship the model. Read the full text before you build it into a product, as you would with any license. NVIDIA publishes the BF16 weights and its NeMo RL training recipes so that you can fine-tune it on your own conversations. The downloads cost nothing, and running locally costs only your electricity.

Bolt E4B errors and fixes

llama.cpp: unknown model architecture: 'gemma4'

Your llama.cpp build predates Gemma 4 support. Download a current release (b11838 or newer on October 23) or update the runtime in LM Studio. An outdated Ollama can fail to load it too, so update Ollama as well.

curl: connection refused on localhost:8080

Since llama.cpp v0.7.0, llama-server defaults to port 9931. Either connect to localhost:9931 or start the server with --port 8080.

chat_template: tool_calls[].function.arguments must be a JSON object (mapping), not a string

You passed the assistant's tool-call arguments as a JSON string (the way the OpenAI API returns them) into the chat template directly, for example with Transformers' apply_chat_template. Run json.loads() on the arguments first. Servers such as llama-server and vLLM normally do this for you.

Replies contain "thought" text or the answer is empty

Your app is not separating the reasoning from the answer, or max_tokens ran out during the reasoning. Use --jinja in llama-server and --reasoning-parser gemma4 in vLLM, raise max_tokens, or switch thinking off.

It thinks in vLLM even though I wanted direct answers, or never thinks

The template defaults to thinking off when nothing is sent, and the official samples always send enable_thinking. Pass chat_template_kwargs with enable_thinking set explicitly in every request.

Ollama: model requires more system memory than is available

Ollama's tag is 7.9 GB because it bundles the image projector and drafter. Close other apps, or pull a smaller Unsloth GGUF with ollama run hf.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF:UD-Q3_K_XL.

It was fast at first and slowed down later

The conversation grew. Time to first token rises with every token you resend, so keep the system prompt and tools lean, trim old turns, and reuse the server's prompt cache.

My GPU is idle while Bolt E4B runs

The runtime fell back to the CPU. Check nvidia-smi, then ollama ps or the llama.cpp log for how many layers were offloaded, and add -ngl 99 in llama.cpp.

Images are ignored or described wrongly

NVIDIA trained Bolt E4B for text only. The image path comes from Gemma 4 and is untested for this model. Use a vision model for images.

Nemotron 3.5 Bolt E4B: frequently asked questions

Two weeks later, the counter in Jake's shop answered its first stock question on its own. A customer asked about a screen, the assistant checked the shelf count, said "Yes, we have two, and the repair takes about forty minutes," and the customer sat down to wait. Jake asked Ethan what it had cost. "A graphics card we already had," Ethan said, "and an afternoon reading what 4 GB really meant." If your own setup does something this page did not predict, please tell us. Guides like this stay right because readers write in.

Revision note. Written October 23, 2026, the day NVIDIA released Bolt E4B, with the files, app tags and commands of that day. "Don't judge a book by its cover," the old saying goes, and with this model you should not judge it by its name either.