Run Nemotron 3.5 Super VL Locally on Windows 11, Mac and Kali Linux: Requirements, GGUF Sizes From 46 GB, Ollama, LM Studio, Images and Video, the Free API and SageMaker JumpStart

Nemotron 3.5 Super VL is NVIDIA's new open vision-language model, released on October 15, 2026: 120 billion parameters with only 12 billion active per token, a 1-million-token context, and it reads images, screenshots, documents and short videos as well as text. Because so little of it is active at once, it runs on hardware a serious hobbyist already owns: the 4-bit GGUF is 86.91 GB, the 2-bit is 57.79 GB, and the smallest is 46.14 GB, so a 128 GB Mac, an NVIDIA DGX Spark, or a 64 GB Windows or Kali PC with a 16 to 24 GB graphics card can run it with llama.cpp v0.7.0 or newer, LM Studio or Ollama. One trap catches almost everyone on the first try: in Ollama the model is called nemotron-3.5-super, without "VL," so ollama run nemotron-3.5-super-vl finds nothing. And if your machine is smaller, the API is close to free: $0.10 per million input tokens and $0.40 per million output on OpenRouter, with a free tier on top. Below: every file size and what it needs, the exact commands for Windows 11, Kali Linux and Mac, how to send it images and video, how to switch its thinking off, the AWS route, and the fixes for the errors people hit first.

Jake's computer shop gets a steady stream of paper: supplier invoices, warranty cards, photos of serial-number stickers taken at bad angles. Ethan, his developer friend, had been looking for a model that could read all of it on the shop's own workstation, without sending customer details to anyone. On Thursday NVIDIA released one built for exactly that kind of document work. By Friday evening it was reading invoices on the shop PC. This page is the setup Ethan used, including the twenty minutes he lost looking for the wrong model name.

New to running models on your own machine? Our free plain-English series explains weights, quantization and VRAM in an hour, and our tool comparison helps you pick between Ollama, LM Studio and the rest. This page gets you from download to a working vision chat.

What Nemotron 3.5 Super VL is, in plain English

Nemotron is NVIDIA's family of open models. The 3.5 generation has a small fast model (Lightning, 30B), edge models for small devices (Bolt), and this one, Super, which is the large general model. "VL" means vision-language: as well as text, it reads pictures. NVIDIA built it for what it calls document intelligence (reading invoices, forms, charts and scanned pages), GUI understanding (reading screenshots of apps and websites, which is what computer-use agents need), video question answering, and agents that combine all of that with tool calls.

Two design choices explain why it runs on a desk. First, it is a mixture of experts: of its 120 billion parameters, only 12 billion work on any given token, picked from 512 experts per layer. You still need memory for the whole model, but the speed is closer to a 12B model than a 120B one. Second, it is a hybrid of Mamba-2 and Transformer layers. Mamba layers carry a fixed-size memory of everything they have read instead of a growing cache, which is how a model this size can offer a million tokens of context without needing a server to hold it.

The license matters if you plan to use it at work. OpenMDW 1.1 allows commercial and non-commercial use, which puts it in the permissive group with Apache-style licenses, not the "free unless you are big" group. Read the full text at openmdw.ai before you build a product on it, as you would with any license.

If you used Nemotron 3 Super, the model this replaces, the shape will feel familiar: the same 120 billion parameters with 12 billion active. What is new is the "VL" half. Nemotron 3 Super read text only, while 3.5 Super VL reads images and video, and its context grows to a million tokens (OpenRouter serves 3 Super with 262,144). In Ollama, the old nemotron-3-super entry is a cloud listing, while nemotron-3.5-super is a download you run on your own machine. The smaller sibling, Nemotron 3.5 Lightning (30 billion parameters, 3 billion active, text only), is the one to pick if you have 32 GB and do not need vision.

"So it's a 120B model that thinks like a 12B one," Jake said. "It remembers like a 120B one," said Ethan. "That's the part your RAM pays for."

Will it run on your computer? The memory table

The one rule for running a big model locally: the file has to fit in your graphics card's VRAM plus your system RAM, with a few gigabytes spare for the context, the vision file and the operating system. On a Mac or a DGX Spark, the GPU and the CPU share one pool of memory, so you just compare the file to that pool. On a Windows or Linux PC, llama.cpp can keep the experts in system RAM and the rest on the graphics card, which is how a 64 GB PC with a 16 GB card runs a 58 GB file.

These are the GGUF files that matter, with what each needs and the fastest speed it could reach when its experts sit in desktop RAM. The speed column is a ceiling (memory bandwidth divided by what each token has to read), and real speed is lower, especially while the model is reading a long prompt or an image.

Add 1.73 GB for the vision file (the "mmproj") if you want to send images; text-only use does not load it. To redo the ceiling for your own machine, divide the file size by 120 billion and multiply by the 12 billion active parameters: UD-Q4_K_XL reads about 8.7 GB per token, UD-Q2_K_XL about 5.8 GB. Dual-channel DDR5-5600 moves 89.6 GB a second, so 89.6 ÷ 5.8 gives roughly 15 tokens a second at best. Unified-memory machines (Macs, DGX Spark, Strix Halo) move data several times faster, and speed rises with them.

In plain terms:

• 32 GB or less: not this model. Use the API, or look at NVIDIA's smaller Nemotron 3.5 Bolt models, which launched alongside Super VL for edge devices.
• 64 GB RAM with a 12 to 24 GB graphics card: yes. UD-Q2_K_XL is the sweet spot. This is the setup most of our readers have, and it is a big deal that a 120B vision model fits it.
• 96 GB RAM, or 64 GB plus a 24 GB card: UD-Q3_K_XL or the LM Studio Q4_K_M.
• 128 GB unified memory (Mac, DGX Spark, Strix Halo): the recommended UD-Q4_K_XL with room for long documents.

Our honest hardware tiers page explains which laptops and desktops land in which row, and why RAM is now the spec that matters most for local AI.

The quickest route: Ollama (and the name that trips everyone up)

If you already use Ollama, this is the fastest way to a working chat, and it handles the vision file for you. The model is in the official library as nemotron-3.5-super. Search Ollama for "nemotron-3.5-super-vl" and you find nothing, because the library drops the "VL." It is the same model: the library entry takes text and images, has the 1-million-token context, and has the thinking switch.

1. Update Ollama first. New architectures need new engines. On Windows and Mac, quit Ollama and install the latest version from ollama.com; on Kali, rerun curl -fsSL https://ollama.com/install.sh | sh. Check with ollama --version.
2. Check your free space. The default tag is about 80 to 90 GB depending on your system, so make sure the drive that holds Ollama's models has 100 GB free. To move the models to another drive on Windows, set OLLAMA_MODELS (for example setx OLLAMA_MODELS "D:\ollama") and restart Ollama.
3. Pull and run it:

ollama run nemotron-3.5-super

The library has these tags, so you can choose the size yourself:

Notice the gap: the official library starts at about 80 GB. If you have a 64 GB PC, the library tags are too big, and you want Unsloth's smaller files instead. Ollama can pull those straight from Hugging Face:

ollama run hf.co/unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:UD-Q2_K_XL

To send an image in the Ollama terminal, put the file path in your message: Read the invoice total and the due date from C:\scans\invoice-0417.png. In the Ollama desktop app, drag the image into the chat. If Ollama answers that the model "requires more system memory than is available," the tag is bigger than your free RAM; our Ollama memory error guide walks through the numbers, and the fix here is a smaller tag. If it runs but feels painfully slow, check that it is using your graphics card at all with our Ollama GPU guide.

"Twenty minutes," Ethan said, "searching for a model that was on the front page the whole time, one word shorter."

Run it on Windows 11 with llama.cpp

llama.cpp gives you the most control on a PC: you choose the exact file, decide what goes on the graphics card, and get a local web chat plus an OpenAI-compatible API. Nemotron 3.5 Super VL support arrived in llama.cpp v0.7.0 on October 15, so any build from that release on works.

1. Check your memory. Press Ctrl + Shift + Esc, open Performance, and note the total under Memory and the "Dedicated GPU memory" under GPU. Add them and pick a file from the memory table that fits with about 8 GB to spare.
2. Download llama.cpp. From the llama.cpp releases page on GitHub (github.com/ggml-org/llama.cpp/releases), take v0.7.0 or any later build, in the Windows x64 zip for your card: CUDA for NVIDIA (plus the matching CUDA runtime zip from the same release), Vulkan for AMD or Intel. Unzip both into one folder such as C:\llama.
3. Choose where the model is stored. llama.cpp caches downloads on C: by default. To use another drive, run setx LLAMA_CACHE "D:\models" in PowerShell, then open a new PowerShell window.
4. Start the server with the command below.
5. Open the chat at http://127.0.0.1:8080 once the log says it is listening, and drag an image in to test vision.

cd C:\llama
.\llama-server.exe -hf unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:UD-Q2_K_XL `
  --cpu-moe -ngl 99 --moe-cache-mib 6000 -c 16384 --jinja `
  --temp 0.6 --top-p 0.95 --host 127.0.0.1 --port 8080

What each part does:

• -hf ...:UD-Q2_K_XL downloads that file and the matching vision file from Hugging Face. Swap the file name for any row of the memory table.
• --cpu-moe keeps the expert weights in system RAM and -ngl 99 puts everything else on the GPU. This split is what lets a 16 GB card run a 58 GB model.
• --moe-cache-mib 6000 is new in v0.7.0: it keeps the most recently used experts on the GPU in a 6,000 MB cache, and the llama.cpp team reports up to about 2.2x faster generation when experts live in RAM. Give it most of the VRAM left after loading (about 6,000 on a 16 GB card, 12,000 on 24 GB) and lower it if you get an out-of-memory error.
• -c 16384 is a 16K-token context, enough for several pages of documents. Raise it once things work.
• --jinja uses the chat template inside the file, which is what makes thinking on and off, tool calls and image messages work properly.
• --temp 0.6 --top-p 0.95 are the settings meant for vision work; use --temp 1.0 for pure text.
• --port 8080 matters: since v0.7.0 the server listens on port 9931 by default. Keep 8080 if your other tools expect it, or drop the flag and use 9931.

If you have a bigger card, try --n-cpu-moe 30 in place of --cpu-moe: it keeps only 30 layers' experts in RAM and moves the rest onto the GPU. Lower the number step by step until VRAM runs out; every layer moved makes generation faster.

Run it on Kali Linux

On Kali, build llama.cpp from source. It takes about ten minutes and gives you a binary tuned to your CPU. For an NVIDIA card:

sudo apt update
sudo apt install -y build-essential cmake git libcurl4-openssl-dev nvidia-cuda-toolkit
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j$(nproc)
./build/bin/llama-server --version

For AMD or Intel graphics, use -DGGML_VULKAN=ON instead and install the Vulkan development packages; for CPU only, drop the flag. If you cloned llama.cpp before October 15, run git pull and rebuild, or the model will not load. Then start the server:

export LLAMA_CACHE=/mnt/models
./build/bin/llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:UD-Q2_K_XL \
  --cpu-moe -ngl 99 --moe-cache-mib 6000 -c 16384 --jinja \
  --temp 0.6 --top-p 0.95 --host 127.0.0.1 --port 8080

Kali users who live inside a virtual machine: a VM only sees the RAM you give it, and almost no desktop hypervisor will hand a guest 64 GB with a GPU passed through. Run llama.cpp on the host or on bare-metal Kali, and connect from the VM to the host's address and port. The newer one-line installer, curl -LsSf https://llama.app/install.sh | sh, installs a prebuilt llama command if you would rather not compile; it works on any modern Debian-based system.

Run it on a Mac

A Mac with 128 GB of unified memory is one of the best machines for this model: the whole 86.91 GB recommended file sits in memory the GPU can read directly, with room for long documents. With 96 GB, use UD-Q3_K_XL; with 64 GB, UD-IQ1_M is the only file that fits, and it is tight.

Two routes work. Ollama's MLX build is the easy one:

ollama run nemotron-3.5-super:120b-a12b-nvfp4     # 80 GB, MLX build for Apple silicon

llama.cpp gives you file choice and the same server as on a PC:

brew install llama.cpp     # or: brew upgrade llama.cpp (needs v0.7.0 or newer)
llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:UD-Q4_K_XL \
  -ngl 99 -c 32768 --jinja --temp 0.6 --top-p 0.95 --port 8080

No --cpu-moe on a Mac: memory is already shared, so the whole model goes on the GPU. If loading stops with a Metal allocation error on a 128 GB machine, raise the GPU's memory limit for this session with sudo sysctl iogpu.wired_limit_mb=110000 and try again; it resets on restart.

Run it in LM Studio

LM Studio is the friendliest option if you prefer clicking to typing. Search for "Nemotron-3.5-Super-VL" in the Discover tab; the LM Studio community upload offers a Q4_K_M at 70.10 GB and an NVFP4 at 75.24 GB, both with the 1.73 GB vision file, and Unsloth's files from 46 GB up appear in the same search. LM Studio shows a memory estimate for each file before you download, and on this model it is worth believing: if the estimate is red, pick a smaller file.

LM Studio runs GGUF files on its own bundled copy of llama.cpp, so it needs a runtime that includes v0.7.0. If the load fails, open the runtime settings (the gear next to the model loader), update the llama.cpp runtime, and load again. Our LM Studio "failed to load model" guide covers the other causes. Once loaded, drag an image into the chat to test vision, and turn on LM Studio's local server if you want other apps to use the model through http://localhost:1234/v1.

Sending it images, screenshots and video

Vision is the reason to pick this model over a text-only one, so here is how to use it outside a chat window. All three local tools, and the API, accept images in the same OpenAI message format.

One image from the command line

llama.cpp includes a small multimodal command-line tool for quick tests. It downloads the model and vision file the same way as the server:

llama-mtmd-cli -hf unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF:UD-Q2_K_XL \
  --cpu-moe -ngl 99 --temp 0.6 --top-p 0.95 \
  --image invoice-0417.png -p "List the supplier, invoice number, total and due date as JSON."

Images from Python, against your local server or the API

This works against llama.cpp on port 8080, LM Studio on 1234, Ollama on 11434, or OpenRouter; only the base URL and key change:

pip install openai

import base64
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="local")   # llama.cpp
img = base64.b64encode(open("invoice-0417.png", "rb").read()).decode()

r = client.chat.completions.create(
    model="nemotron",            # llama.cpp ignores the name; Ollama wants "nemotron-3.5-super"
    temperature=0.6, top_p=0.95,
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "Read this invoice. Return supplier, number, total and due date as JSON."},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img}"}},
    ]}],
)
print(r.choices[0].message.content)

Send one image per message. That is how the model was served in NVIDIA's own setup, and it is the reliable way on local tools too; for a ten-page document, send the pages one at a time and collect the answers. Higher-resolution scans read better, but a phone photo straight-on, in good light, reads fine. For screenshots of apps (the GUI-understanding use), crop to the window you care about; a full 4K desktop wastes most of the image on wallpaper.

Video

The model reads MP4 video up to two minutes long. On a server it samples two frames a second, up to 256 frames, and for 1080p video one frame a second and 128 frames. Video support in desktop tools trails images: llama.cpp and LM Studio handle images reliably today, and for video the dependable route is vLLM on a GPU server or the API. If you only need to ask about a few moments, extracting frames with ffmpeg -i clip.mp4 -vf fps=1 frame-%03d.png and sending the important ones as images works everywhere.

Jake slid a crumpled warranty card under the shop's document camera. The JSON came back with the serial number right and the purchase date wrong, read off a faded stamp. "Better than me," Jake said. "Faster than you," said Ethan. "Check the dates."

Thinking on or off, and the settings that matter

Nemotron 3.5 Super VL is a reasoning model: by default it thinks before it answers, writing out a reasoning trace and then the reply. Thinking makes it better at hard problems (math, multi-step coding, tricky documents) and slower at everything, because the thinking tokens come out of the same budget as the answer. For reading an invoice, you usually want it off. There are three settings: thinking on, thinking at medium effort, and thinking off.

• Ollama: start with ollama run nemotron-3.5-super --think=false, or type /set nothink inside a chat (/set think turns it back on).
• llama.cpp server: add --chat-template-kwargs '{"enable_thinking": false}' when you start the server, or send it per request (below).
• Any OpenAI-compatible call (llama.cpp, vLLM, many providers): add the chat template settings to the request.

# thinking off
extra_body={"chat_template_kwargs": {"enable_thinking": False}}
# thinking on, medium effort
extra_body={"chat_template_kwargs": {"enable_thinking": True, "medium_effort": True}}

Pass that extra_body into client.chat.completions.create(...) from the Python example above. Two sampling profiles are recommended: temperature 1.0, top_p 0.95 for text, and temperature 0.6, top_p 0.95 for vision. If answers get cut off in thinking mode, raise max_tokens: the reasoning and the answer share it, and a long think can use most of the budget before the reply starts.

DGX Spark and GPU servers: the official NVFP4 build with vLLM

NVIDIA's DGX Spark, the small desktop box with 128 GB of unified memory, is one of the machines the official NVFP4 weights (75.1 GB) are validated on, alongside a single H200, two H100s or one GB200. If you have a Spark, this is the fastest local setup, because NVFP4 is the number format its Blackwell GPU runs natively. It uses vLLM v0.32.0 or newer and a small speculative "LiLiCorr" drafter that guesses several tokens ahead for the main model to check:

vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
  --served-model-name nemotron-3.5-super-vl \
  --max-num-seqs 1 \
  --trust-remote-code \
  --gpu-memory-utilization 0.75 \
  --max-num-batched-tokens 8192 \
  --speculative_config.model nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr \
  --speculative_config.num_speculative_tokens 7 \
  --speculative_config.method dflash \
  --mamba-backend flashinfer \
  --mamba-ssm-cache-dtype float16 \
  --enable-mamba-cache-stochastic-rounding \
  --mamba-cache-philox-rounds 5 \
  --enable-mamba-shared-prefix-checkpoint \
  --video-pruning-rate 0.5 \
  --media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}' \
  --block-size 64 \
  --prefix-match-unit 64 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser nemotron_v3

The flags worth understanding: --max-num-seqs 1 serves one request at a time, the right choice for a personal box; --tool-call-parser qwen3_coder and --reasoning-parser nemotron_v3 return tool calls and thinking in the right fields; and the --video-pruning-rate and --media-io-kwargs pair sets how video is sampled. The full-precision BF16 weights (248.9 GB) are for four H100s or two GB200s, served with --tensor-parallel-size 4.

NVIDIA claims up to five times the agentic throughput of other open models its size on a single GPU. That is a vendor number about servers under load; on a desk, what you will notice is that 12 billion active parameters keep it responsive.

Nemotron 3.5 Super VL on AWS: SageMaker JumpStart

The model reached Amazon SageMaker JumpStart on October 16. It is not on Amazon Bedrock, so on AWS you deploy it to an endpoint and pay for the instance by the hour, not per token. Two deployments are offered: the NVFP4 build on G7 instances, which carry NVIDIA RTX PRO 4500 Blackwell GPUs, and the BF16 reference build on ml.p5en.48xlarge (eight H200s). If SageMaker is new to you, our plain-English SageMaker guide explains endpoints and why they bill while idle.

Each RTX PRO 4500 Blackwell on a G7 instance has 32 GiB of memory, so the 75 GB NVFP4 build needs at least the four-GPU size. These are the EC2 on-demand rates for the same instances in US East (N. Virginia); the SageMaker ml. rate for an endpoint is higher, so treat them as the floor:

One day of a g7.24xlarge is $342.16 at the floor rate. At OpenRouter's $0.40 per million output tokens, the same money buys about 855 million output tokens. Run your own endpoint when your documents must stay inside your AWS account, or when a team keeps it busy all day; for one person, the API is cheaper by orders of magnitude.

To deploy:

1. Request the quota. New accounts usually have zero quota for these endpoint instances. In Service Quotas, under Amazon SageMaker, request "ml.g7.24xlarge for endpoint usage" (or the p5en equivalent) in your Region. Our ResourceLimitExceeded guide shows the request step by step.
2. Open SageMaker Studio, go to JumpStart, and search for "Nemotron 3.5 Super VL."
3. Pick the NVFP4 or BF16 deployment, accept the license, choose the instance, and deploy. The endpoint takes several minutes to start.
4. Test it from the Studio notebook the model page offers, sending one image with your question.
5. Delete the endpoint as soon as you finish. It bills by the hour whether you use it or not.

You can do the same from code with the SageMaker Python SDK's JumpStart model class, which is the better route once you deploy it more than once.

The cheap way: the API (including a free tier)

For most people with a laptop, this is the answer, and it is unusually cheap for a model this capable. OpenRouter serves nvidia/nemotron-3.5-super-vl at $0.10 per million input tokens and $0.40 per million output with a 1,048,576-token context, and a :free variant at no cost, with a 1,000,000-token context and the rate limits free models carry. Reading a one-page invoice costs a fraction of a cent.

from openai import OpenAI
import os

client = OpenAI(base_url="https://openrouter.ai/api/v1",
                api_key=os.environ["OPENROUTER_API_KEY"])

r = client.chat.completions.create(
    model="nvidia/nemotron-3.5-super-vl",     # or "nvidia/nemotron-3.5-super-vl:free"
    messages=[{"role": "user", "content": "Explain what a Mamba layer does, in two sentences."}],
    temperature=1.0, top_p=0.95,
)
print(r.choices[0].message.content)

Images use the same message format as the vision example. The practical pattern for a small business: build the tool against the free tier while you test, switch to the paid model name for real work (free tiers can be slow or busy at peak times), and keep the code pointed at an OpenAI-compatible URL so you can move it to your own machine later by changing one line. Free endpoints may log prompts for the provider's own use; for customer documents, use the paid endpoint or your own hardware.

How good is it? The benchmarks against Qwen3.5 and DeepSeek

These are NVIDIA's published scores for the full-precision model, set against the two open models it is most often compared with. They are not independent results yet:

The honest summary: against the Qwen model of the same size, Nemotron is far ahead on coding problems (11.5 points on LiveCodeBench) and on tool-using agent tasks (nearly 14 points on tau2 retail), level on reading text in images and on software repair, and behind on multilingual knowledge and visual math. DeepSeek-V4-Flash beats it on most text tests, but DeepSeek's model has no eyes and needs several times the memory. If your work is English documents, screenshots and agents on hardware you own, this is one of the strongest choices that fits a 64 to 128 GB machine. If it is mostly non-English text, the Qwen family is still worth comparing; our Qwen3.8 local guide covers that side.

Errors and fixes

The problems people hit in the first days, with what fixes each one.

Ollama: "pull model manifest: file does not exist"

You asked for nemotron-3.5-super-vl. The library name is nemotron-3.5-super. If the correct name also fails, update Ollama; an old version does not know the model's architecture.

llama.cpp: "unknown model architecture" or the file will not load

Your llama.cpp is older than v0.7.0. Download a newer release or git pull and rebuild, then check llama-server --version. LM Studio users: update the llama.cpp runtime inside LM Studio.

Out of memory while loading, or the PC freezes

The file plus context is larger than your free memory. In order: add --cpu-moe, lower --moe-cache-mib or remove it, cut the context to -c 8192, close other apps, then move one row down the memory table. If Windows starts swapping to disk and everything freezes, end the process in Task Manager.

It ignores the image, or says it cannot see images

The vision file did not load. With -hf, llama.cpp downloads it automatically; check the startup log for the mmproj file. If you loaded a local file with -m, add --mmproj pointing at the mmproj GGUF from the same repository. In LM Studio, make sure the download included the mmproj file.

Answers stop halfway, or only the thinking comes back

The reasoning used up max_tokens. Raise the limit, switch to medium effort, or turn thinking off for routine document reading.

Connection refused on localhost:8080

Since llama.cpp v0.7.0 the server's default port is 9931. Either open http://127.0.0.1:9931 or start the server with --port 8080.

Very slow on a PC with a good graphics card

Check the GPU is in use (Task Manager > Performance > GPU, or nvidia-smi on Kali). If it shows no load, you have the CPU build of llama.cpp. If it is in use, add --moe-cache-mib, or replace --cpu-moe with --n-cpu-moe and a number that moves some experts onto the card.

SageMaker deploy fails with ResourceLimitExceeded

Your quota for the endpoint instance is zero or too low. Request it for the exact instance type in the Region you deploy to, wait for approval, then deploy again.

Frequently asked questions

By the end of the week the shop's workstation was reading every supplier invoice that came through the door, thinking switched off, one page at a time, into a spreadsheet Ethan had set up. It got the totals right every time and the dates right most of the time, so Jake checked dates and nothing else. "It's an extra pair of eyes," Jake said. "A tireless pair," said Ethan, "that never takes the invoices home."

Revision note. Written October 17, 2026, two days after release. "Many hands make light work," the proverb says; a model that reads the paperwork is a few more hands.