Run vLLM d3 (Decision 3.0) Locally on Windows 11, Mac and Kali Linux: Free Decision Models That Watch Video, Install and the Port 9931 Catch

d3, short for Decision 3.0, is a family of six free, open decision models from the vLLM Semantic Router project, published on Hugging Face on October 10, 2026 under the Apache 2.0 license. They never write a sentence. You hand one a customer message, a photo or a short video, plus questions such as "Is the screen cracked?" or "Which team should handle this?", and it returns a probability for every allowed answer, in about 7 to 55 milliseconds for text on a data-center GPU. The surprise is the pace. This is the project's third generation in 18 days, and the new 9B model, d3-flash, already outscores last week's 27B Decision 2.0 (59.0 against 55.9 on the Jev Decision Index), while the full 27B d3 sits at the top of that public board at 64.7, ahead of TypeSafe's hosted Jev at 60.1. Sizes run from 0.59B, under 1 GB on disk, to 26B, and every one of them installs locally on Windows 11, Mac or Kali Linux. One timing detail decides whether your first try works: llama.cpp learned to run d3 on October 12, 2026, in build b11572, and three days earlier its server quietly moved from port 8080 to port 9931.

Jake's phone repair shop has run on a decision model since last week. Liquid AI's d1-3B sorts his customer messages and tags cracked-screen photos on the shop laptop, and his hour of evening sorting has shrunk to fifteen minutes. Then the videos started. A customer whose screen flickers sends a ten-second clip, because no photo can show a flicker. Another sends her phone stuck restarting on its logo, over and over. "The model can't watch these," Jake told Ethan, his developer friend. Ethan had spent Saturday morning reading about something called d3. "This one watches video," he said. "And it's Apache 2.0, so the license never asks how big your shop gets." This page covers what d3 is, what its benchmarks really show (including the one size that loses), the six sizes and the hardware each needs, how to install it locally with llama.cpp on Windows, Mac, Linux and Kali, the vLLM Semantic Router route that handles video, Python, the Core ML versions for Macs, running it on AWS, and the small mistakes that make it look broken.

Decision models are new enough that most people have never used one. If that is you, our guide to decision models in Ollama explains the idea gently with two small models. You can also start right here; everything below begins from zero.

What vLLM Decision 3.0 (d3) is, in plain English

Most AI models you have met are writers. They produce the next word, then the next, until an answer appears. That is lovely for an email and clumsy for sorting. Ask a chat model "Is this message a repair request? Answer yes or no," and you might get "Yes," "Yes.", "Yes, it looks like one," or a paragraph you never asked for, and your code has to cope with every version.

A decision model is a sorting machine with a probability meter on the front. You give it two things:

• A state: the situation. A customer message, a JSON record, one or more photos, one or more videos, or a mix.
• Questions: one or more named questions, each with a fixed set of answers that you write yourself.

For every question, it hands back a probability for each answer you allowed. There is no writing step, and the usage report proves it: "output_tokens": 0 on every call. So it cannot answer outside your options, cannot ramble, and is quick enough to run on every message, photo or video your shop receives.

Every question has one of three types, the same three that llama.cpp's decision endpoint and the other open decision models use:

• noul: a yes-or-no question. You get one number, the probability of "yes." The odd name is simply the format's own word for yes-or-no.
• choice: pick one of several options you name and describe, such as "repair," "status question," "quote" or "spam." You get the winner, every option's probability (they add up to 1) and a confidence from 0 to 1, where 0 means all options looked equally likely.
• score: place the state on a scale of 2 to 10 levels you describe in words, lowest first, such as "can wait," "this week," "today." You get the expected level, which can land between two levels, plus the probability of each.

Three things set d3 apart from the decision models that came before it. It watches video, not just photos. It comes in six sizes built from one recipe, so you can prototype on a laptop and move to a server without rewriting a single question. And it is Apache 2.0, with no revenue ceiling and no separate business license.

There is one technical detail worth knowing early, because it explains a setting you will meet later. Most language models read strictly left to right, the way you read a sentence. d3's full-attention layers see the whole prompt at once, in both directions, more like glancing at a whole page. That is why llama.cpp has to swallow each d3 prompt in a single batch, and why the default batch size trips people up.

Who makes d3: the vLLM Semantic Router project

vLLM Semantic Router is an open-source project in the vLLM organization on GitHub, licensed Apache 2.0, started in August 2025 and followed by about 6,100 developers who have starred it. Its main job is routing: you give an AI agent one address to call, and the router picks or combines models for each request according to your rules for quality, speed, cost and where data may go. Decision models are the brains behind those choices, and the team publishes them for everyone.

The pace has been remarkable. Decision 1.0 arrived on September 22, 2026. Decision 2.0 reached number one on Hugging Face's trending collections on October 6. Decision 3.0, the d3 family, landed on October 10. The models were trained on AMD Instinct MI325X graphics processors, and every speed figure on this page comes from that same chip unless it says otherwise.

Jake: "Three versions in three weeks? So whatever I set up today is old by Christmas."

Ethan: "Think of it like a parts supplier that keeps improving the same screen. The connector stays the same. Your questions keep working; you just swap in the better model when you feel like it."

The six d3 sizes

"Text score" is the overall Jev Decision Index 0.3.1, explained in the next section. d3-lite and d3-edge start from the same 0.8B base; edge is the trimmed one. All six read text or JSON, photos and video, and all six, along with the Qwen models they are built on, carry the Apache 2.0 license. If the name Qwen3.8 sounds familiar, it is the same family as the chat model in our guide to running Qwen3.8 locally; d3 is a decision-making specialist built on top of it.

d3 benchmarks: what the numbers really say

The main yardstick for decision models is the Jev Decision Index, a public leaderboard named after TypeSafe's Jev, the hosted model that introduced the question format. Version 0.3.1 reports four numbers: the overall index, the public suite (140,178 requests, and d3 answered every one), same-skill tests, and tasks from new domains. The d3 rows are the vLLM Semantic Router team's own runs with the official scoring kit; the other rows are live leaderboard data from October 10. That is a fair method, with one honest caveat: the d3 numbers are the makers' measurements, not leaderboard submissions checked by anyone else.

Three things stand out.

Each d3 does the job of a model two or three times its size from a week earlier. The 4B d3-mini (54.2) beats Decision 2.0's 9B model (48.2), and the 9B d3-flash (59.0) beats Decision 2.0's 27B model (55.9). Against the same size, the gain is 4.4 points at the smallest and up to 11.5 points for d3-lite.

The big one leads the whole board. At 64.7, d3 sits ahead of Perplexity Decider v1.1 (62.8), Fastino GLiDE without thinking (60.2), TypeSafe's hosted Jev (60.1) and Torchcast Decision 27B (59.9). Torchcast is interesting: on the public suite alone it nearly ties d3 (65.1 against 65.5), but on new-domain tasks it drops to 50.8 while d3 holds 56.6. If your questions look nothing like a benchmark, and a phone shop's rarely do, that last column is the one to watch.

One size loses, and the makers printed it themselves. d3-nano scores 37.2, while Liquid AI's d1-3B scores 40.1 on the same board. For text-only sorting at the small end, d1-3B is still the stronger model. d3-nano wins on everything around the score: it is a third smaller (2.21B against 3.12B), it reads video, and its license has no conditions, while d1's LFM Open License 1.0 stops being free for commercial use at $10 million a year in revenue. One more detail if you read our d1-3B guide: it quoted 48.57 on Decision Index 0.2.1. The board has since moved to 0.3.1, where d1-3B shows 40.1. Scores from different versions of the index cannot be compared with each other.

Photos: the vision board, and the receipt test

On the Jev Decision Index vision board 0.3.1, d3 scores 71.6, ahead of Perplexity Decider v1.1 (70.6) and JEV-27B-VL (69.6). Down the family: d3-flash 66.9, d3-mini 65.5, d3-nano 58.1, d3-lite 46.9 and d3-edge 35.6. The makers also ran well-known image benchmarks as decision questions; the ones marked † are approximate rebuilds of the original test:

Read that table the way a shopkeeper would. Reading a receipt or a form is close to perfect at every size down to the 2B d3-nano (97.3), so "Does this receipt list the phone the customer says it does?" is a job even a small model handles. Everyday photos and anything that needs real reasoning fall away quickly as the model shrinks. And meme moderation is weak across the whole family, 43.9 at best, so d3 is the wrong tool for policing images in a community.

d3 watches video: how it reads a clip

This is the headline feature, and it helps to know exactly what the model sees. d3 reads a video as a series of still frames:

• 2 frames per second, with at least 4 and at most 32 frames spread across the whole video.
• Each frame read at up to 200,704 pixels, which is exactly 448 × 448. Small, but plenty for "is the screen flickering?"
• MP4, WebM, MOV (QuickTime) and MKV (Matroska) files, several per request. Photos come first in the prompt, then videos, then your text.
• No sound. d3 reads text, images and video frames; there is no audio input. A crackling speaker is invisible to it.

The 32-frame ceiling matters more than it looks. A 10-second clip at 2 frames per second is 20 frames, so d3 sees every one. A one-minute clip would need 120 frames, so the model spreads its 32 across the minute, about one frame every 1.9 seconds. At the five-minute maximum that vLLM Semantic Router accepts, that becomes one frame every 9.4 seconds, and a one-second flicker can fall between two frames entirely. The practical rule: trim videos to the seconds that matter before you send them.

For a score, the makers use the Perception Test, a set of 19,140 multiple-choice questions about videos with three options each, so guessing scores 33.3:

The pattern is useful. "Semantics," knowing what is in the scene, stays strong even on small models. "Memory," keeping track of what happened earlier in the clip, falls fastest: 80.3 on d3 but 53.8 on d3-nano. A question like "Does the screen flicker at any point?" leans on memory, so for that kind of question start at d3-mini.

Video works through two of the three local routes: vLLM Semantic Router and Python. llama.cpp's decision endpoint accepts text and images but has no video field, so a clip sent there is simply not part of the request.

How fast is d3? Real latency numbers

Speed is the whole reason to use a decision model, so here are the makers' measurements: median times on one AMD Instinct MI325X, one request at a time.

Two patterns hide in there. First, pictures cost far more than words: even the tiny d3-edge needs 308 ms for a video against 6.7 ms for text, because preparing 20 frames is work no matter how small the model is. Second, the middle sizes are nearly as fast as the small ones on text. d3-mini answers a text request in 17.5 ms against 13.1 ms for d3-lite, with almost twice the accuracy. Unless memory is tight, d3-mini is the sweet spot.

On a Mac, an independent team at Fluid Inference timed the Core ML versions on a MacBook Pro with an M5 Pro chip and 24 GB of memory. A median request took 53 ms on d3-lite, 69 ms on d3-nano and 169 ms on d3-mini, two and a half to four times faster than PyTorch on the same machine. Their d3-lite demo sorted 1,000 sample pull requests at 0.12 seconds each, answering two questions per request. More on the Mac versions further down.

A laptop processor running llama.cpp will be slower than either of those. How much slower depends on your machine and the size you pick, so time a handful of your own requests before you promise anyone instant answers.

d3 files, sizes and the hardware you need

The llama.cpp team publishes ready-made GGUF files for all six sizes under the ggml-org name on Hugging Face, posted on October 12. Photos need a second, separate file called an mmproj, short for multimodal projector; text-only use does not. Sizes in GB:

A simple rule turns that into machines: add the model file and the projector, then allow a gigabyte or two of working space.

• d3-edge and d3-lite: about 1 to 1.3 GB of files. Any computer with 4 GB of free memory runs them, on the processor alone.
• d3-nano: about 2.9 GB at Q8_0. Fine on a laptop with 8 GB of RAM.
• d3-mini: about 5.5 GB at Q8_0. Comfortable on a 16 GB laptop, and fast with a graphics card of 8 GB or more.
• d3-flash: about 10.2 GB at Q8_0, which wants a 16 GB graphics card or a 32 GB machine; Q4_K_M brings it to about 7.2 GB for a 12 GB card.
• d3: about 20 GB at Q4_K_M, a tight fit on a 24 GB graphics card or a Mac with 32 GB of unified memory, and about 29.2 GB at Q8_0, which belongs on a 48 GB card or a 64 GB Mac.

Our laptop guide for local AI has the full RAM and graphics tiers if you are shopping. For the small d3 sizes, the honest answer is that the laptop you already own is enough.

Which file? For a decision model, where a small shift in probability can flip a borderline answer, Q8_0 is the sensible default; use Q4_K_M only when memory is genuinely tight, and compare a few borderline cases against Q8_0 when you do. One trap is waiting in the table: d3-nano, d3-lite and d3-edge have no Q4_K_M file at all. llama.cpp's -hf option looks for Q4_K_M by default and, when it is missing, falls back to the first file in the repository. Always add :Q8_0 to the name and you get exactly the file you meant.

How to install d3 locally with llama.cpp (Windows 11, Mac, Linux and Kali)

This is the local install most people want: one program, one command, and the model runs offline on your own machine. llama.cpp is the engine underneath Ollama and LM Studio, and its small web server, llama-server, has a decision endpoint called /v1/systemone that follows TypeSafe's System One request format. The same endpoint serves other open decision models, so code you write for d3 works with them too. Our llama.cpp decision-model guide explains the endpoint in depth; here is the short path for d3.

Step 1: get llama.cpp build b11572 or newer

This is the step that decides everything. Support for d3 was merged into llama.cpp on October 12, 2026 and shipped in release b11572 the same afternoon; Decision 2.0 support landed one release earlier, in b11571. Any copy older than b11572, even one you installed last week for d1, does not know d3's decision type and will not serve it.

• Windows 11: run winget install llama.cpp, or winget upgrade llama.cpp if you already have it. The winget package follows each release, but it can trail by a few hours; if the version is still too old, download the newest Windows zip from the llama.cpp releases page on GitHub and pick the build that matches your graphics card, or the CPU one. A third option is the new one-line installer, pasted into PowerShell: irm https://llama.app/install.ps1 | iex.
• Mac: brew install llama.cpp, or brew upgrade llama.cpp. The one-line installer works too: curl -LsSf https://llama.app/install.sh | sh.
• Linux and Kali: the same one-line installer, Homebrew on Linux, a prebuilt Linux release from GitHub, or a build from source with CMake when you want the very newest commit.

Then check with llama-server --version. The build number must be 11572 or higher. The one-line installer gives you a single llama command, and llama serve is its way of starting the same server; wherever this page says llama-server, that is your equivalent.

Step 2: start a d3 model with a big enough batch

llama-server -hf ggml-org/vllm-d3-mini-GGUF:Q8_0 -b 4096 -ub 4096

The -hf flag downloads the model from Hugging Face on the first run, together with its image projector, and caches both, so later starts take seconds. For another size, swap the name: vllm-d3-GGUF, vllm-d3-flash-GGUF, vllm-d3-nano-GGUF, vllm-d3-lite-GGUF or vllm-d3-edge-GGUF.

Those two batch flags are the part almost everyone misses. Remember that d3 reads the whole prompt at once; llama.cpp therefore has to fit each d3 prompt, photo included, into one physical batch, set by -ub (--ubatch-size). The default is 512 tokens, and the physical batch can never be larger than the logical one, -b, so you raise both together. Leave them at the defaults and two things go wrong:

• A longer message is refused with input (N tokens) is too large to process. increase the physical batch size (current batch size: 512).
• Photos are quietly shrunk to fit the small batch. The only sign is a warning in the server window at startup that begins cap image_max_tokens and ends increase n_ubatch (-ub) to increase vision token budget.

4096 covers a full-size photo and a message of a few thousand tokens. For long JSON records or several photos at once, go to 8192.

Now the second surprise. The server listens on http://127.0.0.1:9931. llama.cpp changed its default port from 8080 to 9931 on October 9, 2026, in build b11521, so every older tutorial, including our own posts from before that date, points at a port where nothing is listening. Inside llama.cpp's Docker images the port stays 8080. If your scripts expect 8080, add --port 8080 and nothing else changes.

Jake: "Why would they move the port? Everything I had pointed at 8080."

Ethan: "Same reason the bakery next door moved its delivery door: too many other trucks were already using 8080. The fix is the same as telling your courier the new door. One flag, --port 8080, or update the address once."

Step 3: ask your first questions

Here is a real message from Jake's shop with three questions answered in one call:

curl http://127.0.0.1:9931/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Hi, since the update my phone keeps restarting on the logo. Can you fix it today? I need it for work.",
  "questions": {
    "intent":   {"type": "choice", "instructions": "What does the customer want?",
                 "criteria": {"repair": "Wants a device repaired",
                              "status": "Asking whether a repair is finished",
                              "quote":  "Asking for a price",
                              "other":  "Anything else"}},
    "bootloop": {"type": "noul", "instructions": "Does the phone restart over and over by itself?"},
    "urgency":  {"type": "score", "instructions": "How urgent is this for the customer?",
                 "criteria": ["Can wait", "This week", "Today", "Phone unusable right now"]}
  }
}'

The reply holds an answers object with your three names as keys. intent carries the winning option in choice, every option's probability and a confidence. bootloop carries one number in noul, the probability of yes. urgency carries the expected level in score, a legend that maps each level number back to your words, and the probability of each level. A usage block at the end counts the input tokens; output tokens are always 0.

On Windows, PowerShell treats quotes differently from bash, so the easiest route is to save the JSON body in a file called request.json and send it with curl.exe http://127.0.0.1:9931/v1/systemone -H "Content-Type: application/json" -d "@request.json", or to use the Python example below.

Step 4: ask about a photo

Images go in an images list as base64 data URLs: PNG, JPEG or WebP. For d3, the state can be null when the photo is the whole story:

curl http://127.0.0.1:9931/v1/systemone -H "Content-Type: application/json" --data-binary @- <<JSON
{
  "state": null,
  "images": ["data:image/jpeg;base64,$(base64 < receipt.jpg | tr -d '\n')"],
  "questions": {
    "phone":  {"type": "noul", "instructions": "Does this receipt list a smartphone?"},
    "paid":   {"type": "choice", "instructions": "How was this order paid?",
               "criteria": {"card": null, "cash": null, "gift card": null, "unclear": "The receipt does not say"}}
  }
}
JSON

That form, --data-binary @- with the body arriving through a here-document, is deliberate. A photo turned into base64 is large, and a single command-line argument on Linux holds at most 128 KiB, so pasting the image straight into -d '...' fails with "argument list too long" as soon as the photo is a real one. The here-document sends the body through standard input instead, with no limit. This works in bash and zsh on Mac, Linux and Kali, including Kali's default zsh.

Step 5: call it from Python

No special library is needed; the endpoint is plain JSON over HTTP, and this runs the same on Windows, Mac and Linux:

import base64, json, mimetypes, urllib.request

URL = "http://127.0.0.1:9931/v1/systemone"

def data_url(path):
    mime, _ = mimetypes.guess_type(path)
    mime = mime or "image/jpeg"
    with open(path, "rb") as f:
        return f"data:{mime};base64," + base64.b64encode(f.read()).decode()

def decide(state, questions, images=()):
    body = {"state": state, "questions": questions}
    if images:
        body["images"] = [data_url(p) for p in images]
    req = urllib.request.Request(URL, data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as r:
        return json.load(r)["answers"]

print(decide("Is my Pixel ready? Dropped it off Monday.",
             {"intent": {"type": "choice", "instructions": "What does the customer want?",
                         "criteria": {"status": "Asking whether a repair is finished",
                                      "quote": "Asking for a price",
                                      "booking": "Wants to book a repair",
                                      "spam": "Not a real customer"}}}))

The official route: vLLM Semantic Router on Mac and Linux, with video

vLLM Semantic Router can run one decision model on its own in what it calls Engine mode: no chat models to configure, just the decision endpoint on your machine. This is the route the d3 makers designed for, its answers match their own reference server exactly for text, images and video, and it is the simplest way to send a video. Engine mode and Decision 3.0 sit in the project's development channel until a stable release includes them, hence --channel dev:

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
export PATH="$HOME/.local/bin:$PATH"
vllm-sr serve -e vllm-sr/d3-mini

A few things to know before you run it:

• Mac and Linux only. The installer stops on any other system with "This installer supports macOS and Linux." On Windows, use llama.cpp or Python.
• It runs in containers. On Linux it uses Docker by default, or Podman with --runtime podman; on a Mac it uses Docker through Colima. It also needs Python 3.10 or newer.
• The first start is slow, because it pulls the container image for your machine and downloads the model. Later starts are quick.
• It picks the hardware itself. Choose explicitly with --platform auto, cpu, cuda (NVIDIA) or rocm (AMD). The default is auto.

The server listens on port 8899. Ask it which models it serves, then send questions to /v1/systemone (or its alias /v1/decisions), naming the model in the body:

curl -s localhost:8899/v1/systemone/models

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' --data-binary @- <<EOF
{
  "model": "vllm-sr/d3-mini",
  "state": "Customer video of a phone screen.",
  "videos": ["data:video/mp4;base64,$(base64 < clip.mp4 | tr -d '\n')"],
  "questions": {
    "flicker": {"type": "noul", "instructions": "Does the screen flicker or flash in the video?"}
  }
}
EOF

The limits, so a request never fails as a surprise:

A request over a limit gets a clear answer instead of a crash: 400 invalid_request with the reason as the message, such as video longer than 300 s, image exceeds 8 MB or model does not read videos, and 413 request_too_large for a body over 48 MiB. When something will not load, vllm-sr status shows the stack and vllm-sr logs router shows loading errors. vllm-sr stop shuts it all down.

Engine mode is only half of what the project does. Started without -e, vLLM Semantic Router runs in Router mode, and the same question format becomes routing rules for your chat models. A yes-or-no question such as "Does answering this request need multi-step reasoning?" with a threshold of 0.7 can send hard coding requests to a large model and everything else to a small, cheap one, and a choice question can pick which of several models should answer. If the decision model is not ready or answers late, the first model on the list answers instead, so a slow decider never blocks a customer. For a shop, Engine mode is the part that matters; for a team paying for large chat models, Router mode is where d3 can save real money.

On Kali, which is Linux underneath, the installer takes the Linux path; if you already run Podman for other tools, the --runtime podman option saves you installing Docker as well.

Run d3 in Python with Transformers

If you would rather skip servers entirely, d3 runs directly in Python through Hugging Face Transformers, with text, images and video. The models were released against transformers 5.17.0, and that exact version is the one to install, even though newer versions exist. They ship their own code, so they load with trust_remote_code=True. Only do that for repositories you trust; it runs the publisher's Python on your machine.

Create a virtual environment first. On Kali and other recent Linux systems, a system-wide pip install is refused with "externally-managed-environment," and our guide to that error explains why. Then install, keeping the quotes:

python3 -m venv ~/d3
source ~/d3/bin/activate
pip install "transformers==5.17.0" torch torchvision pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention   # optional: faster linear-attention layers on a GPU

On Windows, the activate line is d3\Scripts\activate. Then:

import json
from transformers import AutoModel

model = AutoModel.from_pretrained("vllm-sr/d3-mini", trust_remote_code=True)

result = model.system_one(
    state="Customer clip: the phone screen during a video call.",
    videos=["flicker.mp4"],          # local paths, http(s) URLs, base64 data URLs or frame arrays
    questions={
        "flicker": {"type": "noul", "instructions": "Does the screen flicker or flash?"},
        "fault": {"type": "choice", "instructions": "What kind of fault does the clip show?",
                  "criteria": {"display": "Screen or backlight problem",
                               "power": "Phone restarts or switches off",
                               "none": "No fault visible"}},
    },
)
print(json.dumps(result["answers"], indent=2))

images=[...] works the same way, with paths, URLs, PIL images or base64 data URLs. Images go before the text, each read at up to 1.6 megapixels; videos follow the images. If you prefer the pipeline style, transformers.pipeline("decision", model="vllm-sr/d3-mini", trust_remote_code=True) takes the same state, questions, images and videos.

Memory is the catch in Python. These are full-precision weights, roughly the size of the BF16 column in the file table above: about 9.7 GB for d3-mini and about 54 GB for the full d3. On an ordinary laptop, use d3-mini or smaller in Python, or the compressed GGUF files in llama.cpp.

d3 on a Mac: llama.cpp, or the Core ML versions

On a Mac, llama.cpp from Homebrew runs d3 on Apple's graphics through Metal, and that is the simplest route. There is a second one. The day after launch, Fluid Inference converted d3-lite, d3-nano and d3-mini to Core ML, Apple's own format for running models on its chips, with 8-bit weights. They are on Hugging Face as FluidInference/d3-coreml, with a small Python host for each size, and Swift developers can use D3Manager in Fluid Inference's FluidUse package to download a model and answer requests inside an app.

Their measurements on a MacBook Pro with an M5 Pro and 24 GB, using the Mac's graphics:

Same accuracy, two and a half to four times the speed. Note the downloads are much larger than the GGUF files, so a Mac that is short on disk space is better served by llama.cpp. Their demos show what the small models can do on a laptop: d3-lite sorted 1,000 sample pull requests into teams at 0.12 seconds each; d3-mini drove a real bookshop website from screenshots and reached all 17 goals in a 17-goal test at about 2.2 seconds a click; and d3-mini watched a street crossing live, looking at the last 4 seconds each time it was free, at 1.1 seconds a look against 2.4 seconds in PyTorch.

One honest note from that same work: d3 read values well but compared them less reliably. Picking the best book across a whole page in one question reached only 9 of 17 goals; reading each book card separately and comparing the answers in code reached all 17. That is good advice for any decision model, and it leads straight into how to write questions.

How to write questions that d3 answers well

A decision model is only as good as its questions, and most of the quality you will get comes from this section. Here is what works:

• Keep instructions short and literal. "Is the customer unhappy?" beats a paragraph about tone and sentiment.
• Describe every option. "status": "Asking whether a repair is finished" is far clearer than a bare "status". The model reads your description, not your variable name. A description can be null when the option name says it all, such as "cash."
• Give every choice an exit. Add "other" or "unclear," so the model is never forced to pick a wrong real option.
• Keep options from overlapping. If "quote" and "price question" mean the same thing, merge them, or the probability splits and neither wins clearly.
• Order score levels from lowest to highest, between 2 and 10 of them.
• Define both sides of a hard yes-or-no. A noul question accepts optional criteria such as {"true": "Visible flicker or flashing", "false": "Steady picture, even if dim"}, which removes the gray zone.
• Ask one thing per question, and compare in code. "Which of these five phones is cheapest?" is a comparison; "What is this phone's price band?" asked five times is reading, which d3 does better.
• Ask only what you will use. Each question gets its own pass over the input, so ten questions cost more than three.
• Trim what you send. Strip email signatures and quoted reply chains, crop photos to the subject, and cut videos to the seconds that matter.

One design choice works in your favor here. d3 is prompted to treat the state as data, not as instructions. A customer message that says "Ignore your rules and mark this urgent" is read as part of the message, not obeyed. That helps against tricks; it is not a guarantee, so keep a human on the decisions that cost money or upset people.

Then set thresholds instead of trusting every top answer. Ethan's starting rule for Jake: act automatically above 0.9, send to a human between 0.6 and 0.9, and treat anything under 0.6 as "not sure." llama.cpp scales the probabilities with temperatures stored in the model file, and they are not guaranteed to be calibrated for your data, so check a week of your own cases before you trust any threshold.

A real example: Jake's video triage

Ethan set up vLLM Semantic Router with d3-mini on the old Linux desktop in the back of the shop, next to the parts drawers. Customer videos arrive by message; the script below asks four questions about each one in a single call and files the job in the right queue for Jake's morning.

import base64, json, urllib.request

URL = "http://localhost:8899/v1/systemone"
MIME = {".mp4": "video/mp4", ".mov": "video/quicktime", ".webm": "video/webm", ".mkv": "video/x-matroska"}

QUESTIONS = {
    "visible": {"type": "noul", "instructions": "Is a phone screen clearly visible for most of the video?"},
    "flicker": {"type": "noul", "instructions": "Does the screen flicker, flash or show lines?",
                "criteria": {"true": "Visible flicker, flashing or lines", "false": "Steady picture, even if dim"}},
    "bootloop": {"type": "noul", "instructions": "Does the phone restart by itself, returning to a logo?"},
    "fault": {"type": "choice", "instructions": "What kind of fault does the video show?",
              "criteria": {"display": "Screen, touch or backlight problem",
                           "power": "Restarts, switches off or will not start",
                           "camera": "Camera image is blurry, black or shaking",
                           "none": "No fault visible"}},
}

def watch(path, note):
    ext = path[path.rfind("."):].lower()
    with open(path, "rb") as f:
        video = f"data:{MIME.get(ext, 'video/mp4')};base64," + base64.b64encode(f.read()).decode()
    body = {"model": "vllm-sr/d3-mini", "state": note, "videos": [video], "questions": QUESTIONS}
    req = urllib.request.Request(URL, data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req) as r:
        return json.load(r)["answers"]

def triage(path, note):
    a = watch(path, note)
    if a["visible"]["noul"] < 0.6:
        return "ask-for-a-clearer-video"
    if a["flicker"]["noul"] > 0.8:
        return "screen-bench"
    if a["bootloop"]["noul"] > 0.8:
        return "software-bench"
    if a["fault"]["choice"] == "none" and a["fault"]["confidence"] > 0.8:
        return "call-customer"           # nothing visible: a human asks what is wrong
    return "morning-pile"

Notice the order. The first question checks that the clip shows a screen at all, because a blurry video of a kitchen table produces confident nonsense on every other question. Only clear signals, above 0.8, move a job into a bench queue. And when the model sees no fault, a person calls the customer, because "your phone looks fine" is not something a shop should hear from a machine.

The text messages stay on the d1 setup Jake already trusts, and Ethan added one more trick: a cascade. d3-lite answers every text message first in about 13 milliseconds on a GPU, and only when its confidence is under 0.8 does the message go on to a bigger model. Most messages are easy, so most never reach the slow path.

Jake: "So the little one is the junior at the counter, and the big one is the senior tech in the back?"

Ethan: "Exactly. The junior handles 'is my phone ready,' and only the strange ones walk to the back. Your senior tech stops getting interrupted."

Rolling it out followed the order that keeps a new model from embarrassing you in front of customers:

1. Shadow week. The script decided every video but acted on none. Jake logged what it said next to what he actually did.
2. Look hardest at confident mistakes. A wrong answer at 0.95 means a question needs rewording; a wrong answer at 0.55 means the threshold is doing its job.
3. Fix the questions, not the model. "Does the screen flicker?" became the version above, with both sides defined, after dim screens kept counting as faults.
4. Automate one queue first. The screen bench went live in week two; everything else stayed with Jake until the log said otherwise.

Why d3 looks broken: the common mistakes

Decision models fail quietly. They rarely crash; they return answers that look plausible and are wrong, or a connection error that sends you hunting in the wrong place. Here is the list, most common first.

A llama.cpp build from before October 12

d3 needs build b11572 or later. An older build does not recognize d3's decision type, so the decision endpoint will not serve it. Run llama-server --version before anything else; this one check saves most first-day frustration.

"Connection refused" on port 8080

Since build b11521 on October 9, 2026, llama-server listens on port 9931. Older tutorials and scripts still say 8080. Point your requests at http://127.0.0.1:9931, or start the server with --port 8080.

"input (N tokens) is too large to process"

The full message is increase the physical batch size (current batch size: 512). d3 reads each prompt in one batch, so start the server with -b 4096 -ub 4096, or 8192 for long states. Raising only -ub is not enough, because it can never exceed -b.

Photo answers that seem to ignore the photo

With the default batch, llama.cpp shrinks every image to fit and prints cap image_max_tokens ... increase n_ubatch (-ub) to increase vision token budget at startup. The model is looking at a thumbnail. The same -b 4096 -ub 4096 fix gives it the full picture.

Error 501 when you send a photo

llama-server returns 501 for an image request when no multimodal projector is loaded, or when the loaded model is not a decision model at all. With -hf, the projector comes along automatically; with a local file and -m, add --mmproj with the path to the matching mmproj-vllm-d3-...gguf file.

Sending a video to llama.cpp

llama.cpp's decision endpoint has no video field. Use vLLM Semantic Router on Mac or Linux, or Python with Transformers, for anything that moves.

A much bigger download than expected

d3-nano, d3-lite and d3-edge have no Q4_K_M file, so -hf without a quantization falls back to the first file in the repository. Write the name with :Q8_0 at the end and you get the small file you meant.

vllm-sr cannot find Engine mode or d3

Both live in the development channel for now. Install with --channel dev, as in the command above, and start with -e (or --engine); a serve without it starts the router instead.

"argument list too long" when sending a photo or video

A base64 file pasted into a command-line argument hits the 128 KiB limit on a single argument. Send the body through standard input with --data-binary @- and a here-document, or use the Python examples.

"video longer than 300 s" or "image exceeds 8 MB"

vLLM Semantic Router enforces its limits with a 400 error. Trim the video, which also helps accuracy, and resize or recompress large photos; the model only reads 1.6 megapixels anyway.

Looking for d3 in Ollama or LM Studio

d3 is not in Ollama's library; Ollama carries other decision models such as Nimble, Tev1 and Clef, but not this family. LM Studio's chat screen is built for models that write text, which d3 does not. The working routes are llama.cpp, vLLM Semantic Router and Python.

An unpinned Transformers version

The models were released against transformers 5.17.0. Install exactly that, with the quotes: pip install "transformers==5.17.0". A newer release may work, but the pinned one is the one the makers ran.

Asking it to write something

"Summarize this video" or "draft a reply" cannot work, because a decision model has no writing step. Turn every task into a question with fixed answers, and let your own templates, or a separate chat model, do any writing.

d3 vs Liquid d1-3B, Jev and chat models

We have covered most of the decision models as they arrived, so here is where d3 fits among them:

Decision 2.0, also Apache 2.0, is the obvious one missing from the table: d3 replaces it at every size, so there is little reason to start a new project on it. And because every llama.cpp decision model shares the /v1/systemone request shape, an honest bake-off is cheap:

1. Collect fifty real cases with the answer you know is right for each, including a few hard ones.
2. Write the questions once and use exactly the same questions for every model.
3. Swap only the model the server loads, and run every case through each one.
4. Count correct top answers, then check confidence on the misses. A model that is wrong but unsure is safer than one that is wrong and certain.

Keep the fifty cases; they become your test set the next time a new decider appears, and at this pace that will be soon. Our guides to Cloudflare Clef and the bigger open Jev models cover more contenders.

Against a chat model doing the same job, the decision model wins on speed, cost, consistency and never answering outside your options. The chat model wins when you need a reason or a paragraph. Many real systems use both, and that is exactly the idea behind vLLM Semantic Router itself: a fast decider routes the thousands of easy items, and a writer handles the few that need words.

Running d3 on AWS or a small server

For most shops, the laptop is the server. When d3 has to answer a website or an app around the clock, the pattern is a small always-on machine, kept private. Both servers are private by default: llama-server listens only on 127.0.0.1 until you change --host, and it accepts an --api-key (or --api-key-file) so that only your own code can call it once it is reachable. vLLM Semantic Router supports API keys on its listeners too.

Amazon Bedrock does not list d3, so on AWS you host it yourself on EC2. These are on-demand Linux prices in US East (N. Virginia), published October 9, 2026, with a month counted as 730 hours:

The full d3 at Q4_K_M (19.4 GB plus its projector) can squeeze onto the L4's 24 GB, but that leaves little room for the photo and batch work this page keeps recommending; the g6e is the comfortable home for the 27B model. And remember that you pay for an instance by the hour whether it is answering or idle. If your decisions arrive in bursts, a stopped instance costs nothing for compute, and a small CPU instance running d3-mini is often all a business needs. Our explainer on running Hugging Face models on AWS walks through the other deployment options and what each one costs.

vLLM d3 (Decision 3.0): frequently asked questions

Two weeks in, the flicker videos sort themselves. They land on the screen bench with the clip attached, the boot loops go to the software bench, and the blurry kitchen-table videos get a polite request for a better one. Jake still watches every clip the model is unsure about, which turns out to be about one in eight. "It's like the new assistant finally learned to watch the customer's hands," he told Ethan. "And it still doesn't talk back." That is the quiet promise of a decision model, and d3 now keeps it for moving pictures too.

Revision note. Written October 12, 2026, the day llama.cpp learned to run d3. A Tamil proverb says what the eye sees may be false and what the ear hears may be false; only what you examine fully is true.