Ternary Bonsai 2 27B on Windows: Why LM Studio Won't Load It

Ternary Bonsai 2 27B is a 27-billion-parameter reasoning model from PrismML, squeezed into a 5.95 GB file, small enough for an 8 GB graphics card. It keeps 98.2% of its full-size parent's benchmark average. It runs on Windows 11 and Kali Linux today, but only through PrismML's own build of llama.cpp. And here is the part nobody puts in the headline: LM Studio and Ollama, the two tools most people search for alongside this model, cannot load it at all. Even the most-pulled Ollama mirror of the model says so in its own description. This guide gives you the route that actually works on Windows and Kali, which of the two files to download for your GPU, what the 98.2% quietly leaves out, and a fix for every error you are likely to hit.

Why a 6 GB file is worth your evening

Jake runs a phone-repair shop, and for a month he had been saving a plan in his notes app: an assistant that lives on the shop laptop, reads a photo of a cracked screen or a water-damaged board, and drafts the repair quote. No customer photos leave the building. No subscription. The laptop has an RTX 4060 with 8 GB of video memory, which, until September, meant choosing between a small model that could not reason and a big one that would not fit.

Then Bonsai 2 27B arrived on Hugging Face on September 17, 2026, promising a 27B model in about 6 GB. Jake did what almost everyone did. He opened LM Studio, searched for "ternary bonsai," downloaded 7 GB over the shop's capped connection, and got a load error. He tried Ollama next and downloaded it again. Same wall. That was his whole Saturday, and the second download pushed his monthly data bill over the cap.

"So is the model fake," he asked Ethan on Monday, "or am I just bad at this?"

"Neither," Ethan said. "You bought a Blu-ray and tried to play it in a DVD player. The disc is fine and so is the player. They just speak different formats, and nobody put a sticker on the box."

That sticker is what this guide is. If you lost an evening the same way, be kind to yourself. Right now, the two most obvious ways to run this model are the two that cannot.

What Ternary Bonsai 2 27B actually is

Start with the name, because every word in it matters.

Bonsai is PrismML's model family. The company also publishes it as "Prism ML." Its whole idea is the bonsai tree: a full tree, kept deliberately tiny. The family includes 1.7B, 4B, 8B and 27B sizes across two earlier generations, and now Bonsai 2 27B, the first model of the second generation.

Ternary means every weight in the model's layers is stored as one of just three values: minus one, zero, or plus one. A normal model stores each weight as a 16-bit number, with 65,536 possible values. A ternary model keeps three, plus a small shared scaling number for each group of 128 weights, which is what the "g128" in PrismML's documentation means. Three values need about 1.58 bits of information each, because 2 to the power of 1.58 is about 3. That is where the famous "1.58-bit LLM" phrase comes from.

2 is the generation. 27B is the size: 27.36 billion parameters in total, which PrismML breaks down as 24.35B in the language backbone, 2.54B in the embeddings and output head, and 0.46B in a vision tower.

Underneath, Bonsai 2 27B is built from Qwen3.8-27B, Alibaba's 27B model. It keeps that model's shape: a hybrid attention design (roughly 75% linear attention and 25% full attention), SwiGLU feed-forward layers, RoPE position encoding and RMSNorm. The practical results that matter to you:

• A 262K-token context window (262,144 tokens), inherited from the base model.
• Vision. It reads images through a separate projector file of about 0.63 GB.
• Tool calling, for agent-style use.
• Thinking mode on by default, with reasoning effort set to "xhigh" unless you lower it.
• Apache 2.0 license. You can use it commercially, including inside a business like Jake's.

"So it's Qwen on a diet," Jake said.

"It's Qwen after a very strict diet," Ethan said. "Most diets cost you some muscle. The interesting question is which muscles. We'll get to that, because the 98.2% number answers it less honestly than you'd think."

Why LM Studio and Ollama can't load it (yet)

This is the section most readers came for, so here is the plain version.

A GGUF file is a container. Inside it, every block of weights is labeled with a type number that tells the loading program how the numbers are packed: this block is Q4_K, that one is Q8_0, and so on. The loader keeps a list of every type it knows. If it meets a type number that is not on its list, it stops, because guessing would produce garbage.

Bonsai 2 27B uses two brand-new packings, PTQ1_0 and PQ2_0. Their type numbers are not on the list in mainline llama.cpp, and LM Studio and Ollama both run engines built from mainline llama.cpp. So the refusal happens at the very first step, before a single weight reaches your GPU. Depending on the build, stock llama.cpp stops with a message along the lines of tensor 'output.weight' has invalid ggml type 142 or failed to read tensor info. Types 142 and 143 are the two new packings.

There is also a deeper reason, and it explains why no simple "conversion" will ever be the fix. According to PrismML's whitepaper, Bonsai 2's weights are stored in a rotated basis: each weight matrix goes through a fixed, blockwise Hadamard rotation (in blocks of 1,024) before it is reduced to three values. The software running the model has to apply the matching rotation on every single step. A loader that doesn't know about that rotation would be doing the math in the wrong coordinate system, even if it could read the file. So Bonsai 2 needs new code, not just a new label in the file.

Why hasn't mainline llama.cpp just added them? New packings arrive with new compute kernels, the small GPU and CPU programs that unpack three-value weights and multiply them. PrismML maintains those in its fork, PrismML-Eng/llama.cpp, on a branch called prism, with ready-built releases for Windows, Linux and macOS. The pattern so far is encouraging. The first-generation Q1_0 type went from "fork only" to "merged into mainline" in April 2026, and PrismML's own documentation says ternary support for first-generation files has since landed in mainline for CPU, Metal, Vulkan and CUDA. It is reasonable to expect the new types to follow the same path. It is not reasonable to assume a date, so this guide will be updated when it happens.

The LM Studio workaround: a community plugin

If LM Studio is where you live, there is a middle path today. A small open-source project, jostoz/lmstudio-bonsai-2-27b on GitHub (MIT license), lets you chat with Bonsai 2 inside LM Studio's chat window while the model actually runs on PrismML's fork in the background. The plugin passes your conversation to a local PrismML llama-server and streams the answer, the reasoning and any tool calls back into LM Studio.

1. You need LM Studio 0.4.x with developer plugins allowed. The author reports it working on 0.4.24. You also need Node.js, because the plugin installs with npm.
2. Fetch the PrismML runtime with the project's script: .\fetch-runtime.ps1 on Windows. It picks the newest fork release that contains binaries.
3. Download the model: hf download prism-ml/Ternary-Bonsai-2-27B-gguf
4. Install the plugin: cd plugin, then npm install, then lms dev -i -y
5. Start the server with .\serve-bonsai.ps1, then pick the model joz/bonsai-2-27b in LM Studio's chat.

Or run first-generation Bonsai in LM Studio today

LM Studio's own model catalog lists prism-ml/bonsai-27b, the first-generation Bonsai 27B family built on Qwen3.6-27B, with a binary version of about 3.9 GB and a 262K context. Its older file types run on stock llama.cpp, so it loads the normal way. It scores lower than Bonsai 2 (80.98 against 84.78 on PrismML's 14-benchmark average), but if "works in LM Studio tonight with zero tinkering" matters more to you than the last few points, it is a perfectly sensible choice.

"So LM Studio isn't broken," Jake said.

"LM Studio is behaving perfectly," Ethan said. "It's refusing to guess. That's what you want from software that's about to put six billion numbers into your GPU."

PTQ1_0 or PQ2_0: which Bonsai 2 27B GGUF to download

The Bonsai 2 27B GGUF repository on Hugging Face (prism-ml/Ternary-Bonsai-2-27B-gguf) gives you two versions of the same model. They are identical in quality and differ only in how the three-value weights are packed.

• PTQ1_0 packs the three values densely: about 1.75 bits per weight, 5.95 GB on disk.
• PQ2_0 gives every weight a full 2-bit slot, wasting a little space so the GPU can unpack it with less arithmetic: about 2.13 bits per weight, 7.21 GB on disk. This is what searches for "ternary bonsai 27b 2 bit" are looking for.

You would expect the smaller file to be slower, since it needs more unpacking work. On some cards it is the opposite, and this is the second surprise in this model: on an RTX 4090 the smaller PTQ1_0 file generates faster than PQ2_0, 91.1 against 81.2 tokens per second, according to PrismML's published measurements. The 4090 is limited by memory bandwidth, so moving fewer bytes wins. On compute-heavy cards like the RTX 5090 and H100, the roomier PQ2_0 packing wins instead.

The rule this table suggests: Ada-generation cards (the RTX 40 series, RTX 6000 Ada, L4, L40S) prefer PTQ1_0. Ampere, Hopper and Blackwell cards prefer PQ2_0. PrismML's model card says as much: unpacking trits costs arithmetic, and that cost lands differently on different GPU generations.

Both files also need the vision projector if you want image input: the Q8_0 mmproj file (0.63 GB). That explains a small mystery on the Ollama mirror page, where the tags read 6.58 GB and 7.84 GB. Those sizes are the model file plus the projector: 5.95 + 0.63 and 7.21 + 0.63. A full 16-bit reference copy also exists at 53.8 GB, and you do not want it unless you are doing research.

Bonsai 2 27B hardware requirements: VRAM, context and the KV cache

The file size is only half of the memory story. The other half is the KV cache, the working memory the model uses to remember your conversation. It grows with every token of context you allow, and on a 27B model it grows fast.

PrismML's demo documentation gives the figure: about 64 KiB per token with the default 16-bit cache, or about 18 KiB per token with a 4-bit cache. Converted to the context sizes people actually pick:

Put the two halves together and you get a realistic picture of what each machine can do. The GPU tiers below match the context estimates in community install guides for this model.

Two practical notes. First, the Bonsai-demo scripts choose a context size automatically based on your free memory, somewhere between 8K and 131K. You only have to think about this table when you run llama.cpp by hand. Second, the vision projector (0.63 GB) needs memory too. On an 8 GB card, the demo lets you keep it in system RAM with BONSAI_MMPROJ_CPU=1, so the GPU memory goes to the model.

"So my 8 GB card is enough," Jake said.

"Enough to run a 27B model, which a year ago was a sentence nobody said," Ethan said. "Just don't paste a whole service manual into it and expect it to remember page one."

Bonsai 2 27B benchmarks: what the 98.2% hides

PrismML's headline claim is that Bonsai 2 27B keeps 98.2% of the full-precision model's intelligence. That is an average across 14 benchmarks in thinking mode: 84.78 for Bonsai against 86.32 for the original 16-bit Qwen3.8-27B. As a headline it is accurate. As a guide to your use case it is incomplete, because an average can hide a weak spot behind a strong one.

Here is the category view from the model card.

Read that table the way you would read a used-car report. Math and short coding tests came through the compression essentially untouched, and on some coding and instruction tests Bonsai even edges past the original. Small gains like that are normal noise in benchmark runs, not proof that compression makes models smarter. The real losses sit in two places: multi-step reasoning over long text (MuSR fell from 79.63 to 70.63) and vision, especially reading text in images (OCR Bench v2 fell from 60.99 to 56.88, and MMMU-Pro from 81.73 to 75.49).

The loss the headline table doesn't show: long agentic coding. PrismML's whitepaper reports two harder, multi-step tests that aren't in the 14-benchmark table. On SWE-bench Verified, where a model has to fix real bugs in real code repositories, Bonsai 2 scores 60.8 against 80.6 for the full model. On Terminal-Bench 2.1, long tasks in a command line, it scores 52.8 against 69.7. That is roughly three-quarters of the full model's ability, as PrismML itself says, and still remarkable for a 6 GB file. But if you want an unattended coding agent working through a big repository, this is the number to plan around, not 98.2%.

PrismML is candid about why this matters. Its whitepaper notes that when low-bit models fail, long-context reasoning and coding deteriorate first, while fluent writing and short tool calls hold up. The failure "is easy to miss in casual testing and expensive to discover in production." So test it on your real, long tasks before you trust it with them.

One more detail for careful readers. The model card averages 14 benchmarks (84.78 against 86.32), and the whitepaper averages a wider 20-benchmark suite (83.9 against 85.4). Both come out at 98.2%, so the headline holds either way.

For Jake that is the most useful line in this whole guide. His plan leaned on reading photos of repair tags and serial stickers, and that is exactly the category that lost the most. "So it'll write a perfect quote," he said, "but it might misread the model number on the sticker."

"Which is why a human glances at the sticker," Ethan said. "Use it for the reasoning and the writing, and double-check anything it reads off a photo. That's good practice with every vision model, compressed or not."

The fairer comparison is not against the full 54 GB model, which you could never run on this laptop anyway. It is against the other ways of making Qwen3.8-27B small:

This table is the real reason the model matters. At almost exactly the same size as an ordinary 2-bit squeeze of the same model (7.27 GB), Bonsai scores 84.78 where the 2-bit version scores 72.59. On the hardest math test in the set, AIME26, the ordinary 2-bit version drops to 57.50 while Bonsai holds at 95.83. Ordinary 2-bit quantization damages a model. Bonsai's approach mostly does not.

"Bonsai 27B vs Qwen 3.6 27B" and "Bonsai 27B vs Qwen 3.5 9B" are popular searches too. The first compares the first-generation Bonsai 27B with the model it was built from. For the current model, the right comparison is the Qwen3.8-27B table above. For Qwen3.5-9B and Gemma 4 there is no published head-to-head on the same benchmark suite, and we would rather tell you that than guess. What the size math does say: at around 6 GB, Bonsai 2 27B costs about the same memory as a 9B model at 4-bit, while carrying a 27B model's knowledge.

Ternary LLMs, 1.58-bit, BitNet: what the words mean

If you have been reading around this model you have met a cloud of terms: ternary LLM, 1-bit LLM, 1.58-bit, BitNet, bitnet.cpp. Here is the short map.

The research origin is Microsoft's BitNet. In February 2024, Microsoft Research and the University of Chinese Academy of Sciences published "The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits" (arXiv 2402.17764). It described BitNet b1.58, a model whose weights are all minus one, zero or plus one. People call these "1-bit LLMs" loosely. Strictly, three values need about 1.58 bits, and that is the "b1.58" in the name. The paper reported that at 3B parameters, BitNet b1.58 matched a 16-bit LLaMA model on perplexity and zero-shot accuracy, while using 3.55 times less GPU memory and running 2.71 times faster.

bitnet.cpp is Microsoft's inference framework for those models (the microsoft/BitNet repository on GitHub). BitNet b1.58 2B4T is its first official model, with 2.4B parameters trained on 4 trillion tokens. If you searched "what does BitNet stand for": it is not an acronym. It reads as "bit network," and its core piece is a layer Microsoft calls BitLinear, which stands in for a normal layer that uses 16-bit weights.

Here is how Bonsai differs, and it matters:

"Ternary vs binary" is the last piece. Binary (true 1-bit) weights are only minus one or plus one. Ternary adds zero, which lets the model switch a connection off entirely. That third value is a large part of why ternary models hold their quality better than 1-bit ones, and it is why PrismML now offers both a 1-bit and a ternary line. The earlier 1-bit Bonsai 27B was a 3.80 GB file. The ternary Bonsai 2 27B is 5.95 GB and scores 84.78 against the older model's 80.98.

"So the zero is the clever bit," Jake said.

"The zero is the whole trick," Ethan said. "It's the difference between a light switch and a dimmer that also has an off position. You'd be amazed how much of a model's job is knowing what to ignore."

How to run Ternary Bonsai 2 27B on Windows 11 (the easy way)

For most Windows users the easiest route is PrismML's own Bonsai-demo repository. Its setup script downloads the right prebuilt llama.cpp binaries for your hardware, fetches the model from Hugging Face, and gives you a chat server with a browser interface. PrismML lists Windows x64 and Windows on Arm, with CPU, CUDA 12.4, Vulkan and HIP/ROCm builds.

Before you start you need Git for Windows (from git-scm.com) and a current graphics driver. NVIDIA users need a driver recent enough for CUDA 12.4. AMD and Intel GPU users get the Vulkan build.

1. Open PowerShell (not Command Prompt). Press Win + X and choose Terminal.
2. Move to a drive with at least 15 GB free. The model, the binaries and a Python environment add up. For example: cd D:\
3. Clone the demo: git clone https://github.com/PrismML-Eng/Bonsai-demo.git
4. Enter the folder: cd Bonsai-demo
5. Allow scripts for this window only: Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass. This changes nothing permanently. Close the window and the setting is gone.
6. Run setup: .\setup.ps1. It installs the uv Python manager, creates a virtual environment, downloads the model and binaries, and optionally installs Open WebUI. It takes a while, mostly spent on the download.
7. Start the chat server: .\scripts\start_llama_server.ps1
8. Open http://localhost:8080 in your browser and say hello.

git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\setup.ps1
.\scripts\start_llama_server.ps1

If you skip step 5, Windows stops you with "running scripts is disabled on this system." That is not an error in the demo. It is Windows' default script policy doing its job, and the one-line bypass above is exactly what PrismML's troubleshooting page recommends.

Ternary Bonsai 2 27B on Windows without the demo: llama.cpp by hand

If you already have a local-AI setup, you may prefer to skip the demo and use PrismML's prebuilt llama.cpp directly. This is also the route to take when you want to plug Bonsai into another program that talks to an OpenAI-style API.

1. Go to github.com/PrismML-Eng/llama.cpp/releases. At the time of writing, the newest release is prism-b10743-adfffbe, published September 25, 2026. Releases are built automatically and appear every few days, so take the newest.
2. Download the Windows asset for your hardware: the CUDA 12.4 x64 build for NVIDIA cards (it includes the CUDA runtime DLLs), the Vulkan x64 build for AMD and Intel GPUs, or the CPU x64 build for no GPU.
3. Unzip it to a short path such as D:\bonsai\bin.
4. Install the Hugging Face download tool: pip install -U huggingface_hub
5. Download one model file. Pick PTQ1_0 or PQ2_0 using the table above: hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir D:\bonsai
6. For image input, download the Q8_0 mmproj file from the same repository too.
7. Start the server from the bin folder with all layers on the GPU and flash attention on:

.\llama-server.exe -m D:\bonsai\Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 99 -fa on -c 16384 --port 8080

Open http://localhost:8080 and you get llama.cpp's built-in chat page. Any app that can talk to an OpenAI-compatible endpoint can use http://localhost:8080/v1 as its server address. That includes Open WebUI and many coding tools, and it is the honest workaround for "I want Bonsai inside my usual app."

For a single prompt instead of a server, PrismML's model card uses llama-cli:

.\llama-cli.exe -m D:\bonsai\Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 99 -fa on -c 32768 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.05 -p "Explain quantum computing in simple terms." -n 16384

"Why -ngl 99?" Jake asked.

"It means 'put up to 99 layers on the GPU,' which in practice means all of them," Ethan said. "It's the most common thing people forget. Leave it out and the model runs on your CPU and you decide ternary models are slow. They aren't. You just parked the car in the garage and walked."

How to install Ternary Bonsai 2 27B on Kali Linux

Kali is Debian underneath, so the Linux path applies unchanged. PrismML publishes Linux builds for x64 and arm64: CPU, CUDA 12.4 and 12.8, Vulkan, and ROCm 7.2.

Step zero, if you use an NVIDIA card: Kali's own documentation installs the proprietary driver and CUDA toolkit with one command. Reboot afterward.

sudo apt update
sudo apt install -y nvidia-driver nvidia-cuda-toolkit

Then the demo route mirrors Windows:

1. Install the basics: sudo apt install -y git build-essential cmake
2. Clone the demo: git clone https://github.com/PrismML-Eng/Bonsai-demo.git && cd Bonsai-demo
3. Run setup: ./setup.sh
4. Start the server: ./scripts/start_llama_server.sh
5. Open http://localhost:8080 in Firefox.

If you would rather compile the fork yourself, which is common on Kali where people like to see exactly what they run, the model card gives the CUDA build:

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

Kali users often run inside VirtualBox or VMware. Be realistic there: a virtual machine usually cannot reach your GPU, so Bonsai will run on the CPU build. It works, slowly. For real use, run it on the host system and connect to it from the Kali VM over http://host-ip:8080.

Running it well: sampling settings, thinking mode and vision

Bonsai 2 27B has two personalities, and PrismML publishes separate sampling settings for each.

Thinking mode is on by default with reasoning effort at "xhigh." The model writes out its reasoning inside think tags before it answers. That is why it scores so well on math, and also why a simple question can take a while. For everyday questions you can lower the effort to "medium," or turn reasoning off entirely: BONSAI_THINKING=0 in the demo. Know the trade before you do. On PrismML's 20-benchmark suite at medium effort, Bonsai 2 averages 79.3 against 82.6 for the full model, about 96% retained instead of 98.2%. The hardest math feels it most (AIME25 drops to 74.58). PrismML also notes that a "low" setting does not actually reduce thinking, so xhigh and medium are the two real choices. If you use the demo's browser interface, check its Reasoning effort picker. It is saved in your browser and overrides the server setting, which is the usual reason people say "I turned thinking off and it is still thinking."

The recommended system prompt is refreshingly short: "You are a helpful assistant."

Vision needs the mmproj file loaded alongside the model. In the demo it is handled for you. By hand, add --mmproj with the path to the projector file on the llama-server line. Remember the benchmark lesson: it reasons well about images but reads small text in photos less reliably than the full-size model.

Bonsai 2 27B errors and how to fix them

Every error below is one real users hit. They come from PrismML's troubleshooting documentation and the model's community install reports. They are ordered by how often they show up in searches.

How to tell "slow" from "broken" in ten seconds. Open Task Manager, go to Performance, then GPU, while the model answers. If Dedicated GPU memory holds roughly the model size plus your context, and the GPU graph is busy, everything is working. If dedicated memory barely moved and your CPU is pinned, the layers never reached the GPU. On Kali, nvidia-smi tells you the same thing.

"Most of this table is one mistake," Jake said, scanning it. "Using the wrong llama.cpp."

"Most of any troubleshooting table is one mistake wearing different hats," Ethan said. "The skill is recognizing the hat."

For IT admins: rolling Bonsai 2 27B out to a team

A 27B model that fits on an ordinary 8 GB workstation GPU changes the budget conversation for offline AI. If you look after a fleet, here is the checklist.

• License: Apache 2.0 covers commercial and internal use. Keep the license and notice files with any internal package you build.
• Supply chain: the fork is maintained by PrismML, and its releases are produced automatically by GitHub Actions with signed commits. Treat it as you would any third-party binary. Pin a specific release tag, such as prism-b10743-adfffbe, in your deployment scripts rather than "latest," and mirror the zip internally so every machine runs the same build.
• Execution policy: the demo's Set-ExecutionPolicy -Scope Process bypass applies to one window only and leaves Group Policy alone. On managed machines, deploy the prebuilt binaries through your normal software-distribution tool (Intune, Configuration Manager or your RMM) instead of letting users run setup scripts.
• Network exposure: llama-server listens on port 8080. Keep it on localhost for single users. For a shared team server, put it behind your reverse proxy with authentication. The built-in server is not meant to be exposed to the open internet.
• Data handling: inference is fully local. No prompts or images leave the machine unless you connect it to something that sends them. That is often the whole business case, and worth writing into your data-protection record.
• Hardware planning: 8 GB cards work for single-user chat at modest context. For shared use or long documents, plan on 16-24 GB per server, using the KV cache table above.
• Change watch: when mainline llama.cpp, and after it LM Studio and Ollama, add the new types, you can move users onto a tool they already know. Revisit your deployment when that happens.

Bonsai 2 27B review: is it worth installing?

Here is the honest verdict, by reader type.

Worth it if you have an 8-12 GB GPU and want the best reasoning you can fit. At about 6 GB, nothing else we know of offers a 27B model's math and everyday coding ability at this size. For long, unattended agentic coding, plan on roughly three-quarters of the full model (SWE-bench Verified 60.8 against 80.6). The comparison with an ordinary 2-bit version of the same base model (84.78 against 72.59) is the entire argument, and it is a strong one.

Worth it if you want offline, private, commercial-friendly AI. Apache 2.0, local inference and no account needed. For a shop like Jake's, that is the whole point.

Wait, or use a stopgap, if LM Studio or Ollama is your comfort zone. Nothing is wrong with waiting for mainline support. Meanwhile, the community LM Studio plugin works for chat, and first-generation Bonsai 27B runs in LM Studio today.

Think twice if your main job is reading text from images. Vision and OCR are exactly where Bonsai gave up the most ground. A larger vision model, or the full Qwen3.8-27B on bigger hardware, will serve you better.

Skip it for now if you only have integrated graphics and 16 GB of RAM. The CPU build runs, but a smaller model at 4-bit will feel far more responsive for everyday chat.

Jake ended up in the first group. He installed the demo on the shop laptop that evening, pinned BONSAI_CTX=8192 so his 8 GB card stayed comfortable, and turned thinking off for simple quote drafts. His one rule, pinned above the counter: the assistant drafts, a human checks the model number on the sticker.

Ternary Bonsai 2 27B FAQ

What is Ternary Bonsai 2 27B?

Ternary Bonsai 2 27B is PrismML's 27.36-billion-parameter reasoning model, released September 17, 2026 under Apache 2.0. It is built from Qwen3.8-27B, with weights stored as minus one, zero or plus one, which shrinks it to a 5.95 GB file while keeping 98.2% of the original's 14-benchmark average.

What is Bonsai 27B's model name on Hugging Face?

The GGUF files are in the repository prism-ml/Ternary-Bonsai-2-27B-gguf, as Ternary-Bonsai-2-27B-PTQ1_0.gguf (5.95 GB) and Ternary-Bonsai-2-27B-PQ2_0.gguf (7.21 GB). There is also an MLX version for Apple Silicon, Ternary-Bonsai-2-27B-mlx-2bit.

Can I run Ternary Bonsai 2 27B in LM Studio?

Not natively yet. LM Studio's built-in engine does not recognize the new PTQ1_0 and PQ2_0 types, and the weights also need PrismML's Hadamard rotation at run time. A community plugin (jostoz/lmstudio-bonsai-2-27b) lets you chat with it inside LM Studio 0.4.x while PrismML's llama.cpp fork does the work. The first-generation Bonsai 27B loads in LM Studio normally.

Does Ternary Bonsai 2 27B work with Ollama?

Not with stock Ollama. Its GGUF parser rejects the new types, and even the popular Ollama mirror of the model says it requires PrismML's llama.cpp fork. The workaround is to run the fork's llama-server and point your apps at its OpenAI-compatible address, http://localhost:8080/v1.

How do I run Ternary Bonsai on Windows?

The easiest way is PrismML's Bonsai-demo: clone the repository, run Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass, then .\setup.ps1 and .\scripts\start_llama_server.ps1, and open http://localhost:8080. The manual alternative is the fork's prebuilt Windows release plus llama-server.

How much VRAM does Bonsai 2 27B need?

The PTQ1_0 file is 5.95 GB, so an 8 GB GPU runs it with about 16K tokens of context. 12-16 GB gives you 56K-118K tokens, 24 GB about 240K, and a 32 GB RTX 5090 the full 262K. The KV cache costs about 64 KiB per token, or 18 KiB with a 4-bit cache.

Should I download PTQ1_0 or PQ2_0?

Choose PTQ1_0 for 8 GB cards and for RTX 40-series and other Ada-generation GPUs, where it is both smaller and faster. Choose PQ2_0 for Ampere, Hopper and Blackwell cards with enough memory, such as the RTX 3090, A100, H100 and RTX 5090. The quality is the same; only the packing differs.

Is Ternary Bonsai 27B really 2-bit?

The PQ2_0 file stores each weight in a 2-bit slot, about 2.13 bits per weight. The values themselves are ternary, so they need only about 1.58 bits, and the denser PTQ1_0 file gets closer to that at 1.75 bits. Including scales, PrismML puts the true figure at 1.72 bits per weight.

What is a ternary LLM?

A ternary LLM stores each weight as one of three values: minus one, zero or plus one. Three values need about 1.58 bits each, which is why these are also called 1.58-bit LLMs. The zero lets the model switch connections off, which helps ternary models keep more quality than true 1-bit models.

What are 1-bit LLMs?

1-bit LLMs store weights in one or very few bits instead of 16. Strictly, a 1-bit model uses only minus one and plus one, while 1.58-bit (ternary) models add zero. The term became popular through Microsoft's 2024 paper "The Era of 1-bit LLMs," which introduced BitNet b1.58.

What is BitNet, and how is Bonsai different?

BitNet b1.58 is Microsoft Research's ternary model design, trained in low precision from scratch and run with the bitnet.cpp framework. Its largest official model has 2.4B parameters. Bonsai 2 27B is built from an existing 27B model, Qwen3.8-27B, and runs through PrismML's llama.cpp fork.

Is ternary Bonsai better than the 1-bit Bonsai?

For quality, yes. The first-generation 1-bit Bonsai 27B was a smaller 3.80 GB file with a 14-benchmark average of 80.98, while the ternary Bonsai 2 27B scores 84.78 at 5.95 GB. The 1-bit files do have one advantage: they run on stock llama.cpp today.

Can I run Ternary Bonsai 8B in LM Studio?

The first-generation Ternary Bonsai 8B is a separate, earlier model. PrismML says its Q2_0_g64 files run on stock llama.cpp builds, and LM Studio's catalog already lists the first-generation Bonsai family (prism-ml/bonsai-27b). Choose the Q2_0_g64 file, not the legacy Q2_0 file, which loads silently but produces gibberish.

How does Bonsai 2 27B compare with Gemma 4 or Qwen 3.5 9B?

No head-to-head results on the same benchmark suite have been published, so any ranking would be a guess. On size alone, Bonsai 2 27B uses about the memory of a 9B model at 4-bit while carrying a 27B model's knowledge. On your own tasks, it is worth trying both.

Can Ternary Bonsai 2 27B run on a CPU without a GPU?

Yes. PrismML publishes CPU builds of its fork for Windows and Linux, and the demo runs CPU-only with BONSAI_NGL=0. Expect it to be much slower than on a GPU. For CPU-only everyday chat, a smaller model often feels better.

Does Bonsai 2 27B support images?

Yes. It has a 0.46B-parameter vision tower, loaded through a separate 0.63 GB mmproj file. Its vision scores dropped more than any other category in compression (MMMU-Pro 75.49 against 81.73), so double-check anything it reads from small text in photos.

Does lowering reasoning effort make Bonsai 2 27B worse?

A little. At the default xhigh effort it keeps 98.2% of the full model's 20-benchmark average; at medium effort it keeps about 96% (79.3 against 82.6), with the hardest math dropping most. PrismML says a low setting does not reduce thinking, so xhigh and medium are the real options.

Can I use Bonsai 2 27B commercially?

Yes. Bonsai 2 27B is released under the Apache 2.0 license, which permits commercial use, modification and redistribution, provided you keep the license and notice files.

Why does stock llama.cpp say invalid ggml type 142?

Types 142 and 143 are Bonsai 2's new PTQ1_0 and PQ2_0 packings. Mainline llama.cpp does not know them yet, so it refuses to load the file. Download a prebuilt release from github.com/PrismML-Eng/llama.cpp instead.

If you lost a Saturday and a data allowance to a model that would not load, you did nothing wrong. You followed the path every tutorial points to, and the path simply was not built yet. Take the fork route and you will be chatting with it tonight. And if something in this guide does not match what you see on your screen, please tell me through the contact page. Posts like this stay accurate because readers write in, and this one will be updated the day LM Studio and Ollama catch up.

Revision note. Written September 26, 2026, nine days after Bonsai 2 27B's release. It covers PrismML's llama.cpp fork release prism-b10743-adfffbe (September 25, 2026) and the Bonsai-demo scripts as of that date. Benchmark and speed figures are PrismML's published numbers. The next update comes when mainline llama.cpp, LM Studio or Ollama add PTQ1_0 and PQ2_0 support, because that changes the easiest install path for everyone. Good luck with the install. It is a genuinely remarkable little model once it is running.