Run Qwen3.8 Locally on Windows and Kali (and Why GLM Can't)

Two open models are sitting at the top of Hugging Face’s trending list this week: Alibaba’s Qwen3.8-27B and Z.ai’s GLM-5.3-Flash, both with same-week GGUF files from Unsloth, both licensed for anything you like (Apache 2.0 and MIT), both claiming to trade blows with the paid frontier chatbots on coding. Only one of them can run on a computer you own. Qwen3.8-27B is a dense 27-billion-parameter model that fits, in compressed form, in about 16.5 GB: a 24 GB graphics card runs it fast, a 32 GB PC runs it at coffee speed, and a 16 GB laptop can run a smaller 11 GB cut of it, slowly. GLM-5.3-Flash is a 320-billion-parameter mixture-of-experts model whose smallest download, the 1-bit one, is 93 GB, and the honest 4-bit version is 200 GB. That is not a laptop. It is not a desktop either. Ollama, the tool most people use to run these, lists GLM-5.3-Flash with exactly one tag, :cloud, which means "we will run it for you on our servers," and that is the tell. This page installs Qwen3.8-27B on Windows 11 and Kali Linux step by step, tells you which cut of it your machine can honestly run, shows you the thinking switch and the image input, and explains what to do about GLM when the thing you wanted was the private, on-your-own-disk model and not a rented one.

Jake read about GLM-5.3-Flash on Wednesday, in a post that said it "approaches the best paid model on coding," and sent the link with one line: "Free and better than the paid one? Put it on the shop PC." Last week we put IBM’s Granite 4.2 on that PC, a five-year-old desktop with 16 GB of memory and no graphics card, and it has been quietly summarizing warranty PDFs ever since, so the request was reasonable. The answer was not what he wanted. The file he was asking for is bigger than the drive in that machine. Not bigger than the free space; bigger than the drive. Then he asked the better question, which is the one this page is really about: "So which of these trending ones can I actually have?"

Ethan: "There are two kinds of free. A free cookbook you can take home and cook from is the first kind. A free meal at a restaurant is the second: it costs you nothing, it is often excellent, and you still had to walk into their building and eat at their table. Qwen3.8-27B is a cookbook, and it is a heavy one, so you need a decent kitchen. GLM-5.3-Flash is the restaurant. It says free on the sign, the license really is MIT, and the kitchen is the size of a warehouse, which is why nobody is taking it home. Neither is a trick. You just need to know which one you are being offered before you clear space on your counter."

What actually trended this week, and what each one is

Hugging Face is the site where open models are published, and its trending page is a fair proxy for what the people who run these things are excited about. At the time of writing, the top of that list is one company’s wave with one interloper:

Two terms from that table matter for the rest of the page. Dense means every parameter participates in every word the model produces, so the whole file must sit in memory and the whole file is worked through for each token; that is Qwen3.8-27B, and it is why 27B is a heavy lift for a home machine even though the number sounds modest next to 320B. Mixture of experts means the model is made of many specialist blocks and only a few fire per token, so GLM-5.3-Flash does the work of an 18B model while needing the memory of a 320B one. On a server with the memory to hold it, that makes it cheap to run and fast, which is exactly why Z.ai can sell it at a tenth of its predecessor’s price. On your PC, the memory requirement is the wall, and no amount of "only 18B active" gets you over it.

A word on the benchmark claims, because they are what made both models trend. Alibaba says Qwen3.8-27B beats its own Qwen3.6-27B and Qwen3.7-Plus on coding and scores 61.7 on SWE-bench Pro against 53.4 for Anthropic’s top model, and 84.3 on a computer-use test. Z.ai says GLM-5.3-Flash approaches Anthropic’s flagship model on coding and agent benchmarks and posts 84.3 on Terminal Bench 2.1. These are the makers’ own numbers on the makers’ chosen tests. They are not lies, and they are not the experience you will have on a 32 GB desktop running a 4-bit cut with the context trimmed. A model that matches a frontier chatbot on a leaderboard, at full precision, on a rack of accelerators, becomes a very good, somewhat slower assistant on your machine. That is still a remarkable thing to own for free, and it is a different thing from the headline.

Which cut of Qwen3.8-27B your machine can honestly run

The only question that matters before you download anything is memory: graphics-card memory (VRAM) if you have a real card, otherwise system RAM. The model file has to be loaded in full, plus room for the context you feed it, so the working rule is about 1.2 to 1.4 times the file size in free memory. Because the same 27B model is published in many compressed ("quantized") cuts, from a 6 GB 1-bit file to a 31 GB near-original one, "can I run Qwen3.8" is really "which cut," and the answer runs like this. The file names are Unsloth’s, which are the ones Ollama and llama.cpp pull; the "UD" prefix is their dynamic quantization, which keeps the sensitive layers at higher precision and is the reason the small cuts are usable at all.

Speed, so nobody is surprised: on a 24 GB card the Q4 cut writes faster than you can read, dozens of tokens per second. On a CPU alone, expect roughly one to four tokens per second on the Q4 and a little better on the IQ3, depending on how new the processor is and how fast the RAM is. A token is about three-quarters of a word. With thinking mode on, which it is by default, the model writes several hundred tokens of reasoning before it writes the first word of the answer, so on a CPU a hard question can mean two or three minutes of watching a cursor. That is why the thinking switch section exists.

And the line Jake needed: if your machine is a 16 GB laptop with no graphics card, the honest comparison is not "Qwen3.8-27B or nothing," it is "the IQ3 cut of Qwen at two tokens a second, or Granite 4.2’s 8B at eight." For PDFs and drafts, the 8B at full precision is the better daily driver on that hardware; for code and hard reasoning, the bigger model in its squeezed form still wins, if you are willing to wait for it. Try both. They are free, and Ollama makes switching a one-line affair.

Install on Windows 11 (and 10): the ten-minute version

Ollama is the tool. It downloads, stores and runs models, uses your graphics card if it finds one and your CPU if it does not, and gives you both a chat window in the terminal and a local server that other apps can talk to. It is free and open source, and it runs on Windows, Linux and Mac with the same commands, which is why one page can cover both operating systems.

1. Install Ollama. Open Terminal (right-click Start → Terminal) and run winget install Ollama.Ollama. Or download the installer from ollama.com and run it. Either way, a small llama icon appears in your system tray when it is running.
2. Check it is alive. In a new Terminal window: ollama --version. If it prints a version, continue. If it says not recognized, close and reopen Terminal so the new PATH is picked up.
3. Pull and run Qwen3.8. If you have 24 GB of VRAM or 32 GB of RAM: ollama run qwen3.8:27b. It downloads about 18 GB, once, then drops you into a chat prompt. The first load after each reboot takes a while; after that it is quick.
4. Sixteen-gigabyte machine? Pull a smaller Unsloth cut directly from Hugging Face instead: ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-IQ3_XXS (10.9 GB). Close the browser first; on a 16 GB laptop, Chrome is the difference between running and thrashing.
5. Ask it something. Type a question, press Enter. You will see it think first (a block of reasoning), then answer. To leave: type /bye.
6. Where the files went. Models live under C:\Users\<you>\.ollama\models. If your C: drive is small, set the OLLAMA_MODELS environment variable to a folder on a bigger drive before you pull anything (Settings → System → About → Advanced system settings → Environment Variables), then restart Ollama from the tray.

Windows 10 works identically; Ollama does not care which one you are on. The one honest line for Windows 10 users: Microsoft ended support on October 14, 2025, and a PC that is not enrolled in Extended Security Updates has been unpatched since. Running a local model does not change that either way, but if you are setting up a new workflow on that machine, it is the moment to look at whether it should still be on Windows 10 at all.

Install on Kali Linux (and any Debian-family distro)

1. Install Ollama with the official script: curl -fsSL https://ollama.com/install.sh | sh. It installs the binary, creates an ollama system service, and detects NVIDIA or AMD cards if the drivers are present. On a bare Kali install without the NVIDIA driver, it will say so and fall back to CPU; that is fine to start with.
2. Confirm the service is up: systemctl status ollama. Should say active (running). If not: sudo systemctl enable --now ollama.
3. Pull and run: ollama run qwen3.8:27b, or the Hugging Face path for a smaller cut, exactly as on Windows: ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-IQ3_XXS.
4. In a VM? Kali in VirtualBox or VMware only sees the memory and cores you gave it, and it never sees the host’s graphics card. Give the VM at least 20 GB of RAM for the IQ3 cut or 24 GB for Q4, and expect CPU speed. For anything serious, run Ollama on the host and point the VM at it over the network (OLLAMA_HOST=0.0.0.0 on the host, then http://<host-ip>:11434 from the guest); the VirtualBox guide covers the network side.
5. Model location: /usr/share/ollama/.ollama/models. Change it with Environment="OLLAMA_MODELS=/path" in a systemd override (sudo systemctl edit ollama), then sudo systemctl restart ollama.

For the Kali crowd specifically: a 27B model that reads images is a genuinely useful thing to have next to a terminal. Paste a screenshot of a tool’s output and ask what it means; feed it a config file and ask what is wrong; ask it to write the one-liner you cannot remember. It is not a replacement for knowing the tools, and it will confidently invent a flag now and then, but as a second pair of eyes that never leaves the machine, on an engagement where nothing may leave the machine, it earns its disk space. We ran Google’s Gemma the same way in the spring; the Gemma-on-Kali guide is the smaller-model version of this page.

The switches: thinking on or off, effort level, and images

Qwen3.8 ships with thinking mode on and effort set to its highest level. That is the right default for a benchmark and the wrong one for "rewrite this email," where you will wait a minute for reasoning you did not need. Three controls:

• Turn thinking off for a session: inside the chat, type /set nothink. Turn it back on with /set think. Or start it off from the shell: ollama run qwen3.8:27b --think=false.
• Effort level: the model accepts a reasoning_effort of low, medium or xhigh (the default). Through Ollama’s API or any app that exposes the setting, low keeps the reasoning but cuts it to a fraction; on a CPU this is the setting that makes thinking mode livable.
• Sampling settings that Unsloth recommends: with thinking on, temperature 1.0, top-p 0.95, top-k 20; with it off, temperature 0.7, top-p 0.8, top-k 20. Ollama’s defaults are close enough for chat; set them explicitly (/set parameter temperature 0.7) if answers start repeating themselves.
• Images: the model reads pictures and video frames natively. In the Ollama chat, include the file path in your message: What is wrong in this screenshot? /home/jake/error.png (Windows: C:\Users\jake\error.png). Diagrams, invoices, error dialogs, whiteboard photos all work. Video is frame sampling, not real-time; give it a clip, not a stream.
• Context: 262,144 tokens natively, extensible to a million with the right serving settings. Ollama defaults to a much shorter window to save memory; raise it with /set parameter num_ctx 32768 when you feed it a long document, and expect memory use to climb with it. On a 16 GB laptop, a 32K context on the IQ3 cut is about the ceiling.

Jake’s setting, for the record, on the 32 GB office desktop he ended up using instead of the shop PC: Q4 cut, thinking off for the daily PDF work, thinking on with low effort for the once-a-week "why is this invoice different from that one" question. It answers the daily stuff in ten to twenty seconds and the hard ones in about a minute.

GLM-5.3-Flash: what "open" means when the file is 200 GB

Everything Z.ai says about GLM-5.3-Flash is, as far as the published material goes, true. The weights are on Hugging Face under an MIT license, which is about as permissive as licenses get. It is natively multimodal, it has a 300,000-token context, it has a thinking mode with low, high and max effort levels, and the architecture (sparse plus linear attention, only 18 of 320 billion parameters active per token) is genuinely clever. Unsloth’s GGUF files exist, and this is where the trending-list excitement meets the disk:

Ollama’s own library page for the model makes the same point without saying it: the only tag is glm-5.3-flash:cloud. Run that and Ollama sends your prompt to its hosted service (US and EU, with a zero-retention policy stated on the page) and streams the answer back. It works, it is fast, it is currently free to try, and it is exactly the thing the word "local" was supposed to avoid. The same goes for the Z.ai API, which is the route Z.ai actually wants you to take and which is priced at a tenth of the previous model. If your reason for wanting an open model was cost, GLM through the API is a fine answer. If your reason was that the customer’s warranty claim must not leave the building, GLM is not an answer on any hardware you can buy at a store this year, and the honest recommendation is the model in the previous sections.

Ethan on the "but it’s open source" objection: "Open means you are allowed to. It has never meant you are able to. A blueprint for a bridge is open when it is published; you still need the river, the steel and the crew. Z.ai published the blueprint, and that matters, because in two years someone will squeeze it into something a desktop can hold, or the desktop will grow. Today, the bridge is theirs, and you are welcome to drive across for a small toll."

Everything that goes wrong on the first run, and the fix

• "Error: model requires more system memory than is available." The cut is too big for the machine. Drop one row in the size table. On Windows, also check the browser is closed and that Windows is not holding a huge pagefile on a nearly full drive.
• It runs, but a word every few seconds, and the fans scream. It fell back to CPU. On Windows with an NVIDIA card, install the current driver from NVIDIA (not the one Windows Update chose), restart Ollama from the tray, and run ollama ps while a model is loaded: it shows what percentage is on GPU. On Kali, the NVIDIA driver and CUDA toolkit are the missing piece; AMD needs ROCm, which Ollama supports on a shorter list of cards.
• Partly on GPU, partly on CPU, painfully slow. A 16.5 GB model on a 12 GB card is worse than the 9.8 GB model fully on the card. Pick the cut that fits inside VRAM with room to spare rather than the biggest one that technically loads.
• The answer starts with a wall of reasoning and takes forever. That is thinking mode. /set nothink, or set effort to low.
• "Not enough space on disk." Set OLLAMA_MODELS to another drive before pulling (Windows steps above; systemd override on Kali) and delete a failed partial download with ollama rm <name>.
• Download stalls at 99%. Press Ctrl+C and run the same ollama run again; it resumes. Hugging Face pulls occasionally need two attempts.
• The model "doesn’t remember" the document you pasted a few messages ago. The context window filled up at Ollama’s default size. Raise num_ctx as above, and paste the document again.
• It answers in Chinese. Rare, and it happens with Qwen models when the prompt is ambiguous or the temperature is high. Add "Answer in English" to the first message, or set temperature to 0.7. It is a sampling quirk, not a hidden setting.
• Image path is ignored. Use an absolute path, no quotes, and a common format (PNG, JPG). On Windows, forward or back slashes both work; spaces in the path do not.
• Another app can’t reach Ollama. Ollama listens on http://localhost:11434. Point the app there. If it is on another machine, set OLLAMA_HOST=0.0.0.0 where Ollama runs and allow port 11434 through the firewall.

Does it cost anything? The electricity math and the hardware question

The model is free and the software is free; the only running cost is power. A desktop with a 24 GB card draws perhaps 350 to 450 watts while the model is generating and almost nothing while it idles between questions. At a typical U.S. rate of around 17 cents per kilowatt-hour, an hour of solid generation is about 7 cents; a realistic day of on-and-off use is a few cents. A laptop on CPU draws a fraction of that. The cloud chatbots charge $20 a month for the consumer tier; the local model repays a card you already own in roughly nothing, because you were not going to buy a $1,500 graphics card for this. Which is the actual question: should you buy hardware to run Qwen3.8-27B? For a hobbyist, no; run the cut your machine can take and enjoy it. For a small business that would otherwise pay for several seats of a cloud tool, and that has documents it would rather not upload, a used 24 GB card or a 32 GB Mac mini pays for itself inside a year and runs the Q4 cut comfortably. That is a spreadsheet decision, and the size table above gives you the rows.

Qwen3.8-27B vs Granite 4.2 vs the cloud chatbots: which to use for what

The honest limits

• It invents things. A 27B model in a 4-bit cut, offline, with no web access, will state a wrong flag, a wrong date or a wrong API with the same confidence as a right one. Treat it as a fast colleague who does not check, and check.
• The benchmarks are the maker’s. Independent leaderboards score these models lower than the launch tables, sometimes by a lot. That is normal and it does not make the model bad; it makes the headline a headline.
• Squeezed cuts lose the hard stuff first. Summaries survive 2-bit quantization; multi-step math and long code do not. If a small cut disappoints, that is the cut, not the model.
• No web, no plugins, no memory between sessions unless you add software that provides them. Ollama is the engine, not the car.
• Qwen’s training reflects its makers’ choices on certain political topics, as every model reflects its maker’s. For code, documents and diagrams this is irrelevant; for current-affairs questions, know it.
• Flash-Next and GLM are real advances you will not run. If you own a 128 GB Mac Studio, the 1-bit GLM or a squeezed Flash-Next becomes an interesting weekend; for everyone else, they are API models this year.

For IT admins and small businesses

1. One box, many users. A single 24 GB card serving the Q4 cut through Ollama’s API handles a small office; set OLLAMA_HOST=0.0.0.0, put it behind the firewall, and point the desktop apps at it. Nothing leaves the LAN.
2. Choose the license consciously. Qwen3.8-27B is Apache 2.0; Qwen3.8-Flash-Next is under the "Qwen community" license, which is not the same document and needs reading before commercial deployment. GLM-5.3-Flash is MIT, but you will be using it through an API, so the API terms govern.
3. Decide the thinking default centrally. Thinking on with xhigh effort is the quality setting and a throughput killer. Set low for the shared box and let power users raise it per request.
4. Log nothing you would not want subpoenaed. Ollama itself does not log prompts by default; the front-end app you put in front of it might. Check.
5. Pin the model version. ollama pull fetches whatever the tag points at today; for a business workflow, record the digest (ollama show qwen3.8:27b) so a silent update does not change your outputs.
6. Windows 10 boxes: fine for Ollama, but out of support since October 2025 unless on ESU. A machine that now handles company documents through an AI is a machine that should be patched. The local model does not change the exposure; it does raise the value of what is on the disk.

FAQ — Qwen3.8, GLM-5.3-Flash and running AI locally, answered straight

Can I run Qwen3.8 locally?

Yes. Install Ollama and run ollama run qwen3.8:27b (18 GB). It is fast on a 24 GB graphics card, usable on 32 GB of RAM, and runs on a 16 GB laptop only through the smaller Unsloth cuts (UD-IQ3_XXS, 10.9 GB) at a couple of tokens per second.

Can I run GLM-5.3-Flash locally?

Not on a PC. It is a 320-billion-parameter model; the smallest GGUF is 93 GB at 1-bit and the sensible 4-bit cut is 200 GB. It runs on servers, on Ollama’s :cloud tag, or through the Z.ai API. If privacy was the point, use Qwen3.8-27B or Granite instead.

How much RAM does Qwen3.8-27B need?

About 1.2 to 1.4 times the file: roughly 20–24 GB free for the Q4 cut, 14–16 GB for IQ3, 12 GB for the 2-bit cuts. VRAM if you have a card, system RAM if not. Raising the context window adds to it.

Is Qwen3.8 free for commercial use?

Qwen3.8-27B is Apache 2.0: yes, including commercial use and modification, no royalties. Qwen3.8-Flash-Next uses a separate Qwen community license that needs reading first. GLM-5.3-Flash’s weights are MIT.

Is Qwen3.8 on Ollama?

Yes: ollama run qwen3.8 pulls the 27B at 18 GB with a 256K context, and there are 27B variants including an MLX build for Macs. Other cuts come straight from Hugging Face with ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:<cut>.

How do I turn off thinking mode in Qwen3.8?

In the Ollama chat, type /set nothink; from the shell, ollama run qwen3.8:27b --think=false. Through the API, set enable_thinking to false, or keep it on and set reasoning_effort to low.

Does Qwen3.8 understand images?

Yes, natively, including video frames. In Ollama, put the file path in your message. Screenshots of errors, diagrams, invoices and photos of documents all work.

Is Qwen3.8-27B better than Granite 4.2?

At full quality, clearly, especially at code and images. On a 16 GB machine with no graphics card, Granite 4.2 8B at full precision is the better daily tool, because Qwen has to be squeezed to fit and slows to a crawl. Have both; switching is one command.

Can I run Qwen3.8 on Kali Linux in VirtualBox?

Yes, on CPU, with 20–24 GB assigned to the VM. The VM never sees the host’s graphics card, so for speed run Ollama on the host and connect from Kali over the network.

What is the difference between Qwen3.8-27B and Qwen3.8-Flash-Next?

27B is dense: 27 billion parameters, all active, fits a big PC. Flash-Next is a mixture-of-experts model, 125 billion parameters with about 6 billion active per token; it is fast and cheap on a server but needs roughly 96 GB+ of memory to hold, so it is a workstation or API model.

Is GLM-5.3-Flash really as good as the paid frontier models?

Z.ai’s own tables say it approaches Anthropic’s flagship model on coding and agent benchmarks. Those are the maker’s numbers on the maker’s tests; independent leaderboards usually land lower. It is a strong model. It is also one you will use through an API, not on your disk.

Why is GLM-5.3-Flash on Ollama if it can’t run locally?

Because Ollama has a cloud service. The glm-5.3-flash:cloud tag runs the model on Ollama’s servers and streams the answer to your terminal. Convenient, and not local: your prompt leaves the machine.

Which graphics card do I need for Qwen3.8-27B?

24 GB for the Q4 cut with room for context (RTX 3090, 4090, 5090 class). 16 GB cards run the IQ3/Q3 cuts; 12 GB cards run the 2-bit cuts. A model that spills from VRAM into RAM runs slower than a smaller cut that fits entirely.

Does running it locally cost money?

Only electricity: a few cents a day of real use on a desktop GPU, far less on a laptop. The model and Ollama are free. Buying hardware for it is only worth it for a business replacing paid seats or protecting private documents.

Why does it sometimes answer in Chinese?

A sampling quirk of Qwen models with ambiguous prompts or high temperature. Say "Answer in English" in the first message or set temperature to 0.7. Not a hidden setting.

How do I feed it a long PDF?

Raise the context first (/set parameter num_ctx 32768 or higher), then paste the text or use a front-end that handles files. The model’s native window is 262K tokens; your memory is the limit, not the model.

Will Qwen3.8 run on Windows 10?

Yes, identically to Windows 11. Windows 10 itself has been out of support since October 14, 2025 unless enrolled in ESU, which is worth fixing before you make that PC the place where your documents get processed.

Can I use Qwen3.8 with coding tools and editors like VS Code?

Ollama exposes an OpenAI-compatible server at http://localhost:11434; most tools that accept a custom endpoint can point at it. Expect the experience to depend on your hardware far more than on the model.

Revision note. Written August 28, 2026, while both models sat in Hugging Face’s trending top three. Sizes, licenses, context lengths and file sizes are from the Qwen, Z.ai and Unsloth model cards and the Ollama library pages on that date; benchmark figures are the makers’ own and are labeled as such. Ollama commands are current for its August 2026 releases and will be updated when tags change. And if you read the headline about a free model that beats the paid ones, cleared 200 GB of disk, and then found this page: the disappointment is fair, and so is the consolation. The 27B you can run would have been the best model in the world eighteen months ago, and it is yours, on your desk, for the price of a download. Start there.