Run AI Locally on Windows and Kali: The Free Guide Series

This page is the front door to every guide on this site about running AI locally on Windows and Kali Linux: which model to download for the computer you actually own, which app to run it in, what a laptop needs, and which famous models you cannot run at home no matter what the headlines say. Everything is free, nothing needs an account, and the surprise that removes most of the fear up front: you do not need a gaming PC. A laptop with 8 GB of graphics memory, or a Mac with 16 GB, runs models this year that needed a server two years ago. Start with the reading order below, or jump straight to the tier that matches your machine.

Why this page exists, and who it's for

Jake runs a phone-repair shop, and last spring he asked a question that a lot of people ask in a slightly embarrassed voice: "Can I run one of these AI things on my own computer, so my customer notes never leave the shop?"

The answer was yes, and it still is. But between yes and a working chat window sat the same problems every beginner hits. Which model? The internet names forty. Which app? Five, all claiming to be the easy one. Will it run on this laptop? Nobody says, because the answer depends on one number most guides never explain. And the worst one: the model in every headline that week turned out to need more memory than the shop's entire fleet of computers put together.

"So half the internet was telling me to download things that could never work," Jake said.

"Not lying," Ethan said. "Just not telling you the one thing that matters. A model is a file. The file has to fit in your memory. Everything else on this page follows from that sentence."

This hub is the map Jake wished he'd had. Every post it links to is on this site, free, written the same plain-English way, with Jake asking the awkward questions and Ethan answering them. The order below is the order that works: basics, then the app, then a model that fits, then the fun stuff.

Read in this order: from zero to a working local model

1. What is an LLM? And why AI confidently makes things up: the one-page foundation. What a model is, what it isn't, why it sounds sure when it's wrong.
2. The ultimate guide to installing a local LLM on your computer: GGUF files, VRAM, quantization, and the first install, explained from zero.
3. Ollama vs LM Studio vs Jan vs Unsloth: which app to run models in, and what each one can and can't load.
4. Best laptops for local LLMs in 2026: what "8 GB of VRAM" really buys, tier by tier, with the September 2026 chips.
5. Then pick a model from the tier that matches your machine, below.

Which tier is for you? A one-minute decision

Answer one question honestly and the rest of this page shrinks to a few links.

1. How much memory can a model use on your machine? On Windows, that's your graphics card's dedicated memory (Task Manager → Performance → GPU). On a Mac, it's the unified memory on the box. No graphics card at all? Count half your system RAM.
2. Under 6 GB? You're in the small tier: MiniCPM5-2B first, then the smallest Gemma 4.
3. 8 GB, or a 16 GB Mac? The 8 GB tier: MiMo 9B for code, Gemma 4 or Ornith for chat, Bonsai 2 27B for reasoning if you'll do the fork install.
4. 16 to 24 GB, or a 32 GB Mac? The bigger tier: Qwen3.8 27B and Ornith 35B.
5. 128 GB unified (M5 Max, Ryzen AI Max+ 395)? You can run 70B-class models; the laptops guide explains which ones, and how fast.

Best local LLMs for an 8 GB graphics card (or a 16 GB Mac)

This is the most common machine our readers own, and the good news is that it is the tier with the most choice.

"Which one, though?" Jake asked. "I don't want five."

"Coding: MiMo. Writing and everyday questions: Gemma 4 or Ornith. The most brain per gigabyte, if you'll spend an evening on the install: Bonsai 2," Ethan said. "You can have all of them. They're files."

Best small local LLMs: 4 GB cards, old laptops and CPU-only

If your laptop has no graphics card worth the name, or 4 GB at most, you are not locked out; you are choosing from the small end.

• MiniCPM5-2B: a 2B model that runs on almost anything, with a surprisingly capable vision side. The right first model for a weak machine.
• Run Google's Gemma locally, the beginner's guide: the gentlest walkthrough on the site, written for people who have never opened a terminal.
• CPU only? Any model at 4-bit that is smaller than half your RAM will run; it will just answer at reading pace. The laptops guide has a section on the best LLMs for CPU-only computers.

Bigger machines: 16 to 24 GB cards and 32 GB+ Macs

• Run Qwen3.8 locally on Windows and Kali: the 27B general-purpose model most of this year's compact models are built from, and the honest reason GLM can't follow it onto your machine.
• GLM-5.3 local installation guide: what runs, what doesn't, and the sizes that fit 24 GB.
• Gemma 3 27B: how much memory it really needs: the older post that first taught this site's readers the file-size rule.
• Ornith 1.5, the 35B version: the step up from the 9B when you have the room.

The famous models you can't run at home (and what to use instead)

This section exists because it saves more wasted downloads than any other on the site. "Open weights" means you're allowed to run a model. It says nothing about whether it fits.

Beyond chat: images, video and music on your own machine

• Install Qwen-Image-2.1 on Windows and Kali: open image generation that runs on a consumer card, with the honest memory numbers.
• LTX-2.5: open AI video with sound, the honest GPU math: why 16 GB is out and 24 to 32 GB is plausible.
• YuE2-3B: self-host AI song generation and YuE2 in ComfyUI, the complete install: music with vocals, on AWS, Windows or Kali.

The privacy and money questions readers ask most

• Is ChatGPT Plus worth it in 2026?: $20 a month against free, against a local model. The arithmetic, not the hype.
• Why you should stop doing the ChatGPT "80s photo" trend: what you hand over with every uploaded photo, and the local alternative.
• ChatGPT Atlas: how to delete your browsing history and ChatGPT Atlas explained: the AI browser, and its privacy controls.
• Can AI read your spreadsheets? The honest answer and What is an AI agent?: two more plain-English foundations.

What's inside each guide (including the surprise in each one)

Every post in this series carries one true, counterintuitive thing most tutorials skip. Here they are, so you can pick by the fact you most need.

The ten words that make every guide readable

- VRAM: the memory on a graphics card. The number that decides which models fit on Windows laptops. - Unified memory: on Macs and some new AMD and Qualcomm chips, one pool of memory shared by CPU and GPU. Why a 128 GB Mac can load a model no 24 GB graphics card can. - Parameters (7B, 27B, 70B): the model's size in billions of weights. Bigger usually means smarter and always means more memory. - Active parameters: in mixture-of-experts models, how many weights work on each token. It sets speed, not memory; all the parameters still have to be loaded. - GGUF: the file format Ollama, LM Studio and Jan read. One file, one model. - Quantization (Q4_K_M, Q8_0, ternary): compressing weights to fewer bits so the file fits. Q4 is the usual sweet spot; ternary is the extreme end that made Bonsai 2 possible. - Context window: how much conversation the model remembers, measured in tokens. Bigger contexts need more memory, on top of the file. - Tokens per second: speed. Below about 5 feels slow; 20 and up feels like typing. - Thinking mode: the model writes reasoning before answering. Better on hard problems, slower on easy ones. - mmproj / vision: the extra file that lets a text model look at images.

Start today: the shortest path

If you want a model running in the next half hour, on Windows: install LM Studio, search for the model from your tier above, download the Q4_K_M file, and start with a question you already know the answer to. On Kali: install Ollama with its one-line script and run the model's tag. Each guide above has the exact commands for both, and a troubleshooting table for when the first attempt doesn't work; on this site, the failure modes are part of the guide, not an afterthought.

How to get the most out of this series

Three habits, learned from readers. Check the file size first, every time; it is the whole "will it run" question. Read the "can't run" table before the news cycle gets you; the sibling model that fits is usually two clicks away on the same Hugging Face page. And run your own test on your own task before believing a benchmark; every post on this page tells you which numbers are the maker's own and which have been checked by other people.

"I still can't believe the 27B runs on the shop laptop," Jake said.

"Neither could the laptop," Ethan said. "It's fine. It got used to it."

What's next in the series

New open models arrive most weeks, and the pattern on this site is the same each time: if a model can run on a 16 GB laptop, it gets a same-day guide in the same format as the ones above, with the memory numbers and the honest limits. The next candidates on the watch list are the ternary and low-bit models that follow Bonsai 2, and whichever of the big labs ships a compact open sibling next. This page is updated whenever a new guide lands, so it is worth bookmarking.

Local AI: the questions everyone asks first

How do I run an LLM locally on Windows?

Install a runner such as Ollama or LM Studio, download a model file that fits your memory (a 5-6 GB file for an 8 GB graphics card), and open it in the app. The beginner's install guide on this page walks through it step by step, and the tool comparison explains which app suits you.

Can AI run locally on a normal laptop?

Yes. A laptop with an 8 GB graphics card runs 7B-9B models comfortably and the 6 GB Bonsai 2 27B; a 16 GB Mac runs the same. Laptops without a graphics card can still run small models such as MiniCPM5-2B on the CPU, at reading pace.

What is the best local LLM for an 8 GB graphics card?

For coding, Xiaomi MiMo V2.6 9B (5.84 GB). For general chat and writing, Gemma 4 or Ornith 1.5 9B. For the most reasoning per gigabyte, Ternary Bonsai 2 27B, which needs PrismML's own llama.cpp build to run.

What is the best free local LLM?

Every model on this page is free to download and run. Which is best depends on the job: MiMo 9B for code, Gemma 4 for everyday chat, Qwen3.8 27B if you have 24 GB, and MiniCPM5-2B if your machine is weak. None needs an account or subscription.

What is the best local LLM for coding?

On an 8 GB card, Xiaomi MiMo V2.6 9B, which was trained specifically on coding and agent tasks. With 24 GB, Qwen3.8 27B. Bonsai 2 27B is strong on everyday coding tests but weaker on long, unattended agent work.

What is the best small local LLM?

MiniCPM5-2B is the strongest very small model on this site's list, runs on almost any laptop, and can read images. Gemma 4's smaller sizes are the next step up.

Which LLM is best to run on a CPU only?

Pick a model at 4-bit that is smaller than half your RAM: MiniCPM5-2B or a 7B-9B model on a 16 GB machine. Expect answers at reading pace rather than a stream. The laptops guide covers CPU-only choices in detail.

Can I run DeepSeek locally?

Not the headline versions. DeepSeek V4.1 Flash and V4-Pro have hundreds of billions of parameters, and even compressed files run to hundreds of gigabytes. The guides linked above explain what will run instead.

Can I install Claude or Gemini on my PC?

No. Claude Opus 5.5 and Gemini 3.8 are closed models with no weights to download. Google's Gemma 4 is the open family you can run locally, and it is covered on this page.

Which is better for local AI, a Mac or a Windows laptop with NVIDIA?

For models under about 27B, an NVIDIA RTX laptop is faster because its memory bandwidth is far higher. For the biggest models, only machines with large unified memory (an M5 Max with 128 GB, or an AMD Ryzen AI Max+ 395) can load them at all. The laptops guide has the September 2026 numbers.

What does GGUF mean?

GGUF is the file format llama.cpp-based apps (Ollama, LM Studio, Jan) use for models. The letters after the name, such as Q4_K_M, describe how compressed the weights are. Q4 is the usual sweet spot between size and quality.

How much RAM do I need for a 7B or 9B model?

About 6 GB of free memory for a 4-bit file, so an 8 GB graphics card or a 16 GB Mac. On a CPU-only PC, 16 GB of system RAM is comfortable.

Is running AI locally private?

Yes. A local model sends nothing anywhere; your prompts, files and images stay on the machine. That is the main reason to run one for customer data, work documents or personal photos.

Do I need Kali Linux for this, or does Windows work?

Windows works for everything on this page. The Kali steps exist because many readers run it; the tools are the same, and each guide gives both sets of instructions.

Why does the model answer confidently when it's wrong?

Because a language model predicts likely words rather than checking facts. The "What is an LLM?" post explains this in plain English, and it is the single most useful thing to understand before trusting any model's answer.

Can I run AI image or video generation locally too?

Yes. Qwen-Image-2.1 generates images on a consumer card, LTX-2.5 makes short videos with sound on 24 GB or more, and YuE2 makes music with vocals. Each has its own install guide above with honest memory numbers.

Which app should I use: Ollama, LM Studio or Jan?

LM Studio is the gentlest on Windows, Ollama is the most scriptable and works the same on Windows and Kali, and Jan is the lightweight open alternative. The comparison post on this page goes through each, including what they cannot load.

If you've spent an evening downloading something that could never run on your machine, you are in the majority, and it was never your fault: the pages that told you to do it left out one number. Every guide on this page puts that number first. Pick the tier that matches your computer, follow one link, and you will be chatting with a model of your own tonight. If a guide here doesn't match what you see on your screen, tell me through the contact page; this series stays honest because readers write in.

Revision note. Written September 26, 2026. Happy building; a private model on your own machine is a lovely thing to have.