Running AI on your own computer, explained simply

You don't need a data center or a computer science degree. Using open source software, you can run capable AI models entirely on your own machine, with no subscription, no rate limits, and nothing leaving your desk.

Which hardware? NVIDIA, AMD, Intel & Mac

All four can run local models. They just make different trade-offs on speed, memory, and how much setup you'll do.

The key consideration: GPU memory

Smarter models need more memory to run. On a graphics card (GPU) that's called VRAM. Sizes below assume 4-bit quantization (the standard compression that shrinks typical 16-bit model weights to roughly a quarter of their size for a small quality cost). Newer “mixture-of-experts” models bend the rule: they run usefully, if more slowly, on a midrange card plus plenty of system RAM, so don't rule out a smaller GPU to start.

Used cards are the value play

Used 24GB cards like the RTX 3090 cost roughly half of new flagships and run the same models. When yours arrives: stress test it on day one (a long image-generation run works well), watch temperatures, and confirm the full VRAM is usable. Every Jawa order is buyer-protected: if the card isn't as described, report it within 2 days of delivery to get your money back.

Getting started: software

A few free apps cover almost everything. Start at the top and work down as you get comfortable.

Step-by-step setup guides for Windows, Mac, and Linux are on the way. Want a nudge when they're live? Join the Discord.

Getting started: models

Open-weight models live on Hugging Face. For local apps, look for GGUF files (the format LM Studio, Ollama, and llama.cpp use). The landscape moves fast, so treat these as starting points. One caveat: the giant open-weight models in the headlines, like DeepSeek and Kimi, weigh hundreds of gigabytes even compressed and need server hardware. The families below are the ones that run well at home.

Terms you'll keep hearing

Model

The AI itself. You download it once as a file, then run it as often as you like.

LLM

Large language model: the text-based AI behind chatbots, writing help, and coding tools, and the kind most people run at home.

Parameters

A model's internal settings, also called its weights, counted in billions (a '7B' model has 7 billion). More parameters buy more capability but demand more memory.

Token

A chunk of text, roughly ¾ of a word. A model’s speed is measured in tokens per second (tok/s).

Context window

How much text a model holds in mind at once, counting both your prompt and its reply. A bigger window fits more files or a longer conversation.

VRAM

The memory on your graphics card. It is the single biggest factor in which models you can run.

Unified memory

Apple Silicon's shared pool of memory for the CPU and GPU. It lets a Mac load unusually large models.

Quantization

Shrinking a model to fit in less memory. You trade a little quality for a lot of room.

Inference

Running a model to get an answer. It happens every time you send a prompt, and how fast it runs is your tokens per second.

Training

Teaching a model from scratch on huge amounts of data. It takes enormous compute, so almost nobody does it at home.

Fine-tuning

Adapting a finished model to your own data or task. It is lighter than training, but still leans on a strong NVIDIA GPU.

Open-weight model

A model whose trained weights are published, so anyone can download and run it. Qwen, Gemma, DeepSeek, and Kimi are examples.

A word on model sizes

You download each model once and keep it on disk, and the files are big. A single model is anywhere from about 4 to 40GB, and most people collect a few, so a roomy, fast SSD fills up quicker than you'd expect. To run, a model has to load into memory, ideally your GPU's VRAM, or system RAM when it spills over, so disk and memory together decide how much you can keep on hand and how much you can run at once.