Running an AI model on your own PC is no longer reserved for workstation owners, but the hardware still determines what you can use comfortably. A typical computer with 16GB of RAM can handle smaller models in the seven- to eight-billion-parameter range, while a graphics card with 8GB of VRAM can make those models substantially faster. The basic hardware picture is surprisingly approachable: the challenge is matching the model to the memory available.
For newcomers, model size is the most useful rule of thumb. Ollama’s commonly cited guidance calls for roughly 8GB of memory for a 7B model, 16GB for a 13B model, and 32GB for a model around 33B. Those figures describe the memory needed to load model weights, before allowing extra room for the operating system, the application, conversation history, and longer context windows. A machine may technically launch a model while still feeling painfully slow or running out of memory during a larger prompt.
Quantization is what makes local AI practical on consumer hardware. Compressed versions of models use fewer bits to represent their weights, reducing memory requirements at the cost of some precision. In practical terms, a 4-bit version of an 8B model is a much friendlier target for a mainstream PC than a full-precision release. The tradeoff becomes dramatic at the high end: a trillion-parameter model such as Moonshot’s Kimi K2.6 reportedly requires about 600GB of combined RAM and VRAM even at 4-bit quantization, putting it far beyond an ordinary desktop.
The graphics card is helpful rather than universally mandatory. CPU-only inference works for small models, but responses are generally faster when some or all of the model is moved into GPU memory. LM Studio recommends at least 4GB of dedicated VRAM on Windows, while its official documentation recommends 16GB of system RAM overall; Apple Silicon Macs can use unified memory, with 16GB recommended and 8GB workable for smaller models. LM Studio supports Apple Silicon, x64 and ARM Windows systems, and x64 or ARM64 Linux machines. Its current requirements also note that Intel Macs are not supported and that Windows x64 systems need AVX2 support.
Ollama offers another straightforward route, with support for Nvidia GPUs and acceleration options for AMD hardware through ROCm or Vulkan. Apps built around Ollama, llama.cpp, or LM Studio can download models from libraries and let users test different sizes without committing to a new computer. Storage matters too: model files can occupy several gigabytes each, and keeping multiple quantized variants quickly consumes an SSD.
For most people, a sensible starting point is a modern processor, 16GB of RAM, a fast SSD, and either an 8GB graphics card or an Apple Silicon Mac with comparable unified memory. That setup is enough to explore compact chat, coding, and image-capable models, though larger context windows and bigger models will push beyond it. Moving into the 30B class typically means 24GB or more of VRAM or a much larger pool of system memory, while 70B models can require workstation-level hardware. The cheapest upgrade is often not a new GPU but more RAM—provided the motherboard supports it.
