Install Ollama from its official download page, run ollama run llama3.2:3b, and check ollama ps in another terminal while the model is loaded. This is a setup example, not a claim that Llama 3.2 is the newest or best model. If memory is tight, try the 1B variant first.
1. Choose a model your machine can load
“Llama” describes a family, not a single download. Parameter count, quantization, and context change the memory requirement. The 1B and 3B Llama 3.2 variants are text models; do not expect them to interpret an image just because another Llama model can.
| Your starting point | First experiment | What to verify |
|---|---|---|
| Limited RAM or CPU inference | llama3.2:1b | Can it load without heavy swapping? |
| A modest dedicated GPU or Apple silicon | llama3.2:3b | Does the full model fit with your context? |
| More memory and a working setup | Compare an 8B text model | Does answer quality justify its memory and latency? |
Use My rig to estimate the memory budget before downloading. A fit verdict is not a driver compatibility test. Check the official Ollama hardware documentation for your exact accelerator and operating system. CPU operation remains useful for a first setup even when it is slower.
2. Install the runtime and download one model
Get the installer or platform instructions from Ollama. Once the application or service is running, open PowerShell, Terminal, or your Linux shell and enter:
ollama --version
ollama run llama3.2:3bThe first run downloads model data and requires an internet connection. Subsequent local inference uses the downloaded files. A connection failure during download is different from a model failing to fit in memory. Check the exact tag against the runtime library rather than guessing a model name.
3. Verify hardware use and context
Ask for a short explanation, then open a second terminal and run:
ollama psRead the loaded model and processor placement. A CPU/GPU split means some computation uses system RAM and the CPU. A model producing an answer does not prove it is fully accelerated. Compare the actual model and context with the settings used in your memory estimate; a family name alone is not enough.
For a modest first test, set an explicit context in the interactive Ollama session:
/set parameter num_ctx 4096Context covers the conversation and generated output, not just the latest prompt. Shorter context can save memory. The model publisher’s maximum context is a ceiling, and your local configuration may use less. See the context documentation and hardware-placement FAQ for current behavior.
4. Change the limiting variable
| Symptom | Useful next check |
|---|---|
| Service cannot be reached | Confirm Ollama is running before changing the model. |
| Out of memory while loading | Close another loaded model, shorten context, or try the 1B variant. |
| Loads but answers slowly | Inspect processor placement, background memory use, and output length. |
| Weak or incorrect answers | Test a clearer prompt and another model; buying hardware alone cannot fix model quality. |
5. Verify the workflow is local
Choose a downloaded local model rather than a cloud option. After downloading everything required, test the intended chat with networking disabled. External tools, web search, plugins, and cloud endpoints can still send data elsewhere even when the base model is local. Inspect those integrations separately.
Keep the runtime on its default local interface unless you understand how you will control network access. Before commercial use, read the Llama license. This guide describes a reproducible setup path; MyLocalRig has not benchmarked every hardware combination.
Sources and review notes
Reviewed October 21, 2026. Official documentation and publisher data support the setup details. Memory examples are calculations, not firsthand benchmark results. Versions, defaults, and hardware support can change.