Your first local model. From zero to a first answer.
What to prepare, how to choose a first model, and how to check that inference really happens on your Mac.
Start with one concrete task
You do not need to build an entire home AI server on day one. Pick one task: rewrite a paragraph, summarize a text, or explain some code. Make the first success small and easy to verify.
Before installing anything, note your chip, memory capacity and free disk space. On a Mac, find these in Apple → About This Mac and System Settings. A model file’s size is not its total runtime memory requirement.
One option: Ollama
Ollama offers a macOS application and a command-line interface. Get the installer from its official website and check the current system requirements. Start with a smaller model from the library that suits your hardware and intended use.
The command below is a template: replace MODEL with the exact model name and variant from the library. The first run downloads the model and needs an internet connection.
ollama run MODELCheck local inference
Select an actual local model rather than a cloud variant. After downloading the model and required files, try the same prompt with the network disconnected. This checks basic generation without a remote service; it is not an audit of every network activity in the application.
Ask something you can independently verify. Look beyond speed: assess language quality and whether the model handles uncertainty. Fully local models can still produce incorrect answers.
Record your conditions
Save the model name and variant, application version, input length and Mac configuration. Without those details, later speed or quality comparisons have little value.
A friendlier interface or working with your documents can come next. First confirm that the basic setup works and handles your actual task.