Running AI locally on your Mac means your data never leaves your computer. No API keys, no subscriptions, no cloud servers reading your prompts. Apple Silicon makes this practical — here’s how to set it up.

Why Run AI Locally?

• Privacy — prompts and responses stay on your machine
• No internet required — works offline, on planes, in restricted networks
• No cost — no API fees or monthly subscriptions
• No rate limits — run as many queries as your hardware allows
• Speed — no network latency for simple tasks

The tradeoff: local models are smaller and less capable than GPT-4o or Claude Opus. But for everyday tasks — summarization, drafting, code completion, Q&A — they work well.

Option 1: Ollama (Easiest)

Ollama is the simplest way to run local models on Mac. One install, one command.

Install

brew install ollama

Or download from ollama.com.

Run a Model

ollama run llama3.2

That’s it. Ollama downloads the model and starts an interactive chat. First run takes a few minutes to download; after that it starts instantly.

Popular Models for Mac

Model	Size	Good For	RAM Needed
llama3.2 (3B)	2 GB	Quick tasks, drafts	8 GB
llama3.1 (8B)	4.7 GB	General purpose	16 GB
mistral (7B)	4.1 GB	Fast, good at code	16 GB
codellama (7B)	3.8 GB	Code generation	16 GB
llama3.1 (70B)	40 GB	Near-cloud quality	64 GB+
deepseek-coder-v2	8.9 GB	Advanced coding	32 GB

Rule of thumb: You need roughly 2x the model size in RAM. An 8B model needs ~16 GB RAM to run comfortably.

Ollama as a Server

Ollama runs a local API server at http://localhost:11434. Any app that supports the OpenAI API format can connect to it — including Dottie.

# List running models
ollama list

# Pull a model without running it
ollama pull mistral

# Serve the API (runs in background)
ollama serve

Option 2: MLX (Apple-Optimized)

MLX is Apple’s machine learning framework, built specifically for Apple Silicon. Models compiled for MLX run faster than Ollama on M-series chips because they use the GPU’s unified memory directly.

Install

pip install mlx-lm

Run a Model

mlx_lm.generate --model mlx-community/Llama-3.2-3B-Instruct-4bit --prompt "Summarize this email:"

MLX models are hosted on Hugging Face under the mlx-community organization. There are 3,800+ models converted to MLX format.

When to Use MLX vs Ollama

	Ollama	MLX
Setup	One command	Requires Python
Speed on Apple Silicon	Good	Better (native Metal)
Model library	100+ official	3,800+ on Hugging Face
API server	Built-in	Manual setup
Best for	Beginners, quick use	Performance, variety

For most people, start with Ollama. Switch to MLX if you need faster inference or access to specific models. For a detailed feature-by-feature comparison of both runtimes, see our Ollama vs LM Studio guide.

Option 3: Dottie (Local Models + Mac Automation)

Dottie connects to both Ollama and MLX models and adds something neither has alone: the ability to act on the output. Instead of just generating text, Dottie uses local models to control your Mac.

Setup

1. Download Dottie (free)
2. Install Ollama (brew install ollama)
3. Open Dottie → Settings → select a local model
4. Talk or type — Dottie uses the local model to understand your request and execute it

What This Enables

With a cloud AI assistant, you ask “organize my Downloads folder” and get instructions. With Dottie running a local model, it actually moves the files — and none of your file names, folder structure, or data ever leave your Mac.

Example workflows with local models:

• “Summarize the PDF on my desktop” — reads the file locally, no upload
• “Find all screenshots older than 30 days and move them to trash” — executes with system tools
• “What’s on my screen right now?” — OCR runs locally
• “Draft a reply to this email” — reads the email, writes a response, all offline

Which Local Model to Use with Dottie

Mac	Recommended Model	Why
M1/M2 (8 GB)	llama3.2 3B	Fast, fits in memory
M1/M2 Pro (16 GB)	llama3.1 8B	Good balance of speed and quality
M1/M2 Max (32 GB+)	llama3.1 70B	Near-cloud quality, fully local
M3/M4 (any)	mistral or llama3.1 8B	Optimized for latest chips

Performance Tips

Close memory-heavy apps before running large models. Safari with many tabs, Docker, and Xcode compete for the same unified memory.

Use quantized models (4-bit or 8-bit) for the best speed/quality ratio. Most Ollama models are already quantized. For MLX, look for models with -4bit or -8bit in the name.

Monitor with Activity Monitor — check the Memory tab. If you see heavy swap usage, the model is too large for your RAM.

Keep Ollama running in the background. Cold starts take 2-3 seconds; warm responses are near-instant.

Cloud vs Local: When to Use Each

Use Case	Local	Cloud
Sensitive documents	Yes	No
Quick drafts and edits	Yes	Yes
Complex reasoning	No (use cloud)	Yes
Code generation	Yes (8B+ models)	Yes
Image generation	Limited	Yes
Offline / travel	Yes	No
Cost-sensitive	Yes (free)	Varies

The best setup is both: local models for private, everyday tasks and a cloud provider for complex reasoning when you need it. Dottie lets you switch between local and cloud mid-conversation. For a full overview of how Dottie uses local models alongside its 134 system tools, see What Is Dottie Desktop?.

Frequently Asked Questions

What's the minimum Mac for running local AI models?

An Apple Silicon Mac (M1 or newer) with 8 GB RAM can run small 3B-parameter models like Llama 3.2 3B. For 7B–8B models, plan on 16 GB. For 70B models, 64 GB+ is recommended. Intel Macs work but are significantly slower because they lack the unified memory architecture.

Which is better for Mac, Ollama or LM Studio?

Ollama is better for developers and scripting (CLI-first, easy API integration). LM Studio is better for non-technical users and model exploration (GUI-first, built-in HuggingFace browser). Performance on Apple Silicon is within 5% of each other. See our full [Ollama vs LM Studio](/blog/ollama-vs-lm-studio/) comparison.

How much RAM do I need for local AI models?

Rule of thumb: about 2x the model's on-disk size. A 4 GB model needs ~8 GB free RAM. A 40 GB model needs ~80 GB. Quantized 4-bit models use roughly half as much as unquantized versions with minimal quality loss.

Can I run Llama 3 on an M1 MacBook?

Yes. M1 MacBooks with 8 GB RAM run Llama 3.2 3B smoothly. 16 GB M1/M1 Pro machines run Llama 3.1 8B with good performance. The 70B version requires a 64 GB M1 Max/Ultra or better.