What Is Dottie Desktop? The AI Agent That Controls Your Mac

What Is Dottie Desktop? The AI Agent That Controls Your Mac


Last updated on

Dottie Desktop is a free, open-source AI agent for macOS that controls your Mac through voice and text commands. It executes actions directly — moving files, opening apps, reading your screen, sending emails, managing calendar events — using 134 built-in system tools. It runs on Apple Silicon Macs with macOS 15.4 or later.

How Dottie Works

Dottie is not a chatbot. It is an agent. The difference is execution. A chatbot tells you how to do something. An agent does it. Dottie uses tool calling — a technique where the AI model receives your request, decides which system actions are needed, and executes them. If you say “move all the screenshots from my Desktop to a folder called Archive,” Dottie does not print shell commands for you to copy. It creates the folder, finds the screenshots, and moves them.

The architecture behind this is a set of local servers running on your Mac. The Gateway server, built with Node.js and Hono, orchestrates everything on port 1317. It receives your input, routes it to the active AI model (local or cloud), interprets the model’s tool call responses, executes the corresponding system actions, and streams results back to you. An Audio server written in Python with MLX handles speech recognition and text-to-speech on port 1315. A separate LLM server handles local model inference on port 1316. All three run on localhost. Nothing is exposed to the network.

Responses stream token-by-token using Server-Sent Events (SSE). You see the AI’s response forming in real time, and tool results appear inline as they complete. Dottie also includes DotUIKit — a set of rich UI components that render directly in the conversation. When you ask about the weather, you get a weather card with temperature, conditions, and a forecast. When you ask about your calendar, you get interactive event cards. Timers, confirmation dialogs, and media controls all render as native UI elements, not plain text.

Voice-First Design

Dottie is built around voice. Three modes cover different use cases.

Push-to-Talk is for dictation. Hold Option+Space, speak, and release. Your words appear at the cursor position in whatever app is focused — a text editor, a browser form field, a Slack message. Speech recognition runs locally using the parakeet-tdt-0.6b model through MLX. No audio leaves your Mac. The hotkey is configurable: you can set it to Option+Space, Spacebar, Right Option, Right Command, Fn, Tab, or Caps Lock depending on your workflow.

Chat mode is for agent interaction. Click the microphone button in Dottie’s window, and your voice becomes a message to the AI with full tool access. Say “send an email to Sarah about the project deadline” and Dottie composes and sends the email through Mail.app. Chat mode gives the AI the same capabilities as text input — it just starts with your voice.

Wake Word mode is hands-free. Say “Hey Dottie” from anywhere on your Mac. On-device processing via Apple’s Speech framework detects the phrase without sending audio to any server. A floating overlay panel appears with your transcribed words and three options: Send (execute the command), Edit (modify before sending), and Dismiss. This mode is useful when your hands are occupied — cooking, exercising, or working on hardware.

Text-to-speech uses the Kokoro-82M model with multiple voice options. You can adjust speed from 0.5x to 2x and toggle auto-speak so Dottie reads every response aloud. In Realtime mode, barge-in is supported — you can interrupt Dottie mid-response by speaking, and it will stop talking and process your new input. macOS Services integration lets you right-click selected text in any app and choose “Speak with Dottie” to send it as input. For a detailed walkthrough of all three voice modes and tips for better recognition, see the voice control guide.

134 Tools

Dottie ships with 134 built-in tools organized across 15 permission categories. Each category maps to a macOS capability.

Email. Read your inbox, search messages, and send emails through Mail.app. Dottie composes the email, shows you a preview, and sends on confirmation.

Calendar. Read upcoming events, create new ones, update existing events, and delete them. Dottie accesses your Calendar.app data directly. Ask “what’s on my calendar this week” and get a structured list. Say “schedule a meeting with Dan at 2 PM tomorrow” and it creates the event.

Reminders. Full read and write access to Reminders.app. Create, complete, and delete reminders. Dottie can read your lists and add items with due dates.

Contacts. Address book access through Contacts.app. Look up phone numbers, email addresses, and other contact details. Used automatically when you reference people by name in email or calendar commands.

Notes. Read from and write to Notes.app. Search your notes, create new ones, and append to existing notes.

Safari. See your open browser tabs, their URLs, and page titles. Useful for context — “summarize the article in my current Safari tab” works because Dottie can see what you are reading.

Files. Search for files on disk by name, type, or content. Open files, reveal them in Finder, or use them as context for AI tasks.

Photos. Browse your photo library. Search by date, location, or content.

Music. Browse your Apple Music library and control playback. Play, pause, skip, adjust volume, and search for songs or albums.

Screenshots. Capture your screen and analyze the content. Dottie can take a screenshot, read text from it using vision, and act on what it sees. This is how Dottie “reads your screen” — it captures a frame, processes it with a vision-capable model, and uses the extracted information to answer questions or take actions.

Weather. Current conditions, forecasts, and severe weather alerts for any location. Rendered as rich weather cards in the conversation.

Sports. Live scores, upcoming games, standings, and team information.

Stocks. Current prices, daily changes, and basic market data.

Maps. Location search, directions, and distance calculations.

App Control. Open, close, and focus any application. “Open Xcode” opens Xcode. “Close all Finder windows” closes them.

System Commands. Terminal access for running shell commands. Dottie can execute arbitrary commands when you grant system tool permissions.

Web Search. Search the web and return structured results.

Image Generation. Generate images from text descriptions using AI models.

Security is strict by default. All 15 tool permission categories default to OFF. You must explicitly enable each one in Settings. When Dottie attempts a destructive action — deleting a file, canceling a calendar event, removing a reminder — a confirmation dialog appears before execution. You always have the option to deny.

AI Models — Local and Cloud

Dottie supports both local and cloud AI models. You choose based on your priorities: privacy, speed, capability, or cost.

Local models run directly on your Apple Silicon Mac through MLX. The default local LLM is LFM2.5-1.2B-Thinking — a small but capable model that handles conversational tasks and basic tool calling. Speech-to-text uses parakeet-tdt-0.6b for fast, accurate transcription. Text-to-speech uses Kokoro-82M for natural-sounding voice output. All three run on-device with zero network traffic. You can install any model from the mlx-community collection on HuggingFace by pasting its model ID into Dottie’s settings.

Ollama integration lets you use any model served by a local Ollama instance. If you already have Ollama running with a model you like, Dottie can connect to it.

Cloud providers offer more powerful models when you need them. Each requires your own API key:

  • xAI: Grok 4.1 Fast Reasoning, Grok 4 Fast, Grok 3, Grok 2
  • OpenAI: GPT-5, GPT-4o, GPT-4.1, o3-mini
  • Anthropic: Claude Opus 4.6, Claude Sonnet 4.6, Claude Haiku 4.5
  • Cerebras: Llama 3.1 8B, Qwen 3 235B, GPT-OSS 120B, ZAI GLM 4.7

For deeper reasoning on complex tasks, prefix your message with /think. This triggers extended thinking mode, where the model takes additional time to reason through multi-step problems before responding. Useful for debugging, analysis, and planning tasks.

Agent Features — Goals, Triggers, and Scheduled Jobs

Beyond responding to direct commands, Dottie has three features that let it act on your behalf over time.

Goals let you define objectives for the agent to work toward. A goal is a persistent instruction that influences how Dottie responds and what actions it prioritizes. Set a goal like “help me stay on top of my email” and Dottie will proactively surface unread messages, suggest responses, and flag messages that need attention.

Triggers set conditions that automatically invoke actions. Define a trigger like “when I get an email from my boss, notify me immediately” and Dottie monitors your inbox and alerts you when the condition is met. Triggers connect Dottie’s monitoring capabilities to its tool execution, creating reactive workflows without manual scripting.

Scheduled Jobs use CRON-based scheduling to run tasks at specific times. The most common example is Morning Brief — a daily summary of your calendar, weather, reminders, and unread emails delivered at the time you choose. You can create custom scheduled jobs for any repeating task: weekly report generation, daily file cleanup, or periodic system checks.

Memory makes Dottie persistent across sessions. User memory stores editable facts about you — your name, preferences, work context, and anything else you want Dottie to remember. AI memories are automatically generated observations stored in a SQLite-backed knowledge graph. Both persist across app restarts. Dottie references these memories to personalize responses and avoid asking you the same questions repeatedly.

Privacy and Security

Dottie’s privacy model is straightforward: nothing leaves your Mac unless you explicitly choose a cloud provider.

All speech recognition runs on-device through MLX. Text-to-speech runs on-device through Kokoro-82M. Local LLM inference runs on-device. When you use only local models, there is zero network traffic to any AI provider. Your conversations, file contents, screen captures, email text, calendar data, and tool outputs stay on your machine.

Cloud providers are opt-in and require your own API key. When you use a cloud model, your messages are sent to that provider under their terms of service. Dottie does not add any intermediary server — requests go directly from your Mac to the provider’s API.

There is no telemetry and no data collection. Dottie does not phone home, track usage, or report analytics. All 15 tool permissions default to off. Destructive actions require confirmation dialogs. The application is sandboxed — it only accesses what you explicitly permit. To see how Dottie stacks up against ChatGPT, Raycast AI, Apple Intelligence, and other tools, read our ranked comparison of the best AI assistants for Mac.

Dottie registers URL schemes that let you trigger it from scripts, Shortcuts, or other applications.

  • dottie://record — Start voice recording
  • dottie://chat?text=your message here — Send a text message to the AI
  • dottie://type?text=your text here — Type text at the current cursor position
  • dottie://settings — Open the Settings window
  • dottie://show — Show the Dottie window
  • dottie://hide — Hide the Dottie window
  • dottie://stop — Stop the current AI response

These work with macOS Shortcuts, shell scripts, Alfred workflows, Raycast commands, or any tool that can open URLs. You can build complex automations that use Dottie as the AI execution layer — a Shortcut that captures selected text, sends it to Dottie for analysis, and returns the result. For practical examples of voice-driven and scripted automations, see our guide on how to automate your Mac with AI.

How to Get Dottie

Dottie Desktop is free and open source under the MIT License. Download it from GitHub at github.com/stevederico/Dottie.

Requirements:

  • macOS 15.4 or later
  • Apple Silicon (M1, M2, M3, M4, or later)

On first launch, Dottie downloads approximately 2 GB of default AI models — the local LLM, speech-to-text model, and text-to-speech model. After that initial download, everything runs locally. Cloud providers are optional and require your own API keys.

Frequently Asked Questions

Is Dottie Desktop free?

Yes. Dottie Desktop is completely free and open source under the MIT License. Cloud AI providers require your own API key, but local models run at zero cost.

What Mac do I need to run Dottie?

Dottie requires an Apple Silicon Mac (M1 or later) running macOS 15.4 or newer. First launch downloads approximately 2 GB of default AI models for speech and chat.

Can Dottie work without internet?

Yes. Dottie runs local AI models through MLX for chat, speech-to-text, and text-to-speech -- all processing happens on your Mac. Cloud providers are optional and require internet.

How many tools does Dottie have?

Dottie has 134 built-in tools spanning email, calendar, reminders, contacts, notes, files, photos, music, Safari tabs, screenshots, weather, maps, stocks, sports, web search, image generation, and system commands.

Is Dottie safe to use?

Yes. All 15 tool permissions are off by default -- you choose exactly what Dottie can access. Destructive actions require confirmation. Local processing means your data stays on your Mac.

← Back to Blog

More from the blog