Why I'm Buying a Mac Studio M5 Ultra for Local AI

Thomas Prommer Technology Executive & CTO Connect on LinkedIn

Published: October 20, 2026

Updated: October 20, 2026

I'm planning to buy a Mac Studio M5 Ultra as a dedicated local AI box. I haven't ordered it yet, and this page is the reasoning before the purchase, written down so I can check it against what actually happens. The short version: I already run some AI work locally on my laptop, I send the rest to cloud model APIs through a gateway, and there is a set of batch and private workloads that fit neither place well. A machine with a lot of fast memory sitting on a desk, always on, is the gap.

Endurance athletes learn this about equipment early. You buy for the sessions you already do every week, not for the race you might enter. A power meter is worth it once you're riding enough structured sessions to use the numbers. So the test I'm applying here is the same one: which workloads do I already have, today, that this box would run better?

What do I run locally today?

My daily machine is a MacBook Pro with an Apple M4 Max and 64 GB of unified memory. On it:

• Dictation. A push-to-talk agent runs speech-to-text locally with parakeet-mlx on the mlx-community/parakeet-tdt-0.6b-v3 model. It is reachable from my phone over Tailscale, so I can dictate into the laptop from anywhere on my own network without the audio going to a transcription service. How much of my terminal input is now voice is its own setup page.
• Transcription. mlx-whisper handles longer recordings on the same machine.
• Ollama, installed through Homebrew.

Speech is the workload that made local inference real for me. The models are small, Apple Silicon runs them quickly through MLX, and there's a privacy argument I don't have to think hard about: my voice memos don't need to leave the house.

What still goes to cloud APIs, and why?

Most of my AI usage still runs in the cloud. I run a Linux server that hosts my sites, deploy pipelines and cron jobs, and it also runs an AI gateway that routes model calls to cloud APIs. Behind that sit nine agentic coding hosts that do much of the routine engineering work across my sites. I wrote about what that week looks like in an executive's week on an AI agent fleet.

Those calls stay in the cloud for a plain reason: the agentic coding work wants the strongest models available, and the strongest models are hosted ones. A gateway also means I change providers in one place rather than in every script. None of that changes with a Mac Studio. The local box is an addition for a different class of work, and I'm not planning to replace the gateway with it.

The laptop is the wrong place for the rest because it's the machine I work on. A long batch job competes with everything else I'm doing, it stops when the lid closes, and 64 GB caps the model size once the operating system and my own apps have taken their share.

Which workloads would move to a Mac Studio?

These are plans, not things I run today. Each one is a workload I already have in some form that would benefit from a box that's always on and has a lot of fast memory.

1. Batch transcription. Voice memos and podcast audio, transcribed overnight in a queue rather than one file at a time on the laptop.
2. Video ingest and tagging. Running a vision-language model over footage to describe and tag it, so I can find a clip by what's in it.
3. Local coding and agent models. Open-weight models for the agent tasks that don't need a frontier model, and for the ones that touch material I'd rather keep on my own hardware.
4. Image generation for site heroes. The hero images on this site come from a hosted FLUX model today. Generating them locally would let me iterate on a style without paying per image.

Is there an M5 Mac Pro?

No. A lot of people search for one, and the answer is that Apple discontinued the Mac Pro in March 2026 and there is no M5 version. The Mac Studio with the M5 Ultra is the top of Apple's desktop line for this kind of work. If you were waiting for a Mac Pro to get PCIe slots for GPUs, that path is closed on the Mac side, and the RTX PRO 6000 row in the table below is the alternative.

How does the M5 Ultra compare with DGX Spark, Strix Halo and RTX PRO 6000?

For local inference, two numbers decide most of it: how much memory the model can sit in, and how fast the chip can read that memory. Price is the third. The figures below are list prices and specs as of October 20, 2026, from Apple's US store and from the local AI hardware comparison on ctaio.dev, which has the full sourcing and caveats.

The pattern is clear enough. Strix Halo and the DGX Spark get you 128 GB, but at roughly a fifth of the M5 Ultra's bandwidth, and token generation speed follows bandwidth. The RTX PRO 6000 is faster still per card, but 96 GB is the ceiling per card, and it brings a tower, a power supply and a fan curve with it. For an always-on box in a home office running mid-size models, the Mac Studio is the balanced choice. If my workloads were CUDA-specific I'd weigh it differently. For a broader version of this comparison, written for engineering teams rather than one person, see local AI for engineering teams.

What do the first M5 Ultra benchmarks show?

I haven't run any of these myself. The most complete public test I've seen is NetworkChuck's video "i got one....and it's FAST!!!". He tested an Apple-loaned M5 Ultra with the 80-core GPU, 256 GB of memory and 8 TB of storage, a configuration he puts at roughly $14,000 to $15,000, against an M3 Ultra with an 80-core GPU and 512 GB. All of the numbers below are from that video.

The video also reports Whisper tiny at 12 seconds on the same recording, and shows two demos I found more interesting than the charts: LM Studio running Qwen3.8 27B and connected over MCP to DaVinci Resolve, detecting cuts and making edits, and Hermes Agent running with a 131K-token context.

Why did prompt processing get 4x faster but generation only 1.5x?

This is the part worth understanding before buying anything, because it tells you which of your workloads will feel faster. Running an LLM has two phases. In prompt processing, the model reads your entire input at once. That is a large batch of matrix math, and it's limited by compute. The M5 adds Neural Accelerators to every GPU core, which is a big jump in exactly that kind of compute, and the time to first token fell by about 4x.

Generation is different. Each new token needs the model's weights read from memory again, one token at a time, so the limit is memory bandwidth, not compute. Bandwidth went from 819 GB/s on the M3 Ultra to 1.2 TB/s on the M5 Ultra, an increase of roughly half, and generation sped up by roughly 1.5x. The two speedups follow the two hardware changes almost exactly.

For my plans, that split is good news. Transcribing a two-hour podcast, tagging a video, or handing an agent a long codebase are all workloads where the input is huge and the output is comparatively short. Those are prefill-heavy, and prefill is where the M5 Ultra improved most. A chat where I type a line and read a long answer would gain less.

Which configuration am I leaning toward?

Memory is the decision that can't be undone, since unified memory can't be upgraded later. So I'm working backwards from the models. Most of what I plan to run sits in the 14B to 32B range: the Qwen models in the benchmark, a vision-language model for video, Whisper, and an image model. The rule of thumb on the ctaio.dev hardware page is about 0.5 to 0.6 GB per billion parameters at 4-bit, so a 27B model needs something like 14 to 16 GB before context. The memory goes to context and to running several of these at once, not to one big model.

That points me at three choices:

• The 80-core GPU. Prefill is compute-bound and my workloads are prefill-heavy, so the GPU cores are where the money buys speed I would notice.
• More than the 96 GB base, probably around 256 GB. That's the configuration NetworkChuck tested. It leaves room for a 27B to 32B model with a long context, Whisper and an image model loaded together, which a laptop with 64 GB can't do.
• Modest internal storage. Model files are large but replaceable, and I'd rather put bulk storage outside the box than pay for it inside.

I'm not planning to wait for 512 GB. It has no price yet, and it would mostly buy headroom for models far larger than the ones I've listed. If I find myself wanting a 400B-class model locally, that would be a different purchase for a different job.

What would change my mind?

• Independent benchmarks that disagree. One reviewer, one Apple-loaned unit and one configuration is a start. If wider testing shows much smaller gains on long-context work, the case gets weaker.
• The local workloads not materialising. If, after a month of honest accounting, the batch transcription and video tagging stay small enough for the laptop, a several-thousand-dollar box sitting idle is the wrong answer.
• Hosted models getting cheap enough for the batch work. The case for local hardware partly rests on per-call API costs for work that runs every day. If those fall far enough, the maths changes.
• Image and video generation becoming the main job. Then a PC with an RTX PRO 6000 and its 1,792 GB/s is the better tool, power draw and all.
• A need for CUDA. If a tool I depend on only runs on CUDA, a DGX Spark earns its place even with less bandwidth.

The executive version of this question, whether a company should run private AI on its own hardware at all, is on aistrategy.guide under private AI. For choosing models and runtimes once the box exists, ctaio.dev has the best local LLMs and Ollama vs LM Studio vs llama.cpp vs vLLM. The companion essay on what I rent versus what I run is an operator's view of AI compute. Why I run all of this on free tools rather than a paid local AI platform is in my local AI setup.

Sources

• Apple, Mac Studio specifications and US Apple Store pricing, checked October 20, 2026.
• ctaio.dev, Local AI hardware, prices as of October 20, 2026, for DGX Spark, Strix Halo and RTX PRO 6000 figures and the memory rule of thumb.
• NetworkChuck, "i got one....and it's FAST!!!", YouTube, for all M5 Ultra vs M3 Ultra benchmark figures.