Most writing about AI data centers is about gigawatts, chips and the companies spending hundreds of billions of dollars on them. I'm on the other end of that supply chain. I run a portfolio of sites as a solo operator, I rent model capacity by the call, and I own a small amount of hardware. When a hyperscaler can't get a grid connection, I don't read about it in an earnings call. I find out through a rate limit.
So this is the view from the bottom of the stack: what I rent, what I run, why I don't rent GPUs, and what the power crunch is likely to mean for operators my size.
What do I rent?
Model calls. My AI calls go through one gateway, which routes them to cloud model APIs. Among the callers is a fleet of nine agentic coding hosts that do routine engineering work across my sites. That work wants the strongest models available, and those are hosted.
The gateway is the part I'd keep if I had to give up everything else. It means a model or provider change happens in one place, every caller gets the same logging and limits, and moving away from a provider having a bad day is a configuration change rather than a code change in every script.
What do I run?
Less than people assume. One Linux server hosts my sites, deploy pipelines, cron jobs and the AI gateway itself. My daily machine is a MacBook Pro with an M4 Max and 64 GB of memory, which already runs speech-to-text locally: a push-to-talk dictation agent on parakeet-mlx that I can reach from my phone over Tailscale, and mlx-whisper for longer transcriptions.
The next addition is one local box. I'm planning to buy a Mac Studio M5 Ultra for batch transcription, video tagging, local coding models and image generation. The reasoning, the comparison with DGX Spark and the alternatives, and the benchmarks I'm relying on are in why I'm buying a Mac Studio M5 Ultra for local AI.
Why am I not renting GPUs from a neocloud?
Neoclouds such as CoreWeave, Lambda, RunPod and Vast.ai rent GPU capacity by the hour, and ctaio.dev has a comparison of the main ones. They make sense for teams training models or serving one model at steady, high volume. I'm neither, and three things keep me on APIs:
- A GPU hour comes with a stack. Renting the GPU means I also run the inference server, pick the model weights, handle scaling and keep it patched. A model API includes all of that in the price per call.
- My load is bursty. Agents run hard for a stretch and then stop. Per-call pricing charges for what I use. A rented GPU charges for the hours it's reserved, used or not.
- I want the best models, and they aren't downloadable. The frontier models my coding agents depend on are only available as hosted APIs. A rented GPU can only run open-weight models, which is exactly the set the local box will cover.
It works like training: you don't hire a full-time coach to fix one stroke flaw, and you don't buy a pool to swim twice a week. You pay per session until the volume justifies owning it. For hosted frontier models I'm nowhere near that line. For open-weight models running every day on batch work, I think I've crossed it, and the answer there is a box on a desk rather than a rented GPU.
What does the data center power crunch mean for small operators?
The constraint on new AI capacity has moved from chips to electricity. The AI infrastructure overview on ctaio.dev puts it as "power is the binding constraint," and cites IEA figures: data centers used about 485 TWh in 2025, on track for about 950 TWh by 2030. ERCOT alone has interconnection requests for 160 GW of data center load, many of them speculative. The bottleneck is grid connections and electrical equipment. Some hyperscalers are signing nuclear deals for firm power, but the review of microreactors for AI data centers is blunt about timing: "Nothing in this list will power a production AI cluster you are planning for 2027."
What I expect that to mean for someone my size is a forecast, not a measurement. Capacity that can't be built quickly gets allocated, and allocation favours whoever signs the biggest contract. I'd expect small customers to feel it first in availability rather than in list prices: tighter rate limits at peak times, new models reaching large accounts before self-serve tiers, and the occasional provider that is simply overloaded for an afternoon. Prices per token have fallen for a long time, and I don't assume that trend reverses. I'm less confident it keeps going for the newest, largest models, which need the scarcest capacity.
How am I hedging?
- One gateway, several providers. If one provider limits or slows me, pointing the traffic at another is a change in one place. This is the cheapest hedge I have and it's already running.
- Move work that can wait to my own hardware. Batch transcription and video tagging don't need an answer in two seconds. They can sit in a queue on a local box and run overnight, outside anyone's capacity planning.
- Keep the frontier work on hosted APIs. The agentic coding work stays in the cloud, because the gap between the best hosted models and what I can run locally is still large for that job.
The CTO-level version of this, covering build models, power, cooling and how to read a provider's capacity claims, is the AI infrastructure guide on ctaio.dev. For a company deciding whether to bring AI in-house, aistrategy.guide covers private AI from the executive side.
Sources
- ctaio.dev, AI infrastructure, citing the IEA's Key Questions on Energy and AI (April 2026) for electricity and interconnection figures.
- ctaio.dev, Nuclear microreactors for AI data centers.
Frequently Asked Questions
Should a small company rent GPUs from a neocloud or call model APIs?
For most small operators, model APIs. Renting a GPU means running the model, the inference server and the scaling yourself, and paying for the hours it sits idle. That makes sense when you have steady, heavy load or a model you must host yourself. Bursty usage across many tasks is what per-call API pricing was built for.
How does the AI data center power shortage affect API users?
Power and grid connections are now the binding constraint on new AI capacity, according to the IEA figures summarised on ctaio.dev. My expectation, and it is an expectation rather than a measured fact, is that small API customers feel it as rate limits, capacity priority for large contracts and slower access to new models, before they feel it in list prices.
Why route all model calls through one gateway?
A single gateway lets me switch providers or models in one place, apply the same logging and limits to every caller, and fall back to another provider when one is overloaded. When capacity is uneven across providers, that flexibility is the cheapest hedge a small operator has.
When does local AI hardware make sense for a solo operator?
When you have workloads that run every day, can tolerate a queue, and involve material you would rather keep on your own hardware. Batch transcription and video tagging are my examples. Frontier-model work such as agentic coding still belongs on hosted APIs.
Need Expert Technology Guidance?
20+ years leading technology transformations. Get a technology executive's perspective on your biggest challenges.
