On October 7, Microsoft spent a Windows and Surface event in San Francisco arguing that local AI is the next chapter of the PC. Satya Nadella and Jensen Huang shared the stage. The headline hardware was the Surface Laptop Ultra, starting at $2,599 and shipping October 16, built around NVIDIA’s RTX Spark chip. NVIDIA says that chip can run models up to 120 billion parameters on the machine. Microsoft also confirmed that llama.cpp is coming to Windows ML.
That is a real shift. Running serious models on a laptop used to be a hobbyist project. Now it ships in a retail box.
But look at how the coverage describes the design. Copilot can act on files stored on the PC, with the owner’s permission. Work is split between the device and the cloud, which Microsoft and NVIDIA both called hybrid. Nobody said your files never leave the machine. The word “local” is doing a lot of work in the marketing, and founders should notice what it covers and what it doesn’t.
Local compute is not local data
These are two different properties, and people blur them.
Local compute means a model runs on the user’s hardware. Local-first means the user’s data lives on their device as the primary copy, the app works without a network, and any server is optional. You can have the first without the second. A desktop app can run inference on-device and still sync every document, prompt, and embedding to your servers. You can also have the second without the first. A notes app with no AI at all can be fully local-first.
The hybrid model Microsoft describes is a reasonable engineering choice. Small tasks run locally for speed and cost. Big ones go to the cloud. But the boundary between those two is a product decision, and users rarely see where it sits. If your pitch to customers is “it runs on your device,” you now owe them a straight answer to a harder question: what leaves the device, when, and who decides?
We’d rather you could answer that in one sentence. If it takes a diagram and a legal review, it’s not a privacy feature. It’s a routing policy.
What changes when an agent can read the disk
The part of the event that interests us most is not the chip. It’s the agent that acts on local files.
A cloud assistant sees what you paste into it. A local agent with file access sees what’s on the disk. That includes the contract you forgot was in Downloads, the export from your accounting package, and the screenshot with a password in it. Microsoft says its goal is to make Windows the most secure platform for AI agents, and that’s the right thing to be saying. It’s also an admission that the risk is new.
We’ve written before about how approval prompts in AI coding tools get bypassed, and how agents delete things they were never meant to touch. A local agent inherits the same problems with a bigger blast radius, because the files are the user’s real files, not a sandbox. Permission is also a weak control when it’s granted once and forgotten. “Allow access to your documents” is a very wide door.
If you’re building a product that uses an agent against user data, our advice is plain:
- Scope access per task, not per install.
- Log every read and every outbound call in a place the user can inspect.
- Make the default for anything that leaves the device be “no.”
- Keep the user’s data in a format they can open without your app.
That last one is the ownership test. If your local data sits in a proprietary store only your software can read, “local” is just a cache with extra steps.
Where this is the wrong fit
We like on-device models, and we think they’re underused. For classification, transcription, search over a user’s own records, and drafting from private notes, a small local model is often good enough. It’s cheaper to run, works on a plane, and removes a vendor from your data path.
It’s the wrong fit when the task needs a frontier model and you’re pretending otherwise. A 120 billion parameter model on a $2,599 laptop is impressive. It’s also a $2,599 laptop. Most of your customers are on a five-year-old machine, a phone, or a locked-down corporate image. If your product only works well on top-end silicon, you’ve narrowed your market to people who just bought a new Surface. Plan for a fallback, and be honest in your docs about what the fallback does with the data.
It’s also the wrong fit if you haven’t decided who updates the model. Local models drift out of date. Somebody has to ship new weights, test them against your prompts, and handle the user whose old laptop can’t run the new version. That work doesn’t disappear when you move off the cloud. It moves to you.
What we’d do with this
If you’re a founder deciding how to add AI to a product, start from the data, not the model. Write down every category of user data the feature touches. Mark which ones can stay on the device by design. Then pick the model size and the architecture to fit that line, instead of picking a model and discovering the line later.
For a lot of products that lands on a local-first core: the user’s records stored on their device, the AI features running against that store, and an optional sync you can switch off. That’s the thinking behind our local-first software work, and Microsoft’s event is a decent sign that the hardware is catching up to it.
The risk to watch is vocabulary. “Local AI” will end up on a thousand product pages this quarter, and most of them will mean “a small model runs here and your data still goes there.” Customers will figure that out. The products that hold up will be the ones that said exactly where the line was from the start.
If you’re weighing this for a product, get in touch.