Two things happened the same week, and together they may say something real about where local AI is heading.
Unsloth announced it had squeezed Qwen3.8-27B down to 1-bit while "retaining ~77% accuracy"—a strong 27B dense model now fits in about 6GB. It spread fast, mostly framed as "77% of the intelligence, a fraction of the size."
Worth being careful with a number like that. Compression benchmarks and real-world capability aren't measured the same way, and headlines tend to collapse the two. A model that fits in 6GB isn't the same claim as a model that performs like the original—especially once the OS, the runtime and a long task's accumulated small errors are factored back in.
Google had already given some shape to this back in June: AI Edge Gallery on macOS, Gemma 4 12B running fully on-device, writing and executing Python inside a local sandbox to turn your own files into charts and analysis, nothing leaving the laptop. It also opened up a way to expose that same local model as a standard endpoint, so it plugs straight into the coding tools people already use. Eloquent, Google's offline dictation app, added Voice Edit: select text, say "turn this into an executive summary," and it happens on-device.
None of this makes the underlying progress fake. Dynamic quantization genuinely improved, and a local model executing sandboxed Python, or plugging into the same tools cloud models use, could open up a real architectural shift perspective.
An old dream, new hardware
Now a second signal, one step further out: DarkBloom, from Eigen Labs, turns idle Apple Silicon Macs into a paid inference network—already a provider on OpenRouter, billions of tokens served, with allegedly $120–200/month back to laptop owners. Telling, that last part: the same marketplace that routes requests to Anthropic, OpenAI and the rest now lists a fleet of idle Macs as just another provider on the same menu.
The dream of turning millions of idle personal machines into one shared resource is not new. I myself remember being fascinated as a kid by SETI@home—this idea of lending the family computer's idle time to help search the sky for something out there. There was a frugality to it that appealed to me even then: don't build a bigger machine, use the one already sitting idle in the next room. BitTorrent later turned every downloader into an uploader, and the entire crypto-mining economy has run on the same premise for over a decade: your hardware, someone else's job, a market in between. What's changed is the hardware itself. A laptop used to be a weak server. A recent Mac with 36–128GB of unified memory is something else—CPU, GPU and NPU sharing one memory pool, finally able to actually run something useful. So DarkBloom is running an old idea in the age of AI inference, leverage hardware finally capable of carrying it.
That said, this calls for a few caveats—starting with security. DarkBloom talks about a "hardware enclave" and layered attestation, but at least one independent review found the isolation is really software-level, not hardware. Not yet worth trusting with anything sensitive.
It's worth asking whether the same idea would make sense inside a company, where the trust and market problems mostly disappear—the enterprise already owns both the idle laptops and the workload that would otherwise go to the cloud. A quick sketch: 5,000 Apple Silicon laptops, a third recent enough to matter. Laptops don't stay on at the office overnight, but during a normal workday most of them run nowhere near full tilt anyway—reading, writing, a call, waiting on a human—so call it a conservative few hours a day of real spare capacity, spread across the working year. That's still something like 1.3 million machine-hours a year—the rough equivalent, going by DarkBloom's own public pricing, of $70,000–120,000 a month of computed value going unused! I'd be surprised if this actually happens at this stage in the enterprise, though: the workload people would actually want it for, real-time agentic work, needs low, predictable latency that a fleet of laptops going to sleep, changing networks and swapping models in and out of memory can't reliably offer, and Apple's own privacy positioning cuts hard against routing one employee's job through another employee's machine.
But even without replicating DarkBloom's model as such, the math is worth keeping around: it's a rough measure of how much GPU sits idle and dormant inside an enterprise fleet of laptops that's already been bought and paid for.
There's a good reason to think it's not just the GPU going to waste, either. A recent paper out of University of Texas at Austin found agentic workflows to be far more CPU-bound than the GPU-centric mental model most of us carry around. Every tool call, every orchestration decision, every handoff between planning and execution crosses back onto the CPU—so much so that, in their measurements, the same framework swung from 99% CPU / 44% GPU on research-style tasks to 31% CPU / 55% GPU on algorithmic ones. The CPU, not the GPU, often sits on the critical path. Worth remembering next time "send it to the cloud" quietly means "send it to a GPU": a good share of what agentic work actually burns is ordinary CPU cycles—the kind any decent laptop already has available, doing nothing.
Coming either way
Then again, even if we never quite get there, it might still be roughly the direction the enterprise is heading: more inference running locally by default, simply because the tools people work through are converging—the same coding harness talking to a local model one minute, a frontier one the next. Claude's Dispatch already sends a task from your phone to your own desktop, and runs it there. Codex's Handoff already moves a running session, and its state, between your laptop and a remote machine. Neither is DarkBloom's kind of shared compute—but both point at the same instinct: start the work wherever you are, let it continue wherever it needs to. The beginning of a common language between machines, more than a single protocol yet.
What may be taking shape, underneath all three, is a tiered one: frontier models at the top, reserved for the jobs that actually need that much reasoning, reached deliberately over the network. Underneath, more and more local models picking up everything that doesn't—extraction, classification, drafting, a Python script to chart your own data, voice edits, a short agent loop that doesn't warrant calling anything larger. The interesting question stops being "local or cloud" and becomes which layer a given piece of work belongs to by default, and how cheaply it can escalate when it doesn't.
Subsidiarity as an architecture principle?
* * *
PS: when I say on-device inference, I mean the whole family: e.g. MacBooks on Apple Silicon, NVIDIA's RTX Spark, and increasingly the Mac mini/Studio, which has quietly become a popular choice for running autonomous coding agents in an always-on, sandboxed box. Apple's own Mac Studio M5 Ultra update makes the point for me: more memory, and faster memory, explicitly positioned to run serious local models. It landed the same week as Xiaomi's "AI Cube" prototype, aimed at the same kind of workload. The "GPU on your desk" keeps getting more literal.
