I spent a few spare hours over the last few days on something small: forking a browser-agent project, tearing out its cloud brain, and transplanting in a lightweight LLM that runs entirely offline from the browser on the GPU of the machine it's installed on.
No server, no API key, no data leaving the browser—that was the intent, and how far it actually held is most of what follows. The occasion to think, hands-on, about a surface I'd only been reading about—AI at the edge, specifically the edge that is a browser tab.
The starting point wasn't abstract. In day-to-day conversations with business teams at Michelin, the same need keeps coming up: navigating and interpreting data that lives inside internal interfaces—BI tools, dashboards of every kind, some of them genuinely sensitive. It's a real point of cognitive friction, because turning these systems into APIs or MCP servers isn't always possible—let alone legacy systems—while the screen remains the most natural interface for most of what people actually need to do—cross-reference a few figures, dig one level deeper, compare two views side by side, etc.
The last few months have seen a wave of browser extensions built around exactly that gap. Claude for Chrome just merged its side panel into a full Cowork session, carrying a browser tab into the same agentic workbench you'd otherwise reach from the desktop. But some of those interfaces are sensitive enough that you don't want their contents leaving the building, in any form, to any cloud—which is where local, in-browser inference gets interesting rather than merely clever. This isn't a niche direction. Hugging Face's Clem Delangue posted this month that Transformers.js, their library for running models directly in the browser, just crossed 10 million monthly downloads—roughly 10x where it stood six months earlier.
I should say upfront how this actually got prototyped, because it matters for what follows: I'm not a developer, and I didn't read a line of code myself. What I did was the part I know how to do—define the question, then spend a full day having each of a dozen existing projects explained to me, approach by approach, and judging each one as reusable, worth forking, worth adapting, or worth ignoring.
Twelve names came out of that day, and most of them earned a specific role rather than a verdict. Nanobrowser turned out to have the best "body"—the part that reads a page the way a person would and clicks, types, and scrolls on it—so that's what got forked, DOM parsing and action registry included. WebLLM, MLC's runtime, became the engine, reused close to as-is: the piece that takes a model and runs it on the machine's own GPU through WebGPU, no server anywhere in the loop. It won for a specific reason, not just because it worked: it can constrain a model's output to valid JSON by construction, which removes an entire category of failure—the malformed action—that trips up small local models more than anything else.
Page Assist stayed a UX reference rather than code worth borrowing. Browser-use's architecture wasn't portable as-is—it's built for Python, not a Chrome extension—but its way of compressing a page into something a model can act on was worth stealing conceptually. One of the twelve, Runanywhere, had attempted almost exactly this ambition and failed in a useful way: its page-reading logic, as it was explained to me, recognized only nine kinds of elements and capped itself at thirty per page, and once that wasn't enough to act reliably, the project retreated to scripts hand-written for specific sites—Amazon, YouTube—instead of a general agent—a warning about the wall I was about to run into myself, which I did, on schedule :-)
Then I handed that audit to Opus, with explicit instructions not to invent anything the audit hadn't already found, and to treat the whole thing as integration work rather than a green-field build. What got assembled: Nanobrowser's body, with its reasoning swapped for WebLLM's—browser, WebGPU, the machine's own GPU, nothing else in the loop. Out went the cloud connectors, the API key screen, the usage telemetry—about 3,000 lines removed, leaving the extension amputated rather than merely disabled: for a stretch, it was physically unable to reach any AI at all, so nothing could leak mid-transition. Then in went a local model, downloaded once, running on-device from then on.
The browser turned out to be quite a credible place to run this. Nothing to prepare on the machine: whatever laptop someone happens to have, the same extension install just runs—no drivers, no per-device setup, no admin rights. Universal to deploy, not uniform to experience: how much horsepower sits under the keyboard stops being invisible and starts deciding what the thing can actually do for you. And privacy stops being something you take on a vendor's word—it's something the browser enforces on its own, structurally.
Asked to summarize a page it already had in front of it, the model stalled—asked for the content again, repeated its own intention twice, never acted. The cause, once measured rather than guessed at, was almost stupid: the briefing prepared for it ran to 7,233 tokens against a 4,096 window—like handing someone forty pages of documentation to ask them the time, then concluding they can't read. The same overflow explained a second, unrelated-looking symptom—the machine's interface freezing, on and off, not from memory pressure but from a GPU busy digesting a document it was never going to finish. One cause, two symptoms. Not a model that was too small. Input that was too large and too messy.
That's the finding I keep coming back to: the failure had nothing to do with the model's size, and everything to do with what surrounded it—the "harness," as I'd call it elsewhere. Everything built around a model—how you describe a page to it, how you bound its actions, how you carry memory of what it's already tried—survives that model's replacement. Small models keep improving fast. What doesn't work today has real odds of working in a year, on the same harness, without touching that layer again—provided the harness work gets done now rather than deferred.
I'd locked the front door—the AI can't reach any server. An independent audit found the back one still open: a manipulated page could still make the browser navigate to a URL with data appended, leaking it without a single blocked request ever firing. The inherited allowlist meant to prevent exactly that was "on" by default with an empty list—which, read literally, means allow everything. Two leak paths, and I'd only locked one. And I wasn't the first to find that door: five months earlier, in March, the extension most people treat as this field's reference point had shipped a flaw in the same family—any website could inject instructions into the assistant as if the user had typed them. Different direction of travel—instructions coming in rather than data going out—but the same underlying lesson: with an agent, the weak point isn't the inference engine, it's the surface it's allowed to act on.
That kind of hole isn't specific to one weekend project, and neither is the moment that makes any of this worth building in the first place. None of the individual pieces here are new, and that's the point: two catch-ups happened at once. On the AI side, models got smaller through quantization and distillation without losing much capability, and the runtimes to serve them in a browser tab—Hugging Face's Transformers.js, WebLLM, WebGPU itself—matured fast enough that local inference stopped being just a research demo. On the hardware side, the machines people already carry got a lot more capable of running that locally, Apple Silicon MacBooks especially, with the next generation of consumer GPUs from Nvidia and others pushing further in the same direction.
The "ask questions about this page, locally" mode we assembled already exists, published, more than once. What's rarer is anyone publishing the actual numbers instead of the promise—what a local agent costs, where it stalls, why.
I don't think the autonomous version—the offline AI that investigates a dashboard on its own, the way I originally wanted—is proven yet. But what I think is settled, for me, is smaller and more useful: the browser is a credible place to run private AI in an enterprise, right now, and the nearest real value is in assisted reading of sensitive data, not autonomous investigation. That's not a consolation prize. It's deployable without touching a single workstation, against a need that already exists.
* * *
Finally, one thing I'll add on the way out—because it changes how I read everything above:
None of this was me writing code—and I'm quite unable to write any serious production code. It was me doing the part I actually know how to do: asking the right question, having a dozen strangers' repositories explained to me closely enough to judge which parts deserved to survive, then handing that judgment to Opus and holding it to it—less an operation I performed than one I directed remotely, an army of surgeons on the other end of a robotic arm, executing exactly what got specified and nothing else. A few years ago, that whole chain would have stopped at the first step for someone like me. The gap between having the right question and having a working prototype has gotten a lot shorter than it used to be—and that's arguably the more interesting story here.
