In-Browser Models (WebGPU)
Models that run directly inside the browser tab using WebLLM and your machine's GPU via WebGPU — no server, no Ollama, nothing installed. Unlike the ONNX engine, there is no CPU fallback: WebGPU is required.
Overview
Unlike the Ollama-based engines, In-Browser LLM (WebGPU) models never leave your browser tab. The model weights are downloaded once (and cached by the browser), then inference runs entirely client-side using WebLLM (MLC) — no Node.js server, no local Ollama process, no network calls after the initial download.
This engine is a sibling to the In-Browser Models (ONNX) engine, which runs on Transformers.js and falls back to WASM on the CPU when WebGPU isn't available. WebLLM takes the opposite trade-off: it requires genuine WebGPU support and has no WASM fallback, in exchange for faster inference on capable GPUs using MLC's compiled model format.
Where to configure it
Go to Settings → AI Models → Writer Engine and select In-Browser LLM (WebGPU). Pick a model from the list below, click Download, and it becomes available for content generation once ready. See Agentic AI Settings for the full settings reference.
Screen preview
A quick look at the screens. Use the toggle in the corner of each preview to switch between the dark and light themes.
Micro — old or GPU-less machines
Balanced — everyday in-browser chat
High performance — best code quality
Balanced — fast replies on integrated GPUs
Downloads once, then stays in browser storage
Automatic when WebGPU is unavailable
Keeps working with no network once cached
In-browser models with WebGPU
Underlying Technology
This engine is built on WebLLM (mlc-ai/web-llm), the browser port of MLC-LLM:
- Models are distributed in MLC format — MLC's own quantized format, typically
q4f16— unlike the ONNX format used by the sibling engine. - Runtime: a dedicated TVM/WebGPU runtime — it uses WebGPU differently from Transformers.js, via its own compilation pipeline.
Because the format and quantization differ, the same underlying model is often a different download size than its ONNX counterpart on the In-Browser Models (ONNX) page.
Micro / Ultra-Lightweight Models
Starter models that download quickly and run without straining the browser tab:
| Model | Size | Best for |
|---|---|---|
SmolLM2-360M | ~230 MB | Fastest WebGPU model. Short drafts. |
Balanced In-Browser Models
The best speed/reasoning balance for text generation, summarization, and chat directly in the browser:
| Model | Size | Best for |
|---|---|---|
Llama-3.2-1B-Instruct | ~880 MB | Meta's flagship small model, GPU-accelerated. |
High Performance In-Browser Models
Push the browser's limits, but deliver the strongest in-browser results for language understanding, reasoning, and technical work:
| Model | Size | Best for |
|---|---|---|
Qwen2.5-1.5B-Instruct | ~1.1 GB | Best multilingual/reasoning quality, WebGPU-accelerated. |
Phi-3.5-mini-instruct | ~2.2 GB | Microsoft's high-performance small model. |
Note on sizes
Sizes shown are approximate download sizes for each model's MLC-quantized build (q4f16_1) served via WebLLM — actual browser cache usage can vary slightly by browser.
Choosing a Model by Category
Micro / Low-Resource
SmolLM2-360M — fastest download and inference, best first try on any WebGPU-capable machine.
Balanced In-Browser
Llama-3.2-1B-Instruct — the default recommendation for everyday in-browser writing and chat on WebGPU.
High Performance
Qwen2.5-1.5B-Instruct, Phi-3.5-mini-instruct — best language understanding and reasoning quality, at the cost of a larger download and more GPU memory.
Downloading a Model
The first download happens once per browser — after that, the model is served from the browser's own cache, so switching tabs or reloading the page does not require re-downloading. Switching to a different model triggers a new download for that model only.
FAQ
Llama-3.2-1B-Instruct is the recommended default for most users with a WebGPU-capable browser. If you want the fastest possible download and response time, start with SmolLM2-360M instead.