In-Browser Models (WebGPU) - Agentic

What "In-Browser (WebGPU)" Means

Unlike the Ollama-based engines, In-Browser LLM (WebGPU) models never leave your browser tab. The model weights are downloaded once (and cached by the browser), then inference runs entirely client-side using WebLLM (MLC) — no Node.js server, no local Ollama process, no network calls after the initial download.

This engine is a sibling to the In-Browser Models (ONNX) engine, which runs on Transformers.js and falls back to WASM on the CPU when WebGPU isn't available. WebLLM takes the opposite trade-off: it requires genuine WebGPU support and has no WASM fallback, in exchange for faster inference on capable GPUs using MLC's compiled model format.

Where to configure it

Go to Settings → Agentic → Writer Engine and select In-Browser LLM (WebGPU). Pick a model from the list below, click Download, and it becomes available for content generation once ready. See Agentic AI Settings for the full settings reference.

Underlying Technology

This engine is built on WebLLM (mlc-ai/web-llm), the browser port of MLC-LLM:

Models are distributed in MLC format — MLC's own quantized format, typically q4f16 — unlike the ONNX format used by the sibling engine.
Runtime: a dedicated TVM/WebGPU runtime — it uses WebGPU differently from Transformers.js, via its own compilation pipeline.

Because the format and quantization differ, the same underlying model is often a different download size than its ONNX counterpart on the In-Browser Models (ONNX) page.

Micro / Ultra-Lightweight Models

Starter models that download quickly and run without straining the browser tab:

Model	Size	Best for
`SmolLM2-360M`	~230 MB	Fastest WebGPU model. Short drafts.

Balanced In-Browser Models

The best speed/reasoning balance for text generation, summarization, and chat directly in the browser:

Model	Size	Best for
`Llama-3.2-1B-Instruct`	~880 MB	Meta's flagship small model, GPU-accelerated.

High Performance In-Browser Models

Push the browser's limits, but deliver the strongest in-browser results for language understanding, reasoning, and technical work:

Model	Size	Best for
`Qwen2.5-1.5B-Instruct`	~1.1 GB	Best multilingual/reasoning quality, WebGPU-accelerated.
`Phi-3.5-mini-instruct`	~2.2 GB	Microsoft's high-performance small model.

Note on sizes

Sizes shown are approximate download sizes for each model's MLC-quantized build (q4f16_1) served via WebLLM — actual browser cache usage can vary slightly by browser.

Choosing a Model by Category

Micro / Low-Resource

SmolLM2-360M — fastest download and inference, best first try on any WebGPU-capable machine.

Balanced In-Browser

Llama-3.2-1B-Instruct — the default recommendation for everyday in-browser writing and chat on WebGPU.

High Performance

Qwen2.5-1.5B-Instruct, Phi-3.5-mini-instruct — best language understanding and reasoning quality, at the cost of a larger download and more GPU memory.

Downloading a Model

1. Settings → Agentic

→

2. Writer Engine → In-Browser LLM (WebGPU)

→

3. Pick a model

→

4. Download & use

The first download happens once per browser — after that, the model is served from the browser's own cache, so switching tabs or reloading the page does not require re-downloading. Switching to a different model triggers a new download for that model only.

FAQ

How is this different from In-Browser Models (ONNX)?

Both run entirely inside the browser tab with no server involved. The ONNX engine (Transformers.js) works everywhere — it uses WebGPU when available and falls back to WASM on the CPU otherwise. The WebGPU engine (WebLLM) requires genuine WebGPU support with no fallback, but is typically faster on a capable GPU since it uses MLC's compiled model format. See In-Browser Models (ONNX) for the CPU-friendly option.

What happens if my browser doesn't support WebGPU?

Settings will show a warning that WebGPU is unavailable and the Download button is disabled. Use the In-Browser Models (ONNX) engine instead, which works on any browser via its WASM fallback.

Does my data ever leave my computer?

No. Once the model weights are downloaded, generation happens entirely inside the browser tab — no network requests are made for inference.

Which model should I start with?

Llama-3.2-1B-Instruct is the recommended default for most users with a WebGPU-capable browser. If you want the fastest possible download and response time, start with SmolLM2-360M instead.

Is this the same as the Local Language Models (Ollama) catalog?

No. See Local Language Models for the Ollama-based catalog, which requires the separate Ollama application to be installed and running.