In-Browser Models (ONNX)
Models that run directly inside the browser tab's own memory and your machine's GPU (via WebGPU/WASM) — no server, no Ollama, nothing installed.
What "In-Browser" Means
Unlike the Ollama-based engines, In-Browser LLM models never leave your browser tab. The model weights are downloaded once (and cached by the browser), then inference runs entirely client-side using Transformers.js — no Node.js server, no local Ollama process, no network calls after the initial download.
Where hardware acceleration (WebGPU) is available, the model runs on your graphics card directly from the tab; otherwise it falls back to WASM on the CPU. Either way, everything happens in the memory of that single browser tab — closing the tab clears the loaded model from memory (the downloaded weights themselves stay cached by the browser for next time).
This engine is a sibling to the In-Browser Models (WebGPU) engine, which runs on WebLLM and requires genuine WebGPU support with no CPU fallback, in exchange for faster inference on capable GPUs.
Where to configure it
Go to Settings → Agentic → Writer Engine and select In-Browser LLM (ONNX). Pick a model from the list below, click Download, and it becomes available for content generation once ready. See Agentic AI Settings for the full settings reference.
Underlying Technology
This engine is built on @huggingface/transformers (Transformers.js), Hugging Face's JavaScript library built on top of ONNX Runtime Web:
- Models are downloaded in ONNX format, from
onnx-community/...repositories. - Runtime: ONNX Runtime Web — the library automatically picks a WASM or WebGPU backend depending on what the browser supports.
See In-Browser Models (WebGPU) for the sibling engine, which uses a different runtime (WebLLM/MLC) and a different model format.
Micro / Ultra-Lightweight Models
Starter models that download in seconds and run without straining the browser tab — even on old machines with no dedicated GPU:
| Model | Size | Best for |
|---|---|---|
SmolLM2-135M | ~100 MB | Fastest. Short stories, simple drafts. |
SmolLM2-360M | ~200 MB | Fast text completion, balanced quality. |
Qwen2.5-0.5B | ~350 MB | Blog drafts, good instruction following. |
Balanced In-Browser Models
The best speed/reasoning balance for text generation, summarization, and chat directly in the browser:
| Model | Size | Best for |
|---|---|---|
TinyLlama-1.1B | ~630 MB | Best quality in this tier. Real narrative for stories & blogs. |
Llama-3.2-1B-Instruct | ~750 MB | Meta's flagship in-browser WebGPU model — the most widely used, with very stable chat behavior. |
SmolLM2-1.7B | ~1.0 GB | The smartest model in the Smol series; one of the most popular models optimized specifically for in-browser use. |
High Performance In-Browser Models
Push the browser's limits, but deliver the strongest in-browser results for language understanding, reasoning, and technical work like coding:
| Model | Size | Best for |
|---|---|---|
Qwen2.5-1.5B-Instruct | ~1.6 GB | A step up from the 0.5B model — clearly the best multilingual and reasoning quality among in-browser models. |
Qwen2.5-Coder-1.5B | ~1.6 GB | Optimized specifically for in-browser code completion, code analysis, and technical formulas. |
Note on sizes
Sizes shown are approximate download sizes for each model's ONNX build (q4 quantized) served via Transformers.js — actual browser cache usage can vary slightly by browser and quantization.
Choosing a Model by Category
Micro / Low-Resource
SmolLM2-135M, SmolLM2-360M, Qwen2.5-0.5B — best for old or GPU-less machines, or when you just need a fast first try.
Balanced In-Browser
TinyLlama-1.1B, Llama-3.2-1B-Instruct, SmolLM2-1.7B — the default recommendation for everyday in-browser writing and chat.
High Performance
Qwen2.5-1.5B-Instruct, Qwen2.5-Coder-1.5B — best language understanding, reasoning, and code quality, at the cost of a larger download and more memory.
Downloading a Model
The first download happens once per browser — after that, the model is served from the browser's own cache, so switching tabs or reloading the page does not require re-downloading. Switching to a different model triggers a new download for that model only.
FAQ
Llama-3.2-1B-Instruct is the most widely used and stable in-browser chat model for most users. If your machine is very low-powered, start with SmolLM2-135M instead.