In-Browser Models (ONNX) - Agentic

Overview

Unlike the Ollama-based engines, In-Browser LLM models never leave your browser tab. The model weights are downloaded once (and cached by the browser), then inference runs entirely client-side using Transformers.js — no Node.js server, no local Ollama process, no network calls after the initial download.

Where hardware acceleration (WebGPU) is available, the model runs on your graphics card directly from the tab; otherwise it falls back to WASM on the CPU. Either way, everything happens in the memory of that single browser tab — closing the tab clears the loaded model from memory (the downloaded weights themselves stay cached by the browser for next time).

This engine is a sibling to the In-Browser Models (WebGPU) engine, which runs on WebLLM and requires genuine WebGPU support with no CPU fallback, in exchange for faster inference on capable GPUs.

Where to configure it

Go to Settings → AI Models → Writer Engine and select In-Browser LLM (ONNX). Pick a model from the list below, click Download, and it becomes available for content generation once ready. See Agentic AI Settings for the full settings reference.

Underlying Technology

This engine is built on @huggingface/transformers (Transformers.js), Hugging Face's JavaScript library built on top of ONNX Runtime Web:

Models are downloaded in ONNX format, from onnx-community/... repositories.
Runtime: ONNX Runtime Web — the library automatically picks a WASM or WebGPU backend depending on what the browser supports.

See In-Browser Models (WebGPU) for the sibling engine, which uses a different runtime (WebLLM/MLC) and a different model format.

Micro / Ultra-Lightweight Models

Starter models that download in seconds and run without straining the browser tab — even on old machines with no dedicated GPU:

Model	Size	Best for
`SmolLM2-135M`	~100 MB	Fastest. Short stories, simple drafts.
`SmolLM2-360M`	~200 MB	Fast text completion, balanced quality.
`Qwen2.5-0.5B`	~350 MB	Blog drafts, good instruction following.

Balanced In-Browser Models

The best speed/reasoning balance for text generation, summarization, and chat directly in the browser:

Model	Size	Best for
`TinyLlama-1.1B`	~630 MB	Best quality in this tier. Real narrative for stories & blogs.
`Llama-3.2-1B-Instruct`	~750 MB	Meta's flagship in-browser WebGPU model — the most widely used, with very stable chat behavior.
`SmolLM2-1.7B`	~1.0 GB	The smartest model in the Smol series; one of the most popular models optimized specifically for in-browser use.

High Performance In-Browser Models

Push the browser's limits, but deliver the strongest in-browser results for language understanding, reasoning, and technical work like coding:

Model	Size	Best for
`Qwen2.5-1.5B-Instruct`	~1.6 GB	A step up from the 0.5B model — clearly the best multilingual and reasoning quality among in-browser models.
`Qwen2.5-Coder-1.5B`	~1.6 GB	Optimized specifically for in-browser code completion, code analysis, and technical formulas.

Note on sizes

Sizes shown are approximate download sizes for each model's ONNX build (q4 quantized) served via Transformers.js — actual browser cache usage can vary slightly by browser and quantization.

Choosing a Model by Category

Micro / Low-Resource

SmolLM2-135M, SmolLM2-360M, Qwen2.5-0.5B — best for old or GPU-less machines, or when you just need a fast first try.

Balanced In-Browser

TinyLlama-1.1B, Llama-3.2-1B-Instruct, SmolLM2-1.7B — the default recommendation for everyday in-browser writing and chat.

High Performance

Qwen2.5-1.5B-Instruct, Qwen2.5-Coder-1.5B — best language understanding, reasoning, and code quality, at the cost of a larger download and more memory.

Downloading a Model

1. Settings → Agentic

→

2. Writer Engine → In-Browser LLM (ONNX)

→

3. Pick a model

→

4. Download & use

The first download happens once per browser — after that, the model is served from the browser's own cache, so switching tabs or reloading the page does not require re-downloading. Switching to a different model triggers a new download for that model only.

FAQ

How is this different from Ollama?

Ollama runs as a separate local server process on your machine that the browser talks to over HTTP. In-Browser models have no separate process at all — they run inside the browser tab's own JavaScript engine and GPU access, using Transformers.js. Nothing needs to be installed outside the browser.

Does this require WebGPU support?

No — WebGPU is used when the browser and GPU support it, for faster inference. If WebGPU isn't available, the same models still run via WASM on the CPU, just slower. If you specifically want a WebGPU-only engine with a different model catalog, see In-Browser Models (WebGPU).

Does my data ever leave my computer?

No. Once the model weights are downloaded, generation happens entirely inside the browser tab — no network requests are made for inference.

Which model should I start with?

Llama-3.2-1B-Instruct is the most widely used and stable in-browser chat model for most users. If your machine is very low-powered, start with SmolLM2-135M instead.

Is this the same as the Local Language Models (Ollama) catalog?

No. See Local Language Models for the Ollama-based catalog, which requires the separate Ollama application to be installed and running.