Embedding Model

Why Ollama, and Why Local

By default, embeddings for Semantic Search are computed by a local Ollama instance running on your own machine at 127.0.0.1:11434. (You can opt into a hosted model instead — see Hosted (Cloud LLM) Embedding below.) With Ollama this is a deliberate architectural choice: your knowledge library — potentially sensitive documents, notes, or internal material — never has to leave your computer just to be ranked. TotalApp's cloud-hosted server has no network path to your Ollama instance; the ranking always happens directly in your browser talking to your own machine.

This also means the Embedding Engine setting is completely independent of your Writer Engine choice (API, Local LLM, Ollama) for text generation — you can generate content via the Anthropic API while still using a local Ollama model purely for embeddings, or vice versa.

Screen preview

A quick look at the screens. Use the toggle in the corner of each preview to switch between the dark and light themes.

Assisted or automatic

Setting Up

1. Install Ollama on your machine if you haven't already.
2. Pull an embedding model, e.g. ollama pull nomic-embed-text.
3. In TotalApp, go to Settings → AI Models → Embedding Engine.
4. Confirm your Ollama connection is detected (test it under Text Generation Provider if it isn't showing as connected).
5. Select your embedding model from the Semantic Search (Embedding Engine) dropdown — only models with embedding-oriented names (containing "embed", "bge", "nomic", or "minilm") are listed here, so it never gets mixed up with a text-generation model.

Hosted (Cloud LLM) Embedding

In Settings → AI Models → Embedding, set the provider to Cloud LLM (Hosted API) and pick a model from the Modelion or OpenRouter list. Semantic Search then ranks your sources with that model instead of Ollama, with no local install needed.

Model Separation — Why You Can't Pick a Chat Model Here

The Embedding Engine dropdown only lists Ollama models whose names indicate they're built for embeddings (e.g. nomic-embed-text, bge-small, all-minilm). General-purpose chat/completion models (e.g. llama3, deepseek, qwen) are excluded from this list — and, at runtime, TotalApp guards against calling a chat model's embedding endpoint by mistake. This keeps the two roles cleanly separated: one model type talks to /api/generate for writing, another talks to /api/embeddings for ranking.

Understanding Relevance Scores

Cosine similarity between two text embeddings ranges in theory from -1 to +1, but real-world scores from general-purpose models like nomic-embed-text cluster in a much narrower practical band:

Unrelated content rarely scores near 0% because language models share common "general language" structure across nearly any text. This is normal and expected — a 30% score on an unrelated document is not a bug. See the info tooltip next to Embedding Engine in Settings for the same explanation in-app.

What Happens Without an Embedding Model

Nothing breaks. My Knowledge and every Writer Tool's Attach Knowledge picker continue to work exactly as they did before Semantic Search existed — sources are listed in their original order, with a note that ranking is currently unavailable. Ranking and Automatic auto-attach are the only features gated on having an embedding model configured.

FAQ