AI Models

AI Models Sub-tabs

The AI Models tab is split into sub-tabs so that each kind of model has its own place. The OpenRouter free requests (today) card — how many of your daily free requests are left — is shown at the top of every sub-tab, with a Refresh button. General AI settings (usage limits, agent behaviour, governance) are on the Agentic tab.

Sub-tab	What it holds
Chat Model	The OpenRouter free-request quota and the AI Providers card where you pick the AI engine and, for Cloud LLM, the hosted chat model.
Embedding	Where Semantic Search gets its vectors: local Ollama or Cloud LLM (Hosted API). See Embedding Models.
Speech (TTS / STT)	Pick the text-to-speech model used by Audio Creator → Text to Speech and Voice Clone. Besides the Modelion and OpenRouter lists there is a Fish Audio (direct API) group (S2.1 Pro Free, S2.1 Pro, S2 Pro, S1, Drama 3) that calls Fish Audio from the server and needs FISH_API_KEY. Transcription models are listed for reference; Speech to Text uses its own Whisper model.
Rerank	Pick the model that re-scores the best Semantic Search matches before they are shown. Leave it unselected to keep the plain ranking. See Rerank.
Image, Video, Decisions, Audio	Reference lists of the models your providers offer for each type. They are for browsing only — nothing in the app uses a selection from these lists yet.

How the model lists work

• Modelion first, then OpenRouter. Both lists are read live from the providers, so new models appear without an app update. A provider with no models of a given type is simply not shown there.
• Free models are marked. OpenRouter models that cost nothing carry a green free badge, and the OpenRouter: free models only checkbox on the right of the sub-tab bar narrows the OpenRouter lists to them. The checkbox applies to Modelion models in no way — they have no free/paid distinction.
• Models your account cannot use are visible but disabled. A Modelion model that needs your own provider key shows a red byok_binding_required instead of a type label and cannot be selected. Connect that provider's key in the Modelion console and it becomes selectable on the next load.
• Non-chat models stay out of the chat list. Embedding, rerank, speech and similar models appear only in their own sub-tab.

AI Engine Selection

Choose which AI model is used for agentic workflow steps. This is separate from the Terminal Writer Engine — it applies to workflow node execution, task analysis, and assistant panels.

Engine	Description
Hosted API (default)	Calls the Anthropic Claude API (claude-sonnet-4-6) via the server. Requires ANTHROPIC_API_KEY in the server environment.
Local LLM	Uses an on-device BERT model. Supports classification tasks only — generation is not available.

Embedding Models

The Embedding sub-tab of AI Models lets you choose where Semantic Search and knowledge ranking get their vectors: a local Ollama model you download and select, or Cloud LLM (Hosted API) with a Modelion or OpenRouter embedding model. It is separate from the text-generation engine on the Chat Model sub-tab. Changing the embedding model makes vectors already stored on your knowledge sources outdated; they are recomputed automatically the next time those sources are ranked.

Rerank (Second-Pass Ranking)

Semantic Search first shortlists the records closest in meaning to your text. Rerank is an optional second pass: a dedicated model reads your text together with each shortlisted record and re-scores them, so the most relevant ones rise to the top. Today it is used by the Trade Compliance & Sanctions screen's Search Similar Rules step. For the full picture, see Reranking and Embedding.

Turn it on

1. Open Settings → AI Models → Embedding and make sure an embedding model is selected (local Ollama or Cloud LLM).
2. Open the Rerank sub-tab. Tick OpenRouter: free models only if you want a no-cost model, then click a model, for example NVIDIA: Llama Nemotron Rerank VL 1B V2 (free).
3. To turn Rerank off again, click the selected model once more.

What to expect

• It runs only when there is something to re-order. Rerank needs at least two matches above the 60% similarity threshold. With one match, or none, it is skipped — that is normal, not an error.
• A green Reranked label appears above the results when the second pass was applied. The percentage on each row is then the rerank score, not the similarity score, so the percentages of the two orderings should not be compared with each other.
• It never blocks you. If no model is selected, or the rerank service is unavailable or over its quota, the screen quietly shows the plain Semantic Search order — no error, just no Reranked label.
• An embedding model is still required. With none configured, Semantic Search falls back to simple keyword matching and Rerank is never reached.