Manage LLM providers and models, Sync models from external registries, run health checks, and configure per-organization API keys for independent billing.
Entity Enricher supports a wide range of LLM providers. Each provider can have multiple models with individual pricing, capabilities, and configuration.
Many teams route LLM traffic through a corporate AI gateway, a regional endpoint, or a provider that isn't built in — for example an enterprise LiteLLM proxy, Cloudflare AI Gateway, or Alibaba DashScope (for Qwen models). You add these as their own Standard (OpenAI-compatible)provider with a custom base URL.
acme-openai-gw). Built-in names like openai or anthropic are reserved. https://gateway.example.com/v1. This field is required for any provider Entity Enricher has no built-in client for. https:// URLs. Loopback and private ranges (localhost, 10.x, 192.168.x) are rejected to prevent SSRF — a self-hosted server must be reachable over the internet. For a local Ollama, use the dedicated Ollama tunnel instead./v1 protocol (chat completions, /models). {endpoint}/models to verify the key and base URL before you run an enrichment.Every call made with an API key is paced under the budget the provider grants that key — requests and tokens per minute, per model — so a fan-out never runs into 429 errors. The budget is not typed in: it is read off the provider's own response headers, learned from a rejection when the provider states nothing, or, as a last resort, typed by an owner.
This is separate from your plan's max concurrent jobs limit, which caps how many enrichment jobs your whole organization runs at once across all providers.
Each model tracks its capabilities, which are displayed as icons in the model selector:
| Capability | Description |
|---|---|
| Vision | Can process image and visual inputs |
| Tool Calls | Supports function calling / tool use |
| Audio Input | Can process audio inputs |
| PDF Input | Can process PDF documents |
| Prompt Caching | Supports prompt caching for cost reduction |
| Reasoning | Extended thinking / chain-of-thought capabilities |
| Embeddings | Turns text into a vector instead of answering — what semantic IDs are resolved with. Embedding models are their own family, with their own vector size, and never appear in an enrichment picker |
Naming a model is optional. Enrichment, schema generation and sample generation all accept auto — and treat an omitted model as auto — which is resolved on the server, per task, at the moment the job starts. The run reports which model it picked, so automatic never means opaque.
Owners can pin a preferred model per task under Settings → Organization → Model selection. If one is set for the task at hand, it wins.
Failing a pin, the pick is the model with the best blended score from your scoring-source benchmarks — your own measurements of quality, speed and cost on your own schemas. With no scoring source at all the request is refused rather than guessed at.
Turning on web search, or attaching a document that must be sent as-is, restricts the candidates to models that can actually do it — and if none qualifies, you get an explicit error instead of a silent downgrade.
A model can also be barred from one task without being deactivated: a model that enriches well but generates poor schemas can be hidden from the schema and sample generation pickers only, either for your organization or globally by an administrator. It stays fully available everywhere else — a softer instrument than the deactivation below.
Keep model pricing up to date by syncing from external registries. The sync process detects new models, price changes, and removed models automatically.
The default pricing source. Fetches from LiteLLM's community-maintained registry on GitHub with real API model names, pricing, context lengths, and capabilities.
Covers ~30 providers. Does not include display names, benchmarks, or generation speed.
An alternative source from pricepertoken.com. Includes display names, benchmarks (coding and math scores), and generation speed (tokens per second).
Covers ~20 providers. Provides richer metadata than LiteLLM.
An official authenticated catalog for GLM model identifiers, with pricing parsed directly from Z.AI documentation and capability gaps researched there.
Replaces Z.AI entries previously imported from LiteLLM and PricePerToken.
Proactively validate whether models are reachable by running a minimal health check prompt. This catches broken models before users encounter errors during enrichment.
Health checks can be run on all models, a specific provider's models, or a single model. Results stream in real time via SSE with a progress bar showing pass/fail counts.
When an enrichment call fails with a “model not found” error, the model is automatically deactivated to prevent repeated failures. This happens in real time during normal enrichment operations.
| Deactivation Reason | Set By | Auto-Reactivated? |
|---|---|---|
| Model Not Found | Enrichment errors, health checks, or a capability probe no route answers | Yes (by pricing sync or validation) |
| No Structured Output | Capability probe: neither the tool nor the native channel on any reachable route | Yes, by a later capability probe only |
| Sync Removed | Pricing sync (model disappeared) | Yes (if model reappears in registry) |
| Manual | Admin toggle in UI | No (manual reactivation only) |
Organizations can configure their own LLM provider API keys for independent billing and usage tracking. The system uses a two-tier key resolution with LRU selection:
Per-organization keys configured in the API Keys page. Supports multiple keys per provider with LRU rotation. Encrypted with Fernet.
System-wide keys managed by administrators. Shared across all organizations. Also supports multiple keys per provider with LRU rotation.
Each enrichment records which key was used, so you can track costs per key. Keys include health check support and usage counters. Within a pool, the enabled key with the oldest last-used timestamp is picked next; a key leaves the rotation only when you disable it by hand, so a provider error never silently removes a key from service. Learn how to manage keys in the API Keys guide.
Export your entire provider and model configuration as JSON for backup or transfer to another instance. Importing is always an upsert: existing providers and models are matched by name and updated in place, while new ones are added — nothing is deleted.
The export includes provider settings, model configurations, pricing, capabilities, and the canonical model specs — but never API keys, which are stored separately. After importing, configure API keys separately. System admins back up the full global catalog; organization owners export and import only their own organization’s providers and models — the shared global catalog can’t be created or edited through import.
The models page presents the global catalog to anyone: vendor pricing, measured capabilities, and the scores every model earned on the benchmark scenarios published as global scoring sources. It reads two static JSON files that the nightly model refresh rewrites, and that you can download and reuse. A model the provider no longer serves (deactivated as “model not found”) is left out; every other model of the catalog is listed.
/data/models.json — the table: one entry per provider × model, with lookup tables for providers, scenarios and specs./data/benchmarks.json — every public benchmark result, grouped by model key.Both are served with an ETag and a one-hour public cache, gzip-encoded when the client accepts it. The version field increments on any change a consumer would have to adapt to.
generated_at, counts, default_weightsWhen the file was written, how many models, providers and scenarios it holds, and the quality / speed / cost blend (percent) behind every overall score.providers[], scenarios[], specs{}Lookup tables: models reference a provider and the scenarios by index; specs are the public benchmark scores of the weights (intelligence, coding, math, and the rest under extra), keyed by canonical key so resellers of one model share them.models[].key, model, display_name, canonical_keyThe composite key the API accepts (provider::model), the raw model id, its label, and the cross-provider identity.models[].pricingVendor list prices in USD per million tokens: input, output, cache_read, cache_write, cache_write_1h, reasoning_output, plus web_search_per_query with its unit. Before any plan commission.models[].capabilities[]The flags that hold: vision, pdf_input, audio_input, audio_output, video_input, tool_calls, tool_choice, response_schema, strict_structured_output, reasoning, reasoning_effort, web_search, prompt_caching, embeddings, requires_streaming. A missing flag is false or unmeasured.models[].context_length, max_input_tokens, max_output_tokens, deprecation_date, latencyLimits, the vendor's retirement date when announced, and the scraped latency figures (tokens per second, time to first token).models[].enrichment_capable, disabled_tasks[]Whether the model has a structured-output channel at all, and the tasks the app never offers it for (classification and arbitration need tool calls; schema and sample generation follow the schema-generation gate).models[].scores{task}Per task type (enrichment, schema_generation, sample_generation): the mean quality, speed and cost over the public scenarios of that task, the overall under the default weights, and the scenario indexes. Speed and cost are relative to the other models on the same scenario.How the quality, speed and cost scores are computed is explained in Benchmark Scoring.