Non-Determinism Check - Entity Enricher Documentation

Non-Determinism Check

Detect schema properties whose values vary between models or between runs, and fix them before they pollute your data — for enrichments that are reproducible, comparable, and precise.

Why Determinism Matters

Entity Enricher treats LLMs as queryable knowledge bases. That only works when a property name pins down a single reproducible answer. A property like annual_revenue does not: which fiscal year? Which currency? Millions or units? Every model — and every rerun of the same model — is free to pick a different frame, so the values disagree even when no model is wrong.

Volatile properties hurt everything downstream: multi-model fusion sees conflicts that are really framing differences, and benchmark consistency spreads widen for reasons that have nothing to do with model quality. The non-determinism check finds these properties and tells you how to pin them down.

The Four Causes

Every flagged property is classified into one of four causes, each with its own remedy:

CauseExampleRemedy
Temporal
ceo, stock_priceceo_2025
A moving fact with no anchor date — pin the period, or restructure into an array keyed by period.
Ambiguous
annual_revenuerevenue_fy2024_usd_millions
The name doesn't pin the unit, currency, period, or scope — embed the missing frame in the name or description.
Subjective
reputationesg_risk_rating_sustainalytics
No objective ground truth — replace with a measurable proxy or a fixed scale; otherwise it stays flagged as advisory.
Multi-valued
top_competitorstop_3_competitors_by_revenue
Several equally correct answers, usually an unbounded list — bound it and state the ordering criterion.

A rename is not the only fix: for schema properties the analyzer also proposes a description fix that pins the missing frame. A sharper description reaches the enriching AI exactly like the name does, but never breaks your data contract — so it is the preferred remedy for schemas already in use.

Where the Check Runs

During sample generation

After a sample is generated, the analyzer reviews it. Unambiguous renames are applied automatically to AI-invented keys (never to fields you named yourself), and structural fixes are fed back to the generator for up to two correction rounds. Samples grounded in attached documents are skipped — their values come from the source document, not from model recall.

After schema generation

Once a generated schema is saved, a post-pass annotates every property with its non-determinism level. This step is best-effort: if it fails, the generation itself is unaffected.

On demand from the Schema Editor

The Re-check button analyzes only properties that have no annotation yet, and switches to a full re-analysis once every property is annotated.

On pasted samples

Sample JSON you paste for schema creation can be analyzed statelessly — you get a report of risky property names without anything being modified.

The check is advisory only. It never blocks generation, never rejects an enrichment, and its annotations are stripped from every prompt sent to the enriching models — it informs you, not the AI.

Reading the Results

Flagged properties show a “varies” badge in the Schema Editor: amber for medium risk, red for high risk. Hovering the badge shows the analyzer's note explaining what varies and why. Properties judged deterministic carry no badge.

Annotations Follow the Property

A verdict is rendered on a property's name and description together — so renaming a property, or editing its description, drops its annotation. The editor highlights such properties as stale and offers a re-check, which analyzes only what is missing. This is exactly what you want after applying a suggested rename: the re-check confirms whether the new name is deterministic.

Per-Schema Toggle

The check can be switched off per schema from the Schema Editor's overflow menu. When disabled, the post-generation pass is skipped and the badges, Re-check button, and stale warnings are hidden. Existing annotations are kept (just hidden), and re-enabling the check on a schema that was never analyzed runs it automatically.

Schemas generated from attached documents start with the check disabled: their values are transcribed from the source document, so run-to-run variance in model knowledge does not apply.

Good to Know

Notes speak your language

Analyzer notes are written in your interface language — a French user sees French explanations, a Japanese user Japanese ones. Suggested property names stay in English, matching schema naming conventions.

Analysis is a billed AI call

Each analysis is a real (cheap) model call, recorded as its own prompt on the record and deducted from credits like any other AI usage. Incremental re-checks only pay for the properties actually analyzed.

Prevention runs upstream too

Schema generation itself is instructed to produce deterministic names and descriptions (pinned periods, units, ordering criteria), so most schemas come out clean and the analyzer only has to catch the stragglers.

API & MCP Access

The check is available programmatically:

SurfaceDescription
POST /api/schema/analyze-sampleAnalyze pasted sample JSON — stateless report, nothing is modified
POST /api/schema/saved/{id}/analyzeAnalyze a saved schema and write annotations — incremental by default, force=true re-analyzes all
analyze_sample_determinismMCP tool — same stateless sample report from Claude or any MCP client
analyze_schema_determinismMCP tool — annotate a saved schema; combine with edit_schema to apply a suggested rename

See the API Reference and the MCP Server guide for authentication and the full tool catalog.

Next Steps