AI Schema Generation

Generate structured JSON schemas from sample data using AI, with automatic self-correction and intelligent post-processing.

How It Works

Schema generation turns raw entity data into a typed, annotated JSON schema that defines exactly what information to extract during enrichment. Instead of manually writing schemas, you paste sample JSON and let AI analyze the structure, infer types, assign expertise domains, and suggest improvements.

The Generation Pipeline

Generation is not one big prompt. It is a sequence of small, single-concern calls, most of them running concurrently — which is what lets small and cheap models produce a usable schema, since each call answers one narrow question about material it can hold in view.

  1. Canonicalize the sample (no LLM) — a value carrying its own unit becomes a number with the unit in the name ("8.275 h" becomes half_life_seconds: 29790), and a date no native type can hold becomes an integer year. Every observed value has to prove the claim, or the property stays text. The rewritten sample is what gets saved.
  2. Identity scoping — one call, run before all others because it is the last one allowed to change your sample. Where an item in a related array mixes facts about that entity with facts about its pairing to the parent, the item is restructured: pairing facts stay put, the entity's own facts nest under a named subobject. Without this the two kinds of fact share one identity.
  3. Derive the skeleton (no LLM) — the property tree, JSON types and nullability come straight from the sample(s); a shape reused at several sites becomes a reusable definition, and a single-site object earns one only once the relationship check recognizes an entity there. Localized objects (like {"en": "...", "fr": "..."}) collapse to a single multilingual value.
  4. Ask the parallel questions — separate concurrent calls settle the entity's identity and naming, the behavioural flags (key, preserve, multilingual, nullable, plus format proposals), whether whole-number fields are genuinely discrete, which strings come from a closed vocabulary, and how properties route to expertise domains.
  5. Write the documentation — one call per expertise domain, in that domain's persona, producing each property's description and examples — and its own second opinion on whether the value can genuinely be absent.
  6. Assemble, validate, save (no LLM) — the fragments are merged, the 8 validation rules run as a safety net, deterministic post-processing settles flag conflicts, and the schema is saved — deduplicated by content hash, so identical schemas are not duplicated.

Each step retries on its own (3 attempts) and its answers accumulate across attempts, so a model that replies in fragments still converges. After that the step accepts what it got and the gaps are filled deterministically — a weak model degrades the descriptions rather than failing the generation. Only identity and domain routing are allowed to fail the whole run. Every call is billed and logged as its own prompt, so the record shows exactly what was spent where.

You can pass several samples of the same entity type instead of one — the schema then covers the union of their fields, anything missing from a sample becomes nullable, and the values seen across them become real examples. Field names must match: samples describing different entity types are refused, and so are objects inside an array that share no field at all, since nothing would be left to identify their rows. The sample editor flags any such difference before you spend a generation.

Values that carry their own unit are converted to numbers before the schema is derived, because a column of "8.275 h" and "85 ms" cannot be sorted, filtered by range or aggregated. The unit moves into the property name (half_life_seconds), a non-numeric stand-in like "stable" becomes null, and dates reaching before year 1 become an integer year (negative for BCE), which no date type can store and text sorts wrongly. Your sample panel is updated to match, so it always shows the sample the schema describes. Anything the conversion cannot read with certainty is left exactly as you wrote it.

Self-Correction, Step by Step

Because each step answers one narrow question, correction can be narrow too: a step's validator keeps whatever came back usable and asks again only for what is missing. Nothing is regenerated from scratch, so a partially-correct answer is progress rather than a wasted attempt.

Example: the flags step on 30 properties

Attempt 1The model answers for 22 of them, and splits its reply across several tool calls — a common failure mode on small models. All 22 are kept.
RetryThe follow-up asks for the 8 remaining properties only — a shorter question, more likely to be answered whole.
Attempt 26 more arrive. The last 2 fall back to deterministic defaults, and the shortfall is recorded on the generation record instead of failing it.

The eight validation rules still run over the assembled schema as a final check — type correctness, expertise assignment, reference integrity, completeness. By then they are a safety net rather than the correction mechanism. Learn more about each rule in the Validation Rules guide.

What the Schema Contains

A generated schema is more than a simple type definition. Each property includes metadata that guides the enrichment process:

Type

JSON Schema type (string, number, integer, boolean, array, object)

Description

Contextual description that tells the AI what information to find

Expertise

Which expert domain (financial, regulatory, etc.) provides this value

Identifying

Whether this field is part of what identifies the instance. Identifying properties do both jobs at once: they focus the enrichment prompt on the right entity, and they are what fusion matches array items on. One may still be nullable — a qualifier that separates look-alike siblings stays identifying even when whole families genuinely lack it

Value vocabulary

Where a string's values come from a small, conventional set (statuses, grades, classification codes), generation proposes the members — spelled the way your samples spell them — as an open vocabulary enrichment converges on. Values seen beyond the list surface as candidates in the editor, and you close the set once it stops growing; a closed set becomes a hard contract enrichment cannot drift out of

Nullable

Whether the field can be null — non-nullable fields are required for database admission

Multilingual

Whether the field should be enriched across multiple languages

Preserve

Whether to keep the original value unchanged during enrichment

Examples

Realistic example values that guide the AI toward the right format

Format / Pattern

Machine-checkable shape for string values: malformed answers are rejected and retried, and stored values keep the canonical form. Generation only ever claims a named format (date, time, date-time, uuid, email, uri, ipv4, ipv6) its samples prove — a regex pattern is a prediction about values nobody has seen yet, and a wrong one fails every enrichment of the field, so you add that one yourself in the editor

Expertise Domain Detection

The AI groups schema properties into expertise domains based on their semantic meaning. For example, a pharmaceutical company schema might have domains like “Financial Analyst,” “Regulatory Expert,” and “Corporate Information.” These domains are used by the multi-expertise strategy to run parallel, specialized LLM calls for deeper results.

  1. 1Regroups the table by each property's domain
  2. 2Properties this one expert call answers
  3. 3The persona that call is given
The domain is an attribute of each property, not a layout — the toggle only rearranges the view. The sentence beside a domain name is the persona its enrichment call is given, and the count is how many properties that single call has to answer.

Domain Count Limits

The number of expertise domains is automatically limited based on your data's property count to prevent over-fragmentation:

5 properties
1 domain
12 properties
2 domains
30 properties
5 domains
60 properties
10 domains

Post-Processing

Once the fragments are assembled, deterministic steps settle everything that should not be left to a model — using your actual input data as the evidence:

Nullable widening

A field missing or null in any sample becomes nullable whatever the model answered, so an unknown value is an accepted answer rather than a data-quality failure. Samples can only widen: a handful of samples proves presence for those instances, never for every instance of the type — which is why the model gets a vote too, and the two are ORed.

Flag conflict resolution

Attributes that cannot coexist are reconciled by rule rather than by asking again: preserve wins over multilingual and nullable, a surviving closed vocabulary clears multilingual, and a key property never keeps an enum.

Array-item key repair

Every object inside an array is guaranteed at least one key property — it is the unit fusion deduplicates on, so an array item with no key would make merging two models' answers impossible.

Expertise collection

All unique expertise domains are gathered from the schema for metrics and strategy configuration.

The language a schema is written in

A schema describes itself in a language — its type names, property descriptions, expertise labels and suggestions. That is a different thing from the languages you enrich into. Name one when you generate, or leave it out and the language of your sample's own property names decides: a French sample stops producing an English schema. The choice is stored, so later AI edits keep writing in the same language instead of drifting back to English.

Generating the samples themselves

You do not need sample data to start. Describe the sample you want in plain words — the kind of entity, the properties it must carry, how big or how deep — optionally with documents to ground it in, or web search to check it against reality — and the platform writes the samples for you. Ask for several and you get several different instances, not one instance restated.

  1. 1The request: entity kind, properties, size budget
  2. 2How many instances you get — never how many fields
  3. 3Auto lets the model pick the naming convention
“Typical examples” is where you name the instances you already have in mind — one per sample, matched in order; the slots you leave empty are invented for you.

All the instances are chosen at once

The first sample also decides who the others will be about, in the same call. Asking N times independently for “an example” reliably returns the same famous instance N times; naming the whole cast up front is what makes them distinct.

The first sample fixes the shape

The remaining samples are generated in parallel against sample 1's structure as a contract, not merely asked to match it — so a variant cannot rename, add or drop a field. Any that still come back duplicated or malformed are re-requested in a bounded retry wave, and if the full count is not reached you are told rather than silently given fewer.

Your request is binding — and answered with questions when it has to be

Everything in your request is honoured — a size budget beats the generator's urge to be exhaustive, a shape you asked for beats its default — or the response tells you what could not be honoured and why, never quietly dropped. One thing is not a request: how many samples you get is the sample count, and each sample is exactly one instance — asking for “three samples” in the text never yields a list wrapped in one object. When the request is genuinely ambiguous (a kind of entity with several readings, two incompatible scopes), the generator asks you before spending the generation, instead of guessing. Language defaults to auto, inferred from the words of your own request, then from any attached document.

Ambiguity Check

Read a property name in the context of its parent object and count the distinct things it could be asking for. One is clear. Two or more and each model settles on a different one, so the column ends up mixing answers to different questions — annual_revenue on a company can be the group or the entity, gross or net, in one of several currencies. None at all — a name for something this parent simply does not have — is worse: with nothing to look up, the model invents a value.

Generation fights this twice: the prompt itself demands names that admit a single reading, and a post-pass over the finished schema annotates the ones that still do not. At this stage the remedy is a rename — the descriptions were generated from the names, so a description cannot disambiguate the name it came from, and nothing depends on the schema yet. Sample generation runs the same check on the sample and applies the renames before you ever see it. Free-text prose — a description, a summary, notes — is never flagged: the wording varies, but the question asked is clear.

Flagged properties show an “ambiguous” badge in the Workflow Editor, listing the readings the name admits. Once the schema is live the remedy flips to a rewritten description, which pins one meaning without breaking the data contract. See the Ambiguity Check guide for the full rubric and its remedies.

When generating a sample entity from a description, you can enable “Use web search” to let the model look up current facts on the web instead of relying on its training data alone. This produces fresher, more accurate sample values — especially for fast-moving facts like prices, staff counts, or recent releases. The option is only available for models whose provider supports built-in web search, and search calls are billed by the provider like any other model usage.

  1. 1Only for models with built-in search
  2. 2Switching model here can grey it out
The checkbox is tied to the model picked right below it: choose one whose provider has no built-in search and it greys out, stating that as the reason rather than silently ignoring your choice.

AI Schema Editing

After generation, you can modify schemas using natural language instructions. Type a command and the AI applies the change while preserving your existing schema structure. Each edit also produces 5 suggestions for further improvements.

Example Edit Commands

Add an employee_count integer field
Create a nested address object with city and country
Add French descriptions to all text fields
Define a parent company reference using $defs
Mark the website field as nullable

AI edits are validated using a subset of the generation rules (type checking, reference integrity, expertise consistency) without comparing against input data, since you may intentionally add or remove fields.

AI Suggestions

Both schema generation and AI editing produce 5 targeted suggestions covering different improvement categories:

Data completenessMissing fields that could enrich your entity
Data qualityClosed vocabularies, nullability, type corrections
RelationshipsNested structures, entity references via $defs
InternationalizationMultilingual translations, locale support
Business contextDomain-specific fields and expertise groupings

Suggestions appear as clickable chips in the Workflow Editor — click one to auto-fill the AI edit input and apply it.

Next Steps