Database Sync — mirror enrichments into your own PostgreSQL

Link a database to a schema and Entity Enricher maintains your enriched entities as relational tables you own: a real company table with a revenue column — not an export file. Download a ready-to-run SQL snapshot once, then keep your database converged with an incremental delta feed of idempotent upserts.

ENTITY ENRICHERYOUR INFRASTRUCTUREdelta feedsnapshot.sql · first runack · cursor advancesEnrichmentcompletesEntity layerentities · links · keysDelta outboxper-database FIFOYour databasePostgres · MySQL · SQLite
Continuous delta feedAcknowledge to advance

Store once, project on demand. Enrichments write current entity state to the entity layer; each linked database is a projection of that state, delivered as a one-time snapshot plus a delta feed you acknowledge. Editing a schema costs a re-projection, never a data migration.

How it works

  1. 1. Register a database on a schema. Entity Enricher verifies the schema first: the root object and every object stored in an array must have an identity — a database key proposed at link time (or set by hand), a semantic ID, or non-nullable key fields. Registration then confirms the schema’s database keys (reviewed by you) and issues a webhook signing key, viewable anytime on the database’s Overview.
    1. 1Which language keys a multilingual key column — locked once rows ship
    2. 2The keys your rows will merge on, proposed for review
    3. 3Surrogate or natural: the one decision a running sync cannot take back
    Nothing here asks for a connection string: Entity Enricher never connects to your database, it only declares the projection a consumer you run comes to fetch.
  2. 2. Enrich as usual. Every completed enrichment upserts the entity’s current state — new non-null values win, nulls never erase what a previous run found — and queues ready-to-run SQL deltas for each linked database. The amber database toggle in the Workflow Editor and Batch toolbars turns this routing off for your own browser’s runs (API callers pass database_sync: false); other users’ enrichments keep routing normally.
  3. 3. Seed your database. Download the .sql snapshot (tables + data) and apply it to your PostgreSQL. The DDL ships with join indexes and foreign keys (child and link rows cascade when an entity is deleted). The file header tells you which delta cursor to resume from.
  4. 4. Stay in sync. Pull the delta feed — by webhook notification or on a schedule — apply each SQL statement, and acknowledge. Deltas are idempotent and replay-safe: applying one twice, or after a missed batch, always converges to the same rows.

Database keys: what your rows merge on

Each entity type has a Database key — the column set your tables use as their unique index and upsert conflict target. When you link a database, an AI classification pass proposes the whole database model (keys, column types, indexes, ownership) for you to review, falling back to a simple ladder: the object’s semantic ID if it has one, otherwise an Id-like field (id, product_id, …), otherwise the object’s natural keys. In your tables a semantic ID arrives as a semantic_id column — the name id stays free for your own use. You can change them anytime in the database’s “Model” tab — a migration-grade change once published: re-download the snapshot afterwards. Nothing reaches your database until the schema is published from that tab (linking alone ships no tables and no rows).

A database key is never nullable. A null value matches no row, so instead of updating the entity it would insert a new duplicate on every enrichment — which is why an enrichment whose key comes back empty is rejected rather than saved. The editor keeps the two flags apart, and a schema that sets both is refused when you save it or link a database, naming the two ways to fix it: make the property always present, or key the type on a different one.

When a database key is an enriched text field rather than a semantic ID, expect the stored value to be the model’s answer — not the text you sent. Identifying a company as Embraer in your request does not pin that field: the enrichment may answer Embraer S.A. — a fixed spelling, an expanded legal suffix, a dropped disambiguator — and that is what keys the row. So looking the row up by the value you sent can miss it (use the entity_keys returned with each saved enrichment), and a later run that phrases it differently is a different key — it inserts a second row instead of updating yours. This is expected of any key made of enriched text. The fix is a semantic ID on that type: variants of one name resolve to a single stable identity, so the row survives however the model spells it. Turn it on when you generate the schema — adding it later means editing every object.

Objects nested inside arrays become their own tables with junction rows preserving order; value objects without identity stay flattened into their parent’s columns. Property names are used verbatim (quoted) — your column names are your schema’s property names, shortened only where a nested path would outgrow what PostgreSQL can name.

  1. 1Edits stay in the working copy until this is pressed
  2. 2The key this type merges on — here its semantic ID
  3. 3The SQL type, proposed and overridable
One tab per projected table, nested types included — each carries its own keys, column names and indexes. The grey line under a property is the column name it will take in your database.

How your schema becomes tables

The projection is deterministic — the same schema always maps to the same tables and columns. Property names become column names verbatim (quoted); type names are snake-cased into table names (VideoGame → video_game). The one exception is length: PostgreSQL cannot hold an identifier past 63 bytes, so a deeply nested path shortens its parent objects to fit (morphological_description_ → morpdesc_) — the same prefix for every column of that object, shown and editable in the Model tab before you link. A property can also drop its prefix entirely to match a column your database already has (product_identifiers.stock_keeping_unit → sku) — the link toggle next to the name in the Model tab. Every table carries a _sync_revision column used to keep replays convergent.

In your schemaIn your database
Object with identity (semantic ID or keys)Its own table; database keys become the unique index & upsert target
Scalar field (string, number, boolean)A typed column (TEXT, BIGINT, NUMERIC, BOOLEAN)
Closed set (a field limited to a list of values)A plain TEXT column — no CHECK, no database enum type. The list is enforced when the AI answers, so adding a value to it later never migrates your database. Add your own constraint if you want one — the sync never touches it
Nullable or non-nullable fieldA non-nullable field becomes a NOT NULL column by default, paired with the quality gate below — under the strictest gate, required references get NOT NULL foreign keys too; switch the enforcement off — at registration, or later: a change after the first sync ships as a guarded migration in the feed — to keep every column nullable and let the gate alone enforce completeness
Multilingual fieldOne JSONB column holding every language
Embedded value object (no identity)Flattened into prefixed columns (dimensions_width)
Array of value objectsA child table keyed on the parent, ordered, cascading on delete
Array of entities / $ref relationA junction table linking source and target rows, order preserved
Key field (identifying)A secondary index for fast lookups
Query-shaped index (ordered field list)One multi-column index per declared shape, ordered like the list-screen query it serves — facets and closed sets first, the sort or range column last, multilingual fields included (such a shape ships once per language your database has received); proposed by the classification pass with a reason, curated in the Model tab, several per entity
Search field (index intent)A trigram index (pg_trgm) on text your search boxes match by fragment — per language and uncapped on multilingual columns; never on a dropdown value, which belongs in a query-shaped index instead (multilingual ones included — such an index ships once per language your database has received); skipped (no-op) on replicas without the extension until a database owner installs it
Coordinate pair (latitude + longitude)One spatial index over the pair (native PostgreSQL GiST, no extension) — radius, nearest-neighbor and map-viewport queries
Interval pair (start + end bounds)One range index over the pair — overlap and "which value was in force on this date" queries
  1. 1A semantic ID becomes the table's key column
  2. 2An embedded list: its own table, cascading on delete
  3. 3An array of related entities: a junction table
  4. 4A multilingual value: one JSONB column, every language
The rules above, drawn: the sync’s Diagram tab renders the published model, its legend naming every column and edge kind — a trailing question mark marks a nullable column.

One database, several schemas

A database can sync more than one schema. Entity types with the same name across the linked schemas land in the same table, merged by their database key — each schema’s enrichments update only its own columns, so a company enriched by two schemas becomes one row carrying both column sets. Types unique to a schema simply add their own tables, delivered through an automatic migration delta in the feed — no re-download needed.

When you link a schema, a compare step shows exactly which tables will be merged (with their keys and added columns) and which are new; a schema that shares a table adopts that table’s existing database keys, shown for your review. If the schemas share nothing, the flow suggests a dedicated database instead. Unlinking a schema never touches your database — the synced tables stay.

Unlinking a schema, or deleting a sync, leaves two things behind on our side: the stored entity state nothing writes to anymore, and the database properties the schema carries (database keys, column types, indexes, ownership). Both confirmations offer to remove them, and only for schemas left with no database at all — one still synced elsewhere keeps everything. The schema itself, its enrichment records and its costs are never affected.

Schema changes migrate, they never surprise

A linked schema has a published contract: the version your enrichments and your database actually use. Editing the schema only touches a working copy — wording changes flow through automatically, while structural changes (new fields, type or key changes) wait until you press Publish. Publishing previews the exact impact and ships the right migration into the delta feed: new columns arrive as ALTER TABLE deltas, and heavier changes (a new database key, a type change) run as guarded migrations against your own database — if data blocks them (a missing or duplicate key value), the feed pauses with the exact problem and retries automatically once you fix it.

Publishing lives in the database’s Model tab (the Workflow Editor shows a banner pointing there while the schema is linked). Before you publish, it shows both sides: the contract your database is on today, and a diff of everything your working copy would change. Not convinced by an edit? Revert to published puts the contract back — undoable, and the draft you set aside stays restorable for 24 hours.

Re-linking a schema that was edited while unlinked works the same way: the sync remembers what your database already has and sends only the difference, plus a snapshot refresh for the rows written in between. No manual DROP, ever.

The same promise covers our own upgrades. When a new release improves how schemas map to tables, your sync is migrated for you — additive changes arrive in the feed on their own. If an upgrade would re-shape tables you already hold, we never touch your data unannounced: delivery pauses and your Database Sync page asks you to apply it, showing exactly what changes first.

Multilingual, relational

Multilingual enrichment is first-class here too: localized values arrive as JSONB columns carrying every language of the enrichment — {"en": "Headache", "fr": "Céphalée"} — so one database serves all your locales at once. Pick a language right in your queries (name->>'fr'), and the JSON delta payloads carry the same language-keyed objects.

Quality gate and rejection events

Every database answers one question at registration: when an enrichment comes back with gaps — non-nullable fields unfilled — what is written? The three answers form a ladder. Nothing: one gap anywhere, including inside a nested object, and the entity is rejected. The entity, without its incomplete children (the default): the entity’s own row must be complete, but a broken child is skipped and reported instead of sinking the whole enrichment. Everything: gaps land as NULLs and nothing is rejected — but entity state is last-write-wins, the latest enrichment is the row, so a later partial run erases what an earlier one filled. That erasure is exactly what the two strict rungs exist to prevent.

  1. 1The default rung, pre-selected
  2. 2Mirror the same contract in your own database (next paragraph)

Enrichments that fail the gate are still saved as records, and still fire the record.created webhook — with database.saved set to false — telling you exactly which required fields were missing — so incomplete data never silently disappears. Each missing field also says whether a model declared it unknown or just left it out: the first calls for a stronger model, web search or a source document, the second for a look at the schema or the input. Only the database-key fields are always required: an enrichment missing a key value is rejected whatever the rung.

On the strict rungs, a checkbox mirrors the same contract in your own database as NOT NULL columns on every always-present field. Under the strictest rung the foreign keys of required references are constrained too — no admitted row can lack them. Under skip incomplete children they stay nullable, deliberately: a list item missing a value of its own is dropped, and a shared one-to-one reference whose target is incomplete (the stadium whose opening year nobody knows) is detached — that target is neither written nor updated, and the saved row links to nothing there, which writes NULL into exactly those foreign-key columns. Both are reported on the enrichment response, and gaps in top-level fields always reject. Changing the policy after the first sync is never lost work: it ships as a guarded migration in the feed, validated against the rows your database already holds.

A second gate catches duplicate identities: when two items of the same list resolve to the same database key — a model inventing one id for two different companies, or a key that doesn’t tell them apart — only one row can exist, so the last one is written and the earlier ones dropped, the same last-write-wins rule as everywhere else. Each collision is reported with the identifying values of both items and a verdict: a duplicate repeated the same values and lost nothing, a conflicting drop lost the values it names — either the model repeated one thing noisily, or these are different things and the key needs a discriminating property (a region, a year, a version). The list is kept on the record, so a partial write still says what it lost long after the response is gone.

One thing to know before you re-enrich: for a list that belongs to its parent, the newest enrichment’s list is the list. A child row the latest answer doesn’t repeat is deleted from your database — that is how a genuine removal reaches you, and the response does not report it. It matters when the list is one the model recalls rather than enumerates: ask twice for the isotopes of an element or a person’s awards and the second answer may be shorter, which removes rows that were true. Keep your own history if you need the union of every run.

Send results on your terms

Enrichments reach a linked database on their own. Everything else — a result the gate refused before you fixed the schema, a run you deliberately kept out, or output you want to review or correct first — goes through sending records to the database: select them on the History page, or call the API from a workflow.

record savededited outputdeltasrefused · with the fields that failedan edited output becomes its own recordEnrichdatabase sync offYour workflowreview · correctInjectcontract + gateYour databaserows appear

The round-trip. Enrich with database sync switched off, reshape or approve the result in your own workflow, then send it. What you send is re-validated against the schema’s published contract and passed through the same admission gate an enrichment goes through — an injection can never write what an enrichment could not.

Two details worth knowing. Sending a record unchanged stores it under that record. Sending a modified output creates a new record pointing back at the original, because records are an audit trail: they never change under the data that cites them, so what your database holds is always traceable to a record containing exactly those values. And validation uses the contract as it is today — if the schema moved on since the record was produced, the History page flags it before you send.

The History page also shows, per record, whether it reached the database: sent, partially sent, or refused with the reason. Available from the web app, the API, MCP, n8n and Make.

Delivery, acknowledgement and purge

The feed is a strict FIFO queue per database: fetch a window (optionally leased, so a crashed worker’s batch is re-delivered before anything newer), apply, acknowledge. Webhook notifications are debounced — each new delta resets a quiet-period timer so a burst of enrichments is announced once, a configurable maximum delay caps the wait, and a full fetch page fires immediately. Two purge options control what Entity Enricher keeps: delete delivered delta copies on acknowledgement, and — for data minimization — delete the entity state itself once every database linked to the schema received it. Each takes an optional delay in days: delivered copies then linger that long after the acknowledgement (a replay window), and a delivered entity is kept until it has gone that long without an update, an hourly purge deleting what expired. Note that state purge is minimization, not erasure: enrichment records remain until you delete them, and it disables cross-enrichment merging for the purged entities.

A per-table checksum endpoint lets you verify at any time that your replica converged, without re-downloading anything.

The Database Sync page

Everything lives in one place in the app — Database Sync, right below History in the sidebar: register a database on any schema (with the database-key review), link or unlink schemas, pause a linked schema’s enrichment feed with its switch (no new data or notifications until re-enabled — schema publications still ship their DDL, and enrichments run while paused only reach the replica through a snapshot re-pull), edit its options, view the webhook endpoint and reveal its signing key, download the snapshot, browse the current entity state, inspect the pending delta queue (read-only — your workflow’s cursor is never touched), and view an entity-relationship diagram of the generated tables with their keys and junctions. On a multi-database sync the diagram can focus on one schema: tables, columns and links fed by the other schemas gray out — still visible in place — so you see exactly what each schema contributes to the shared tables.

  1. 1Browse the entity state, or the pending queue read-only
  2. 2One switch per linked schema, to pause its feed
  3. 3Per-table checksums, without re-downloading anything
The grey line under the counters is the part of the registration that is settled for good: dialect, primary-key strategy and key language.

Sync hosts

When several databases land on the same machine, the Sync hosts toolbar button removes the per-database pairing ceremony: pair that machine once, then assign registrations to it. The host claims each one, creates the physical database if it does not exist, and starts syncing — so registering a database becomes a decision you make here, not a terminal session on the server. Pairing is per Entity Enricher server, so one machine can serve several instances side by side.

Quarantine

A statement your database refuses — pre-existing duplicates under a new unique index are the usual cause — does not stall the queue behind it. That enrichment's batch is quarantined, the feed keeps flowing, and the batch is listed in the Quarantine tab with the statement your database rejected. Fix the cause and reinject — which re-projects the entity from its current state rather than replaying a stale statement — or drop it.

Automate it

Cloud-managed PostgreSQL: your database can live anywhere

Using Supabase? Our Supabase MCP comparison shows how EE's relationship and sync rules protect a product catalog, with JSON and small table diagrams.

Nothing in the sync ever runs on your database server — every path above is an outbound consumer that connects to whatever DSN you give it. Azure Database for PostgreSQL, OVHcloud, AWS RDS, Supabase or any other managed instance works exactly like a self-hosted one: point the consumer at the cloud DSN (managed providers usually enforce TLS, so add sslmode=require) and apply.

ENTITY ENRICHERYOUR INFRASTRUCTURE · CLOUDdelta feed · webhookSQL over TLSack · cursor advancesEntity Enricherdelta outbox · FIFOee-databaseany host or containern8n workflowFetch → Postgres → AckServerless functionAzure Functions · LambdaManaged PostgreSQLAzure · OVH · AWS RDS
Pick one consumer pathApply over TLS, then acknowledge

Two rules keep any hand-rolled consumer safe: execute each batch’s statements in order, inside one transaction, and acknowledge only after the commit. Deltas are idempotent and revision-guarded, so a crash before the acknowledgement simply means the batch is re-delivered and re-applying converges.

Availability

Databases are available on paid plans (the plan sets how many you can register). PostgreSQL is the launch dialect; each database declares its dialect, with MySQL / MariaDB, SQL Server and Oracle planned.