The Semantic IDs page
Every enrichment that resolves a semantic ID either reuses a concept your organization already knows or mints a new one. The Semantic IDs page is where that growing vocabulary becomes something you can look at: browse it, measure how close two entries really are, add terms by hand, hunt the near-duplicates that slipped below the threshold, retire what you no longer want, and move the whole thing to a different embedding model.
Opening the page
It lives at /semantic-ids in the sidebar, and it is only useful once your organization has an embedding model and at least one schema carrying a semantic ID — concepts are created by enrichments, not by the page.
Who can do what
- •Editor — browse, compare, export, check a text, add a concept, delete concepts.
- •Owner / admin — also create concepts through an import, clear whole concept types, and migrate the embedding model.
Reading the concept table
Each row is one concept: its canonical text (the identity text that first created it), its concept type (the space it lives in — the entity type name by default), how many records use it, and when it was created. Filter by text, by any number of concept types, or by a minimum usage to find the entries worth your attention; Export view downloads exactly what the filters are showing.
Similarity is measured inside one space only. Vectors are comparable only within the same concept type and embedding model. Rows outside the selected concept’s space show —, which means “not comparable” — never “0%”.
Adding terms: the curate loop
Sometimes you know the vocabulary before the data arrives — a list of dishes, statuses, product families. The Curate bar is built for typing that list in: scope it to a concept type once — the scope goes into the page address, so /semantic-ids/<concept type> is a link straight to that slice — then repeat type a value → Check → Add. Enter checks, a second Enter adds, and the field clears and keeps focus, so a fifty-term vocabulary is a few minutes of typing without touching the mouse.
Check is a dry run of the real resolution ladder — the same one an enrichment runs — so its verdict is not an estimate. And because you have already paid for that verdict, it doubles as the duplicate guard: when an existing concept covers your text at or above the threshold, Add concept stays disabled.
That refusal is deliberate: a twin above the threshold could never win a resolution, and would split future matches unpredictably between the two entries. The Threshold slider next to the field decides where that line sits for your checks — raise it to be stricter, lower it to merge more aggressively.
The vocabulary does not have to start from an enrichment either: New concept type… (in the page toolbar, editor and up) names a type and picks the embedding model its concepts will live in — by default your organization’s model. The curate bar is scoped to it right away, and the type joins the vocabulary with the first concept you add; a type that already exists keeps its model, since moving a vocabulary between models is what the migration is for.
Hunting duplicates below the threshold
Anything above the threshold has already been merged for you. The interesting cases are just below it — close enough to be suspicious, not close enough to have collapsed. The Duplicates view lists exactly that band, and the slider widens or narrows it.
Compare → jumps back to the table with that concept selected, so you can see what else sits near it before deciding. A pair you consider a true duplicate is a signal about the threshold on the property that produces it: lower it in the schema editor and future enrichments will collapse the pair on their own. Usage counts tell you which of the two the data actually prefers.
Inspecting one concept
The right-hand panel shows the concept’s ID (copyable — it is what your database joins on), its normalized text, the embedding model behind it, and the records that resolved to it. The selected concept is part of the page address, so the URL in your address bar is a link straight back to it — shareable with a colleague, or worth keeping in a ticket. Two views place it among its neighbours.
Why position isn’t truth
Flattening 1536 dimensions into 3 cannot preserve distances, and the layout exaggerates how tight clusters are. Both views paint every point from the same similarity scale, so read the colour, not the gap. The first map of a session takes a few seconds to lay out; later ones are instant.
Linked records
Each linked record shows the score at which it resolved here. A — means the ID arrived in the input and was passed through, so nothing was ever compared — the free, unambiguous path described in the semantic-ID guide.
Importing and exporting a vocabulary
Import & resolve runs a whole CSV column through the same ladder and annotates every row with what would happen to it — there is no file-size limit; large files resolve in batches of 1000 with live progress. The file is parsed in your browser — only the values themselves are sent.
| Outcome | What it means |
|---|---|
exact | The same text, normalized, already exists — free, no model call. |
matched | A different wording resolved to an existing concept above the threshold. |
would_mint | Nothing was close enough; an enrichment would create a new concept here. |
minted | The same case, with creation switched on — the concept now exists (owner only). |
The resolved rows download as their own CSV, so an import can also be used purely as an audit: which of our 900 supplier names are already known, and which would open a new identity? Exports work the other way and name the file after the filters they were taken with, so a folder of them stays self-describing.
Deleting concepts
Deleting is safe in a way most data deletions are not: the vocabulary rebuilds itself, because the next enrichment simply mints what it needs again. What does not come back is convergence with the IDs you already stored, and the dialog says so with real counts before you confirm.
- •Delete N unused — the shortcut in the filter bar clears concepts no record points to. Nothing downstream can notice.
- •Delete N concepts (owner) — wipes every concept of the types the filter selects, one type or several at once. This is the maintenance action after you change what an identity is made of — adding, removing or swapping the key fields behind a semantic ID. A concept type is only the label its concepts carry, so an emptied type leaves the list on its own.
After such a change, keeping the old concepts is the risky choice, not the cautious one: identities composed from the new keys can still land within the threshold of the old vectors and be quietly absorbed by them, leaving you with IDs that mean neither thing.
Changing the embedding model
Vectors from two different models are not comparable, so switching model means re-embedding every concept. Once concepts exist, this migration is the only sanctioned way to do it — and it is the reason the organization’s embedding-model setting refuses to change on its own.
Dry run first
The preview reports what the re-embedding will cost, and which concept pairs are likely to collide — land within the threshold of each other in the new space and start resolving together. It measures the realistic candidates rather than every pair, and says so.
No pause, no window to babysit
Enrichments keep resolving in the old space while the new vectors are built beside it; concepts minted meanwhile are picked up by a later pass. The switch happens in a single step at the end, which also flips your organization’s default embedding model. Interrupting it is harmless — starting again continues where it stopped.
What the page costs
Checks, adds and imports are billed like any other embedding usage, and exact hits cost nothing because they never reach a model. Browsing, comparing, the orbit, the 3D map, exporting and deleting are free. A day’s worth of interactive spend is folded into a single line on your credit history, so the ledger stays readable instead of filling up with fractions of a cent.