Versioning, Evaluation & Health

Semantic Model Versioning

Version the entire semantic model — ontology, shapes, bindings, policies — as a single coherent snapshot. Promote through environments: dev → staging → production. Every version is diffable, comparable, and rollback-ready. No more "what changed and when did it break."

Diff, Compare & Rollback

Side-by-side comparison of any two semantic model versions. See what topics were added, which shapes changed, which bindings were rewired. If a promotion causes issues, roll back to the prior version with one action — the entire model reverts atomically.

Evaluation Datasets & Scorecards

Build evaluation datasets — curated question-answer pairs with known-correct evidence chains. Run them against the Brain and score: retrieval precision (did it find the right content?), faithfulness (did it stay true to the evidence?), groundedness (can every claim be traced to a source?). Scorecards track quality over time.

Consumption Trace Explorer

Trace any Brain decision end to end: the query → which retrieval path resolved → which policy gates fired → which subgraphs were assembled → what confidence score was produced → what evidence was cited. Every production decision is inspectable after the fact.

Coverage & Gap Analysis

Which concepts in the ontology have no semantic bindings? Which topics have thin connectivity? Which queries frequently trigger abstention? Coverage analysis reveals the holes in the Brain's knowledge — each gap becomes a prioritized work item feeding back to Phases 2 and 3.

Drift Detection & Re-Attestation

Source data changes. Business rules evolve. Regulations update. Drift detection monitors the gap between what the Brain knows and what's true today. When drift exceeds a threshold, the affected bindings, policies, and workflows are flagged for re-attestation — a governed re-review, not a silent degradation.