Skip to content

feat: ontology-gap detection — recommend new entity TYPES from extraction misfits + eval gaps #77

Description

@rajnavakoti

Description

Extraction is currently ontology-constrained: when the LLM hits a concept that doesn't fit, it marks it status: proposed and suggests an existing type, and flags new jargon — but it does not recommend a brand-new entity type. Evals surface content gaps (MISSING/INCOMPLETE) but not type gaps. So enriching the ontology from demand is today a semi-manual loop: a human runs an LLM over the proposed-flags + eval gaps to spot missing types.

This issue: make CB recommend new entity types/relationships natively — analyze the proposed/misfit flags + eval coverage gaps and surface "you likely need a new type X / relationship Y" for human review. Part of the bidirectional write-back vision (agent/pipeline proposes → human curates). Related: #19 (MCP write server), #75.

Why (and why it's downstream, not upstream)

The DDC → ontology-enrichment story (e.g. "DDC surfaced the ADR-compliance link that NORIA-O's top-down ontology missed") works semi-manually today and is sufficient for the experiment. Build this AFTER running the manual loop a few times — the real type-gaps that emerge tell you what the feature should recommend. Demand-driven tooling: let the need define the feature.

Acceptance Criteria

  • Aggregate extraction proposed/misfit signals + eval MISSING/INCOMPLETE gaps
  • LLM analysis recommends candidate new entity types and/or relationship fields (with rationale + example instances)
  • Recommendations surfaced for human review (never auto-applied to the ontology)
  • Output is actionable: proposed type name, layer, directory, example entities
  • Tests

Out of Scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    frameworkScientific frameworks

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions