feat: add verification fixtures and graph.json schema validator

[Ujikintoki]

Summary

Add synthetic test data and tools for verifying the Reafiner harness (Level 0 trace validation + Level 1.5 evidence audit) without requiring LLM, graphify, or any agent platform.

What's Included

File	Purpose
fixtures/synth-minimal.json	10-node synthetic KG conforming to graphify schema
fixtures/trace-valid-early-exit.json	1-step trace for early exit path
fixtures/trace-valid.json	2-step trace for refinement path
fixtures/trace-invalid-order.json	Malformed trace for negative testing
fixtures/refinement-good.txt	Valid refinement → MEDIUM confidence
fixtures/refinement-ambiguous.txt	Fuzzy node name → LOW confidence
fixtures/refinement-invalid.txt	No <refinement> block → parse error
scripts/validate_graph_schema.py	Reusable schema conformance checker
docs/verification-data.md	Design rationale + quick-start commands

How to Verify

No dependencies beyond deeprefine CLI (already installed with pip install -e .):

cd DeepRefine-Skill

# Schema check
python scripts/validate_graph_schema.py fixtures/synth-minimal.json

# Trace validation
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-valid-early-exit.json
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-valid.json --refinement-file fixtures/refinement-good.txt
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-invalid-order.json

# Review + Apply (needs temp KB project)
mkdir -p /tmp/test_kb/graphify-out
cp fixtures/synth-minimal.json /tmp/test_kb/graphify-out/graph.json
python -m deeprefine_skill.cli review --refinement-file fixtures/refinement-good.txt --trace-file fixtures/trace-valid.json --project-root /tmp/test_kb
python -m deeprefine_skill.cli apply --refinement-file fixtures/refinement-good.txt --trace-file fixtures/trace-valid.json --project-root /tmp/test_kb

All 11 verification steps pass. See docs/verification-data.md for detailed expected outputs.

What This Is NOT

- Not a pytest test suite
- Not new features or functional changes to the main package
- Does not require graphify, LLM API keys, or any agent platform

[Ujikintoki]

This is a pure python scripts to verify harness of the reafiner using mock llm outputs.

[Ujikintoki]

Still working...

[Ujikintoki]

What

Add a verification harness (fixtures + schema validator + docs) that lets anyone confirm the Python CLI and Reafiner loop work correctly in 2 minutes, no graphify required.

12 files, +1041 lines. 3 commits squashed over the past week.

Why

Before: test the harness → install graphify → configure API key → generate KG → run CLI blind. No fixtures, no schema checks, no documented procedure.

After: pip install -e . → 7 commands → exit code tells you pass/fail.

What's Included

Thing	Files	Tests What
Synthetic KG	fixtures/synth-minimal.json	10-node KG covering both Reafiner paths (early exit + refinement)
Trace fixtures ×3	trace-valid.json, trace-valid-early-exit.json, trace-invalid-order.json	loop validate — refinement path, early exit, malformed trace
Refinement fixtures ×3	refinement-good.txt, refinement-ambiguous.txt, refinement-invalid.txt	review/apply gates — happy path, LOW rejection, parse error
Schema validator	scripts/validate_graph_schema.py	Any graph.json — keys, fields, confidence enums, referential integrity
Docs ×3	docs/verification-data.md, docs/phase-b-report.md, docs/feat-verification-fixtures.md	Design doc, real KG findings, quick-start for collaborators

Each Fixture → Python Gate

Every fixture maps to a hard-coded rule in validate_trace() or review_action():

Fixture	Gate
trace-valid.json	Refinement path: abduction present, refinement present, step order, triple cap
trace-valid-early-exit.json	Early exit: no abduction, no refinement, early_exit: true
trace-invalid-order.json	Negative: step-order violation caught
refinement-good.txt	Happy path: nodes exist → MEDIUM; apply succeeds
refinement-ambiguous.txt	Ambiguity detection → LOW; apply rejects
refinement-invalid.txt	No <refinement> block → parse failure

Quick Start (2 min)

pip install -e .

# Schema
python scripts/validate_graph_schema.py fixtures/synth-minimal.json

# Trace validation
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-valid-early-exit.json
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-valid.json --refinement-file fixtures/refinement-good.txt
python -m deeprefine_skill.cli loop validate --trace-file fixtures/trace-invalid-order.json  # should fail

# Review + Apply gates
mkdir -p /tmp/test_kb/graphify-out
cp fixtures/synth-minimal.json /tmp/test_kb/graphify-out/graph.json
python -m deeprefine_skill.cli review --refinement-file fixtures/refinement-good.txt --trace-file fixtures/trace-valid.json --project-root /tmp/test_kb
python -m deeprefine_skill.cli apply --refinement-file fixtures/refinement-good.txt --trace-file fixtures/trace-valid.json --project-root /tmp/test_kb
python -m deeprefine_skill.cli apply --refinement-file fixtures/refinement-ambiguous.txt --trace-file fixtures/trace-valid.json --project-root /tmp/test_kb  # should fail
rm -rf /tmp/test_kb

Verification Summary

- Phase A — All CLI commands verified against synthetic 10-node KG
- Phase B — Validated against graphifyy 0.8.42 (361n/646e); found & fixed 3 schema differences
- Phase C — End-to-end Copilot CLI run; 4 Reafiner paths confirmed: refinement, early exit, apply, LOW gate