feat(rag): add answer and KB quality evals

README.md

@@ -34,6 +34,7 @@ Two ways in, depending on what you're doing:
   - *"Tune retrieval for a knowledge base"* → `runRetrievalImprovementLoop` in the [Agent-Eval integration](#agent-eval-integration) section.
   - *"Improve the whole RAG knowledge base"* → `runRagKnowledgeImprovementLoop` in the [Agent-Eval integration](#agent-eval-integration) section.
     It exposes retrieval tuning, gap diagnosis, knowledge acquisition/update, answer-quality checks, and promotion as one typed lifecycle.
+  - *"Evaluate RAG answers or a wiki/KB"* → `ragAnswerQualityJudge`, `createRagAnswerQualityHook`, and `scoreKnowledgeBaseIndex` in the [Agent-Eval integration](#agent-eval-integration) section.
   - *"Does this candidate KB actually improve task success?"* → run an [agent-eval improvement loop](#agent-eval-integration) over KB variants, then `knowledgeReleaseReport` for the promotion decision.
   - *"Keep live authorities fresh"* → [pluggable sources](#pluggable-knowledge-sources) + `detectChanges` → eval re-runs.
 
@@ -99,6 +100,10 @@ from `@tangle-network/agent-knowledge`.
   answer-quality eval, and promotion are separate typed phases so products can
   plug in browser agents, coding agents, connectors, or deterministic policies
   without this package hardcoding an agent runner.
+- RAG answer evaluation follows the common open-source shape used by Ragas,
+  DeepEval, TruLens, and RAGChecker: context quality, answer relevance,
+  support/faithfulness, citations, abstention, and failure diagnosis.
+  External tools stay pluggable via score normalization and row exporters.
 - Zod schemas define the stable wire shape.
 - Graph/search/lint are deterministic and fast.
 - `searchKnowledge` returns hits with three score fields. `score` and
@@ -123,6 +128,33 @@ readiness/eval machinery without making `agent-knowledge` own the database.
 
 ## Agent-Eval Integration
 
+Use `ragAnswerQualityJudge` or `createRagAnswerQualityHook` when the product already has answer traces and needs SOTA-style RAG scoring without rebuilding metrics.
+The built-in checks are deterministic and general; pass external scores from Ragas, DeepEval, TruLens, RAGChecker, or a custom evaluator when you want model-based judging.
+
+```ts
+import {
+  createRagAnswerQualityHook,
+  scoreKnowledgeBaseIndex,
+} from '@tangle-network/agent-knowledge'
+
+const evaluateAnswers = createRagAnswerQualityHook({
+  scenarios: answerScenarios,
+  run: async (scenario) => runRagAnswerTrace(scenario),
+  externalEvaluator: async ({ scenario, artifact }) => runRagasOrDeepEval({
+    input: scenario.query,
+    output: artifact.answer,
+    contexts: artifact.contexts,
+  }),
+})
+
+const answerQuality = await evaluateAnswers()
+const kbQuality = scoreKnowledgeBaseIndex(index, {
+  strict: true,
+  minCitationRate: 0.8,
+  maxStaleSourceRate: 0.02,
+})
+```
+
 Use `runRagKnowledgeImprovementLoop` when the product question is broader than retrieval:
 can the system find the gaps, gather or update knowledge, prove generated answers still behave, and decide whether to promote?
 `agent-knowledge` owns the knowledge/eval contract; the caller supplies the research, coding, connector, and answer-eval hooks.
@@ -143,7 +175,7 @@ const result = await runRagKnowledgeImprovementLoop({
   diagnose: async ({ retrieval }) => diagnoseRagGaps(retrieval),
   acquireKnowledge: async ({ findings }) => researchMissingSources(findings),
   knowledgeResearch: { root: './kb' },
-  evaluateAnswers: async ({ knowledgeUpdate }) => runAnswerEval(knowledgeUpdate),
+  evaluateAnswers,
   promote: async ({ retrieval, answerQuality }) =>
     decidePromotion({ retrieval, answerQuality }),
   requiredPhases: ['retrieval-tuning', 'knowledge-update', 'answer-quality', 'promotion'],

docs/eval/rag-eval-roadmap.md

@@ -26,13 +26,13 @@ Done:
 - The retrieval judge reports recall, MRR, nDCG, precision@k, cost, and held-out promotion.
 - The loop is tested with a real `agent-eval` run where `{ k: 2 }` beats `{ k: 1 }`.
 - The lifecycle loop is tested both with pluggable phase hooks and with a real local KB update through `runKnowledgeResearchLoop()`.
+- `ragAnswerQualityJudge()` and `createRagAnswerQualityHook()` score context precision/recall/relevance/sufficiency, faithfulness, answer relevance/correctness, citation support, abstention, and unsupported-answer rate.
+- `normalizeExternalRagScores()` and the row exporters make Ragas, DeepEval, TruLens, RAGChecker, and custom evaluator outputs pluggable instead of hard dependencies.
+- `scoreKnowledgeBaseIndex()` validates generic wiki/KB health: citation coverage, source-backed pages, stale sources, duplicate source hashes, and lint/validation errors.
+- `calibrateRagAnswerJudge()` enforces the strong-vs-weak metric check before trusting a RAG answer metric.
 
 Not done:
 
-- Generated-answer evaluation.
-- Context relevance and context sufficiency judges.
-- Citation support and claim-level groundedness.
-- Abstention and unanswerable-question scoring.
 - Slice-level reporting for freshness, distractors, multi-hop, and long-tail cases.
 - Packaged runtime adapters for browser/coding/research agents.
   The lifecycle API accepts those agents as hooks today; it does not hardcode provider-specific workers.
@@ -113,9 +113,7 @@ Ship criteria:
 
 ## Next Implementation Steps
 
-1. Add `RagAnswerEvalScenario`, `RagAnswerEvalArtifact`, and `ragAnswerQualityJudge()`.
-2. Add context relevance and context sufficiency judges over retrieved hits.
-3. Add forbidden/stale source targets to retrieval scenarios.
-4. Add slice-level aggregation helpers for the required six eval slices.
-5. Add packaged adapters that turn `agent-runtime` browser, research, and coding agents into `runRagKnowledgeImprovementLoop()` hooks.
-6. Add a CLI command that runs the lifecycle loop and writes a reproducible report under `.agent-knowledge/eval/`.
+1. Add slice-level aggregation helpers for freshness, distractors, multi-hop, long-tail, and unanswerable cases.
+2. Add forbidden/stale source targets to retrieval scenarios.
+3. Add packaged adapters that turn `agent-runtime` browser, research, and coding agents into `runRagKnowledgeImprovementLoop()` hooks.
+4. Add a CLI command that runs the lifecycle loop and writes a reproducible report under `.agent-knowledge/eval/`.

src/index.ts

@@ -21,6 +21,7 @@ export * from './material-facts-metric'
 export * from './memory/index'
 export * from './proposals'
 export * from './propose-from-finding'
+export * from './rag-eval'
 export * from './rag-improvement-loop'
 export * from './release'
 export * from './research-driving-driver'