feat(rag): expose knowledge improvement lifecycle

README.md

@@ -32,6 +32,8 @@ Two ways in, depending on what you're doing:
   - *"Grow the KB as a researcher"* → [`runKnowledgeResearchLoop`](#research-loop) (deterministic mechanics; your agent owns judgment), [`runTwoAgentResearchLoop`](#two-agent-research-loop) (researcher proposes, verifier checks + fills gaps, offline), or the sandbox [researcher profile](#researcher-profile) for `runLoop`.
   - *"Spawn one researcher per sub-topic and stop when the KB is ready"* → [`runResearchSupervisor`](#research-supervisor) (a supervisor brain sizes the topology over a `Scope`; LIVE, needs creds).
   - *"Tune retrieval for a knowledge base"* → `runRetrievalImprovementLoop` in the [Agent-Eval integration](#agent-eval-integration) section.
+  - *"Improve the whole RAG knowledge base"* → `runRagKnowledgeImprovementLoop` in the [Agent-Eval integration](#agent-eval-integration) section.
+    It exposes retrieval tuning, gap diagnosis, knowledge acquisition/update, answer-quality checks, and promotion as one typed lifecycle.
   - *"Does this candidate KB actually improve task success?"* → run an [agent-eval improvement loop](#agent-eval-integration) over KB variants, then `knowledgeReleaseReport` for the promotion decision.
   - *"Keep live authorities fresh"* → [pluggable sources](#pluggable-knowledge-sources) + `detectChanges` → eval re-runs.
 
@@ -92,6 +94,11 @@ from `@tangle-network/agent-knowledge`.
 - `createKnowledgeControlLoopAdapter()` maps those mechanics into
   `agent-eval`'s `runAgentControlLoop()` so products can plug in their own
   proposer, reviewer, and driver policies.
+- `runRagKnowledgeImprovementLoop()` coordinates the whole RAG improvement
+  lifecycle. Retrieval tuning, diagnosis, acquisition, KB update,
+  answer-quality eval, and promotion are separate typed phases so products can
+  plug in browser agents, coding agents, connectors, or deterministic policies
+  without this package hardcoding an agent runner.
 - Zod schemas define the stable wire shape.
 - Graph/search/lint are deterministic and fast.
 - `searchKnowledge` returns hits with three score fields. `score` and
@@ -116,6 +123,38 @@ readiness/eval machinery without making `agent-knowledge` own the database.
 
 ## Agent-Eval Integration
 
+Use `runRagKnowledgeImprovementLoop` when the product question is broader than retrieval:
+can the system find the gaps, gather or update knowledge, prove generated answers still behave, and decide whether to promote?
+`agent-knowledge` owns the knowledge/eval contract; the caller supplies the research, coding, connector, and answer-eval hooks.
+
+```ts
+import { runRagKnowledgeImprovementLoop } from '@tangle-network/agent-knowledge'
+
+const result = await runRagKnowledgeImprovementLoop({
+  goal: 'Improve the support RAG KB',
+  retrieval: {
+    baseline: { k: 5, hybrid: false },
+    scenarios: trainRetrievalScenarios,
+    holdoutScenarios,
+    index,
+    searchSpace: { k: [5, 10, 20], hybrid: [false, true] },
+    targetRecall: 0.9,
+  },
+  diagnose: async ({ retrieval }) => diagnoseRagGaps(retrieval),
+  acquireKnowledge: async ({ findings }) => researchMissingSources(findings),
+  knowledgeResearch: { root: './kb' },
+  evaluateAnswers: async ({ knowledgeUpdate }) => runAnswerEval(knowledgeUpdate),
+  promote: async ({ retrieval, answerQuality }) =>
+    decidePromotion({ retrieval, answerQuality }),
+  requiredPhases: ['retrieval-tuning', 'knowledge-update', 'answer-quality', 'promotion'],
+})
+
+console.log(result.promotion)
+```
+
+If a required phase is missing its hook, the loop throws.
+That keeps the public API from reporting a fake “RAG improved” result when the caller only wired retrieval or only wired a researcher.
+
 Use retrieval eval when the question is whether a retrieval/RAG config can find the right knowledge before an agent reasons over it.
 The labels should name stable pages, source records, anchors, or source spans, not ephemeral chunk IDs.
 The completion roadmap is in [`docs/eval/rag-eval-roadmap.md`](docs/eval/rag-eval-roadmap.md).

docs/eval/rag-eval-roadmap.md

@@ -20,9 +20,12 @@ SOTA RAG evaluation requires retrieval quality, context quality, generated-answe
 Done:
 
 - `runRetrievalImprovementLoop()` auto-searches retrieval configs through `agent-eval`.
+- `runRagKnowledgeImprovementLoop()` exposes the whole RAG lifecycle as typed phases:
+  retrieval tuning, gap diagnosis, knowledge acquisition, knowledge update, answer-quality eval, and promotion.
 - Retrieval scenarios can label pages, page paths, sources, source anchors, and source spans.
 - The retrieval judge reports recall, MRR, nDCG, precision@k, cost, and held-out promotion.
 - The loop is tested with a real `agent-eval` run where `{ k: 2 }` beats `{ k: 1 }`.
+- The lifecycle loop is tested both with pluggable phase hooks and with a real local KB update through `runKnowledgeResearchLoop()`.
 
 Not done:
 
@@ -31,6 +34,8 @@ Not done:
 - Citation support and claim-level groundedness.
 - Abstention and unanswerable-question scoring.
 - Slice-level reporting for freshness, distractors, multi-hop, and long-tail cases.
+- Packaged runtime adapters for browser/coding/research agents.
+  The lifecycle API accepts those agents as hooks today; it does not hardcode provider-specific workers.
 
 ## Completion Criteria
 
@@ -101,6 +106,7 @@ Never tune on holdout.
 Ship criteria:
 
 - `runRetrievalImprovementLoop()` gates retrieval config changes.
+- `runRagKnowledgeImprovementLoop()` is the default front door when retrieval changes, source acquisition, KB updates, answer checks, and promotion must run together.
 - Answer-quality eval gates prompt and synthesis changes.
 - Reports persist run id, commit, config hash, dataset hash, metric versions, cost, latency, and traces.
 - A promoted candidate must improve the target metric without violating faithfulness, abstention, cost, or latency limits.
@@ -111,4 +117,5 @@ Ship criteria:
 2. Add context relevance and context sufficiency judges over retrieved hits.
 3. Add forbidden/stale source targets to retrieval scenarios.
 4. Add slice-level aggregation helpers for the required six eval slices.
-5. Add a CLI command that runs the retrieval loop and writes a reproducible report under `.agent-knowledge/eval/`.
+5. Add packaged adapters that turn `agent-runtime` browser, research, and coding agents into `runRagKnowledgeImprovementLoop()` hooks.
+6. Add a CLI command that runs the lifecycle loop and writes a reproducible report under `.agent-knowledge/eval/`.

src/autodata/powered.ts

@@ -438,11 +438,11 @@ async function main(): Promise<void> {
   printReport(stats, spendUsd)
 
   // Emit the machine-readable result alongside the prose, for the doc + any re-analysis.
-  console.log('RESULT_JSON ' + JSON.stringify({ ...stats, spendUsd }))
+  console.log(`RESULT_JSON ${JSON.stringify({ ...stats, spendUsd })}`)
 }
 
 // Only auto-run when invoked directly (keeps `analyzeTrails` importable + unit-testable).
-if (process.argv[1] && process.argv[1].endsWith('powered.ts')) {
+if (process.argv[1]?.endsWith('powered.ts')) {
   main().catch((err) => {
     console.error(err)
     process.exit(1)

src/index.ts

@@ -21,6 +21,7 @@ export * from './material-facts-metric'
 export * from './memory/index'
 export * from './proposals'
 export * from './propose-from-finding'
+export * from './rag-improvement-loop'
 export * from './release'
 export * from './research-driving-driver'
 export * from './research-loop'

src/lint.ts

@@ -141,8 +141,10 @@ function extractSourceRefs(text: string): Array<{ sourceId: string; anchorId?: s
   const refs: Array<{ sourceId: string; anchorId?: string }> = []
   const regex = /\[\^([A-Za-z0-9_-]+)(?:#([A-Za-z0-9_.:-]+))?\]/g
   let match: RegExpExecArray | null
-  while ((match = regex.exec(text)) !== null) {
+  match = regex.exec(text)
+  while (match !== null) {
     refs.push({ sourceId: match[1]!, anchorId: match[2] })
+    match = regex.exec(text)
   }
   return refs
 }