Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,31 @@ uv run hyrule-knowledge observe --profile safe-health

`.github/workflows/ingest.yml` runs nightly and on manual dispatch. When source knowledge changes, it opens a refresh PR. The PR must be reviewed before merge; generated knowledge is never auto-merged.

## Knowledge Loop agent

`hyrule-knowledge loop --once` is the governed producer loop for the knowledge repo. It is separate from the read-only Knowledge MCP server: the MCP serves context; the loop refreshes, validates, enriches/imports when explicitly enabled, and opens reviewable PRs.

Phase 1 is deterministic by default:

```bash
uv run hyrule-knowledge loop --once --dry-run
```

It acquires a singleton lock, updates a per-day ledger, runs ingest plus the standard validation/export/eval/ledger/secret-scan gates, and reports JSON. Set `--create-pr` or `HYRULE_KNOWLEDGE_LOOP_CREATE_PR=1` to push a branch and open a PR when changes exist.

Phase 2 is opt-in producer work:

```bash
uv run hyrule-knowledge loop --once \
--create-pr \
--enrich-target infrastructure \
--enrich-live \
--max-openrouter-calls-per-day 1 \
--learning-event /var/lib/engineering-loop/learning-events
```

Live enrichment requires the Knowledge Loop credential scope (`OPENROUTER_API_KEY` rendered from `kv/knowledge-loop`) and an explicit daily OpenRouter call budget. Learning-event inputs may be files, directories, or globs; imported artifacts remain review-gated ledger proposals.

## Governed agent consumption

Humans can browse `okf/index.md`. Agents should first read `okf/index.md`, then directory indexes, then individual concepts. Machine consumers should prefer `exports/knowledge.sqlite` and the JSONL exports:
Expand Down
69 changes: 69 additions & 0 deletions docs/plans/2026-06-23-knowledge-loop-agent-phases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Knowledge Loop Agent — Phases 1 and 2

## Intent

Turn `AS215932/knowledge` from a manual/tooling repository plus read-only MCP server into a governed autonomous Knowledge Loop agent, comparable in shape to Engineering Loop and NOC Agent.

The read-only Knowledge MCP remains a serving dependency for Engineering Loop. The Knowledge Loop is a separate producer agent that refreshes, validates, enriches, imports learning artifacts, and opens reviewable PRs/issues. It must never auto-merge and must not promote generated synthesis to canonical truth without human review.

## Runtime boundaries

- Run on `loop` as a dedicated unprivileged `knowledge-loop` runtime.
- Use a systemd one-shot service plus timer.
- Use Vault-backed runtime credentials scoped to Knowledge Loop only.
- Do not mount fleet SSH credentials, Docker socket, app runtime credentials, wallet data, or broad Vault access.
- Do not write directly to `main`.
- All repository mutation exits as a PR.
- All generated synthesis remains advisory unless reviewed/promoted by humans.

## Phase 1 — deterministic loop skeleton

Build `hyrule-knowledge loop --once` with:

1. Singleton state lock under the Knowledge Loop state directory.
2. Per-day ledger with cycle and PR counters.
3. Deterministic refresh and validation commands:
- `hyrule-knowledge ingest`
- `ruff check src tests`
- `mypy --strict src`
- `pytest`
- `hyrule-knowledge validate okf`
- `hyrule-knowledge quality --check`
- `hyrule-knowledge export --check`
- `hyrule-knowledge eval --check`
- `hyrule-knowledge ledger --check`
- `hyrule-knowledge ledger lifecycle --check`
- `hyrule-knowledge scan-secrets okf exports reports evals ledger schema`
4. Optional dry-run enrichment plumbing for proving source-pack/writer paths without provider calls.
5. Git branch/commit/push/PR creation when there are changes.
6. JSON report output for every cycle.
7. Optional passive Icinga heartbeat.
8. Tests for lock, budget, command sequencing, PR gating, and heartbeat behavior.

## Phase 2 — Knowledge production work

Extend the loop with opt-in bounded producer work:

1. OpenRouter LLM enrichment targets using the Knowledge Loop credential scope.
2. Learning-event import from sanitized artifact directories.
3. Budget gates for provider calls and imported artifacts.
4. Validation/export/eval/secret-scan after phase-2 mutation.
5. PR output for enrichment/import changes.
6. Tests that prove phase-2 command sequencing and budget behavior without live provider calls.

## Deployment follow-up

Add NetOps source-managed runtime:

- `knowledge_loop` Ansible role.
- Vault Agent template for `/etc/knowledge-loop/knowledge-loop.env`.
- systemd service/timer.
- host_vars on `loop` for disabled-by-default canary rollout.
- monitoring passive check for loop freshness/status.

## Human gates

- Humans review Knowledge Loop PRs.
- Humans merge Knowledge PRs.
- Humans decide curated A1 promotion.
- Deployment of updated Knowledge MCP remains a NetOps pin-bump/apply path.
87 changes: 87 additions & 0 deletions src/hyrule_knowledge/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@
from .evals import eval_check, load_eval_cases, write_eval_reports
from .exporter import exports_match, write_exports
from .github_source import collect_snapshot
from .knowledge_loop import KnowledgeLoopConfig
from .knowledge_loop import run_once as run_knowledge_loop_once
from .learning_ledger import (
LearningLedgerError,
import_learning_events,
Expand Down Expand Up @@ -767,6 +769,69 @@ def cmd_eval(args: argparse.Namespace) -> int:
return 0


def _truthy(value: str | None) -> bool:
return value is not None and value.strip().lower() in {"1", "true", "yes", "on"}


def _env_paths(name: str) -> list[Path]:
value = os.environ.get(name, "").strip()
if not value:
return []
return [Path(item) for item in value.split(os.pathsep) if item]


def _env_csv(name: str) -> list[str]:
value = os.environ.get(name, "").strip()
if not value:
return []
return [item.strip() for item in value.split(",") if item.strip()]


def _env_int(name: str, default: int) -> int:
value = os.environ.get(name, "").strip()
if not value:
return default
try:
return int(value)
except ValueError:
return default


def _env_str(name: str, default: str) -> str:
value = os.environ.get(name, "").strip()
return value or default


def cmd_loop(args: argparse.Namespace) -> int:
if not args.once:
print("knowledge loop currently requires --once", file=sys.stderr)
return 1
config = KnowledgeLoopConfig(
repo_path=Path(args.repo_path),
config_path=Path(args.config),
state_dir=Path((args.state_dir or "").strip() or _env_str("HYRULE_KNOWLEDGE_LOOP_STATE_DIR", ".cache/hyrule-knowledge/loop-state")),
branch_prefix=args.branch_prefix,
base_branch=args.base_branch,
create_pr=bool(args.create_pr or _truthy(os.environ.get("HYRULE_KNOWLEDGE_LOOP_CREATE_PR"))),
dry_run=bool(args.dry_run),
git_user_name=args.git_user_name,
git_user_email=args.git_user_email,
max_cycles_per_day=args.max_cycles_per_day,
max_prs_per_day=args.max_prs_per_day,
max_openrouter_calls_per_day=args.max_openrouter_calls_per_day,
max_learning_events_per_day=args.max_learning_events_per_day,
enrich_targets=tuple(args.enrich_target or _env_csv("HYRULE_KNOWLEDGE_LOOP_ENRICH_TARGETS")),
enrich_live=bool(args.enrich_live or _truthy(os.environ.get("HYRULE_KNOWLEDGE_LOOP_ENRICH_LIVE"))),
dry_run_enrich_targets=tuple(args.dry_run_enrich_target or _env_csv("HYRULE_KNOWLEDGE_LOOP_DRY_RUN_ENRICH_TARGETS")),
learning_event_paths=tuple([*(Path(path) for path in (args.learning_event or [])), *_env_paths("HYRULE_KNOWLEDGE_LOOP_LEARNING_EVENTS")]),
replace_learning_events=bool(args.replace_learning_events),
run_validation=not args.skip_validation,
)
report = run_knowledge_loop_once(config)
print_json(report.as_json())
return 0 if report.outcome in {"published", "changes_detected", "idle", "locked"} else 1


def cmd_mcp(args: argparse.Namespace) -> int:
from .mcp_server import main as mcp_main

Expand Down Expand Up @@ -929,6 +994,28 @@ def build_parser() -> argparse.ArgumentParser:
ledger.add_argument("--replace", action="store_true", help="replace existing proposed event on ledger import")
ledger.set_defaults(func=cmd_ledger)

loop = subparsers.add_parser("loop")
loop.add_argument("--once", action="store_true", help="run exactly one Knowledge Loop cycle")
loop.add_argument("--repo-path", default=".")
loop.add_argument("--state-dir")
loop.add_argument("--branch-prefix", default="bot/knowledge-loop")
loop.add_argument("--base-branch", default="main")
loop.add_argument("--create-pr", action="store_true", help="push a branch and open a PR when changes are detected")
loop.add_argument("--dry-run", action="store_true", help="run the cycle but do not commit/push/open PR")
loop.add_argument("--skip-validation", action="store_true")
loop.add_argument("--git-user-name", default=os.environ.get("HYRULE_KNOWLEDGE_LOOP_GIT_USER_NAME", "hyrule-knowledge-loop[bot]"))
loop.add_argument("--git-user-email", default=os.environ.get("HYRULE_KNOWLEDGE_LOOP_GIT_USER_EMAIL", "knowledge-loop@as215932.net"))
loop.add_argument("--max-cycles-per-day", type=int, default=_env_int("HYRULE_KNOWLEDGE_LOOP_MAX_CYCLES_PER_DAY", 1))
loop.add_argument("--max-prs-per-day", type=int, default=_env_int("HYRULE_KNOWLEDGE_LOOP_MAX_PRS_PER_DAY", 1))
loop.add_argument("--dry-run-enrich-target", action="append", help="phase-1 enrichment plumbing target; never calls a provider")
loop.add_argument("--enrich-target", action="append", help="phase-2 enrichment target; dry-run unless --enrich-live is set")
loop.add_argument("--enrich-live", action="store_true", help="allow live OpenRouter enrichment for --enrich-target")
loop.add_argument("--max-openrouter-calls-per-day", type=int, default=_env_int("HYRULE_KNOWLEDGE_LOOP_MAX_OPENROUTER_CALLS_PER_DAY", 0))
loop.add_argument("--learning-event", action="append", help="phase-2 learning-event file, directory, or glob to import")
loop.add_argument("--replace-learning-events", action="store_true")
loop.add_argument("--max-learning-events-per-day", type=int, default=_env_int("HYRULE_KNOWLEDGE_LOOP_MAX_LEARNING_EVENTS_PER_DAY", 100))
loop.set_defaults(func=cmd_loop)

mcp = subparsers.add_parser("mcp")
mcp.add_argument("--transport", default="stdio", choices=["stdio", "sse", "streamable-http", "http"])
mcp.add_argument("--db", help="SQLite export path (default: configured exports/knowledge.sqlite)")
Expand Down
Loading