Skip to content

Commit 11b2c67

Browse files
committed
docs(user-config): document git incremental, Hub Bolt, operator purge API
Made-with: Cursor
1 parent c382661 commit 11b2c67

1 file changed

Lines changed: 22 additions & 0 deletions

File tree

scripts/user-config/README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,28 @@ All failures are collected with their source and reason into the `IngestionResul
3434
- Printed in the **INGESTION COMPLETE** banner (so you can see what failed and why at a glance)
3535
- Returned as JSON from `POST /api/v1/data/load-references` for programmatic inspection
3636

37+
## Git incremental ingestion (optional)
38+
39+
You can ingest **only files that changed** in a git work tree since the last run, instead of scanning the whole directory every time.
40+
41+
- In your profile YAML, set **`guide.git-ingestion.enabled: true`** and **`guide.git-ingestion.state-file`** to a JSON path (often under **`scripts/user-config/`** so it stays local and gitignored if you prefer).
42+
- The runner records the **git HEAD** per configured **`guide.directories`** entry. If HEAD is unchanged, that directory is skipped on the next startup ingest.
43+
- To **force a full re-ingest** of one directory (for example after a bad partial run), remove that directory’s entry from the state file, or call **`POST /api/v1/data/git-ingestion/revision/reset`** with a JSON body **`{ "directory": "~/path/to/repo" }`** while Guide is running (same path style as in YAML). See **`scripts/README.md`** for curl examples and security notes.
44+
45+
## Shared Neo4j (Embabel Hub or custom Bolt)
46+
47+
Ingestion scripts can target **compose Neo4j** (default) or **another Bolt endpoint** (Hub, remote, etc.):
48+
49+
- **`append-ingest.sh`** is for **adding** content; use **`USE_EMBABEL_HUB_NEO4J=1`** for the common Hub layout (Bolt on **27687**, credentials preset in the script header). Override **`NEO4J_BOLT_PORT`**, **`NEO4J_PASSWORD_HUB`**, etc. if your setup differs.
50+
- **`fresh-ingest.sh`** **wipes** `ContentElement` data in the connected database. Do **not** point it at a **shared Hub** or production graph.
51+
- For MCP against the **same** Bolt DB as ingest, you can use **`scripts/run-mcp-guide-against-hub.sh`** (see **`scripts/README.md`**).
52+
53+
Embabel Hub and the default Guide stack both use **384-dimensional** local ONNX embeddings (`all-MiniLM-L6-v2`) for vector indexes; mixing embedding models against one graph requires a deliberate re-index / re-ingest strategy (see operator docs in **`scripts/README.md`**).
54+
55+
## Operator API (purge and resync)
56+
57+
With Guide running, you can **preview or delete** ingested chunks by **URI prefix** or by **directory** (resolved to a `file:` prefix), and reset **git** revision state for one directory. Endpoints are **`POST /api/v1/data/content-elements/purge-preview`**, **`.../purge`**, and **`.../git-ingestion/revision/reset`**. Full table, examples, and safety notes (**prefix length**, **`confirm: true`**) are in **`scripts/README.md`** under **Operator API (shared Neo4j)**.
58+
3759
## MCP tools
3860

3961
All ingested content -- both URLs and local directories -- is immediately available through the MCP tools (`docs_vectorSearch`, `docs_textSearch`, etc.). The MCP tools and the ingestion pipeline share the same Neo4j store, so there is no separate sync step. Once ingestion completes, MCP clients (Cursor, Claude Desktop, etc.) can search the content right away.

0 commit comments

Comments
 (0)