feat(governance): LlamaIndex adapter#360
Open
aditik0303 wants to merge 7 commits into
Open
Conversation
Registers a BaseEventHandler on the root instrumentation dispatcher (LLMChatStartEvent -> BEFORE_MODEL, LLMChatEndEvent -> AFTER_MODEL, AgentToolCallEvent -> TOOL_CALL). Self-registers via the uipath.governance.adapters entry point; unit-tested and verified firing through the framework's real execution path. BEFORE/AFTER_AGENT remain owned by the uipath-runtime wrapper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…, framework-only can_handle) Mirror radu's LangChain-adapter review across the LlamaIndex adapter: - __init__: drop the import-time registration side-effect; registration only via the uipath.governance.adapters entry point. - can_handle: claim only a real workflows.Workflow; remove the duck-typed (run / Workflow-shaped name) fallback. - docstring: 'governance host' instead of uipath-runtime internals. - tests: can_handle uses a real stepped Workflow; a duck-typed look-alike is now rejected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a UiPath governance adapter for LlamaIndex workflows, integrating with LlamaIndex’s global instrumentation dispatcher and exposing registration via an entry point for governance-host discovery.
Changes:
- Introduces
LlamaIndexAdapter,GovernanceEventHandler, and callback routing for BEFORE_MODEL / AFTER_MODEL / TOOL_CALL. - Adds entry-point based adapter registration (
uipath.governance.adapters) with idempotent registration logic. - Adds unit tests exercising real LlamaIndex instrumentation events and dispatcher attach/detach behavior; updates dependencies/lockfile to include
uipath-core.
Reviewed changes
Copilot reviewed 4 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| packages/uipath-llamaindex/uv.lock | Adds uipath-core to the locked dependency set. |
| packages/uipath-llamaindex/tests/governance/test_adapter.py | New tests covering can_handle, dispatcher attach/detach, event routing, argument coercion, and exception semantics. |
| packages/uipath-llamaindex/tests/governance/init.py | Adds governance tests package marker. |
| packages/uipath-llamaindex/src/uipath_llamaindex/governance/adapter.py | Implements the LlamaIndex governance adapter, dispatcher handler, and payload extraction helpers. |
| packages/uipath-llamaindex/src/uipath_llamaindex/governance/init.py | Adds entry-point callable register_governance_adapter() with idempotent registry registration. |
| packages/uipath-llamaindex/pyproject.toml | Adds uipath-core dependency and the uipath.governance.adapters entry point. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Module docstring: registers via the uipath.governance.adapters entry point, not at import time. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These files were swept into the branch by a broad add; they are unrelated to the governance adapter. Reverting/removing them so the PR contains only governance changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An earlier cleanup commit compared against a stale local main and wrongly removed SETUP.MD and reverted the LlamaIndex docs change. Both files come from main (PRs #352/#356), not this branch. Restore them to the main version so this PR is governance-only with no spurious deletions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Core PR #1761 removed BaseAdapter from uipath-core. Migrate to the factory-evaluator pattern (matching #899): - governance/adapter.py -> event_handler.py: replace the BaseAdapter subclass (name/can_handle/attach/detach) with module-level install_governance() that registers the GovernanceEventHandler on the root instrumentation dispatcher; keep the handler + callbacks. File named for its seam (the event handler), like LangChain's callbacks.py. - runtime/factory.py: new_runtime reads `evaluator` from kwargs and calls install_governance. - governance/__init__.py: drop register_governance_adapter + registry import; expose install_governance. No import-time side effects. - pyproject.toml: remove the uipath.governance.adapters entry point. - tests (test_adapter.py -> test_event_handler.py): drop can_handle/ attach/detach; cover install_governance + factory wiring. ruff + mypy clean; 17 governance tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review findings (Viswa) for PR #360: - Re-install now rebinds the single process-global handler to the new run's evaluator / session instead of silently no-oping. The dispatcher is process-global, so a reused process serving a new runtime previously kept governing under the *first* run's evaluator; last install now wins. - Added uninstall_governance() (public detach) and wired it into the factory's dispose, so the global dispatcher does not retain the evaluator (and its resources) after the runtime is gone. Tests use it instead of mutating dispatcher.event_handlers directly. - Documented the process-global model explicitly: single active governance handler per process; two *concurrently* executing runtimes in one process would share the latest-installed evaluator — a property of LlamaIndex's global instrumentation, matching the one-workflow-per-process runtime model. - Documented that handle() is a deliberately synchronous gate: a BEFORE_MODEL / TOOL_CALL decision must complete (and be able to BLOCK) before the underlying call proceeds; an async out-of-band check could not gate it. - Drop the per-callbacks uuid trace_id (identical for every call); trace correlation is owned by the layer below, matching LangChain. Requires uipath-core >= 0.5.20 (removed trace_id from EvaluatorProtocol) — bumped. - Count llm/tool calls only after governance passes (no inflation on block). Tests: reinstall-rebinds-single-handler, uninstall-removes-handler, and made the swallow-on-error caplog assertion propagation-independent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a LlamaIndex governance adapter to
uipath-llamaindex. It lets UiPath governance evaluate what a LlamaIndex workflow/agent does at the model and tool level, and block disallowed actions, without the agent author writing governance code. This package contains only the LlamaIndex-specific bridge.What it does
Detects a
workflows.Workflow. LlamaIndex routes LLM and tool calls through its process-global instrumentation dispatcher (the same mechanism used for OpenInference tracing), so the adapter governs by registering an event handler on the root dispatcher rather than mutating the agent;attachreturns the agent unchanged.Maps each instrumentation event to a governance check:
LLMChatStartEventBEFORE_MODELLLMChatEndEventAFTER_MODELAgentToolCallEventTOOL_CALLLlamaIndex emits no tool-end event, so
AFTER_TOOLis not wired; a tool result is instead governed at the nextBEFORE_MODEL, where it is fed back to the model as input.Enforces by letting a
GovernanceBlockException(raised on a DENY decision) propagate, stopping the model call or tool. Any other error inside a governance hook is logged and swallowed, so a governance failure cannot break an otherwise-healthy agent run.Installed by the runtime factory: passing an
evaluatortonew_runtimewires governance onto the resolved agent in place. No adapter registry, no entry point, no import-time registration.What it does not do
BEFORE_AGENT/AFTER_AGENT); those are owned by the governance host.uipath-core.