Skip to content

[failproofai-626] Support multiple custom policy files - #623

Merged
nk-ag merged 10 commits into
mainfrom
luv-626
Jul 29, 2026
Merged

[failproofai-626] Support multiple custom policy files#623
nk-ag merged 10 commits into
mainfrom
luv-626

Conversation

@NiveditJain

@NiveditJain NiveditJain commented Jul 29, 2026

Copy link
Copy Markdown
Member

Summary

  • add ordered, additive customPoliciesPaths configuration while retaining legacy customPoliciesPath support
  • allow repeated --custom flags and load, list, remove, and display all configured files
  • add independent dashboard toggles for every policy in explicit and convention files
  • persist off policies as source-qualified IDs, so duplicate names across files/scopes remain independently controllable
  • exclude off policies before event matching and function execution
  • update dashboard, documentation, changelog, unit tests, and end-to-end coverage

Toggle edge-case coverage

  • multiple explicit files, including duplicate policy names
  • duplicate convention names at project and user scope
  • sibling policies in the same file remain active
  • disabled IDs merged across project/local/global scopes
  • legacy singular and current plural custom-path config
  • stale disabled IDs are harmless
  • re-enabling removes the persisted opt-out

Validation

  • TypeScript typecheck passed
  • ESLint passed with 5 pre-existing warnings and 0 errors
  • focused unit/integration suites: 105 passed
  • custom-policy end-to-end suite: 51 passed
  • repository-wide unit run: 2460 passed; 18 pre-existing HOME/filesystem-sensitive failures reproduced in unrelated cache, Hermes, convention-sync, and first-run suites
  • CI and supply-chain checks passed

Summary by CodeRabbit

  • New Features
    • Added repeating --custom support for multiple explicit custom policy files, with correct install/uninstall and load order.
    • Dashboard and runtime now apply independent enable/disable toggles per explicit/convention policy file, backed by merged disabledCustomPolicies.
    • Configuration now supports customPoliciesPaths (array) with legacy customPoliciesPath retained.
  • Documentation
    • Updated CLI help and custom policy/configuration docs for multi-file and toggle behavior.
  • Bug Fixes
    • Fixed dashboard toggle labeling/state display, corrected the detail-panel disclosure glyph, and prevented Configure tab toggle state from getting stuck.
  • Tests
    • Expanded E2E and unit coverage for multi-path loading, deduping, and disabled policy merge/toggle rules.

Latest CLI status fix

  • failproofai policies now renders dashboard-disabled explicit policies as OFF without a green checkmark
  • convention files render OFF when all policies are disabled and MIXED when enabled and disabled siblings coexist; disabled names are labeled (OFF)
  • clean Git-equipped Docker E2E: 314 passed, 6 intentionally skipped
  • clean Docker production build, npm package, global install, and two-file custom-policy discovery passed
  • fix the Policies dashboard event-detail disclosure so it renders the intended ▾ glyph instead of literal Unicode escape text
  • merged-scope dashboard correctness: displayed state matches runtime, enabling clears contributing project/local/user opt-outs, and unreadable explicit files are isolated
  • fix the Policies loading ellipsis so it renders … instead of literal Unicode escape text

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds support for multiple explicit custom policy files through repeated CLI flags and ordered customPoliciesPaths. It also adds source-qualified per-policy enable/disable state for explicit and convention files across runtime loading, configuration, dashboard actions, UI rendering, tests, and documentation.

Changes

Custom policy management

Layer / File(s) Summary
Configuration contract and scope merging
src/hooks/policy-types.ts, src/hooks/hooks-config.ts, app/actions/get-hooks-config.ts, __tests__/hooks/hooks-config.test.ts
Adds ordered custom paths, legacy normalization, disabled-policy merging, and dashboard policy metadata with IDs and enabled states.
CLI installation and lifecycle management
bin/failproofai.mjs, src/hooks/manager.ts, __tests__/hooks/manager.test.ts
Collects repeated --custom flags and updates validation, persistence, telemetry, removal, listing, and replacement behavior for multiple paths.
Runtime loading and policy filtering
src/hooks/custom-hooks-loader.ts, src/hooks/handler.ts, __tests__/hooks/loader-path-dedup.test.ts, __tests__/e2e/hooks/custom-hooks.e2e.test.ts
Loads unique explicit files, tags explicit and convention hooks with IDs, and skips disabled hooks before registration.
Dashboard toggles and UI state
app/policies/hooks-client.tsx, app/actions/update-hooks-config.ts, __tests__/actions/*, __tests__/hooks/list-convention-column.test.ts
Adds independent policy toggles, persists disabled IDs, and renders OFF/MIXED states.
Documentation and release notes
docs/*, CHANGELOG.md
Documents repeated custom paths, compatibility with the legacy field, merge behavior, and dashboard changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI as policies CLI
  participant Installer as installHooks
  participant Config as hooks config
  participant HookEvent as handleHookEvent
  participant Loader as loadAllCustomHooks
  participant Dashboard as Policies dashboard
  CLI->>Installer: collect repeated --custom paths
  Installer->>Config: store ordered customPoliciesPaths
  HookEvent->>Config: read paths and disabled IDs
  HookEvent->>Loader: load and identify policy hooks
  Loader-->>HookEvent: return eligible hooks
  Dashboard->>Config: persist toggle state
Loading

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

Possibly related PRs

Suggested labels: enhancement

Suggested reviewers: chhhee10, hermes-exosphere

Poem

I’m a rabbit with paths in a row,
Policy toggles now softly glow.
Flags repeat, IDs align,
Hooks hop past the disabled line.
Fresh files load—what a fine design!

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.10% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description is detailed, but it omits the template's Type of Change and Checklist sections. Add the Type of Change checkbox section and the required checklist items, or note why they are intentionally omitted.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: support for multiple custom policy files.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added the enhancement New feature or request label Jul 29, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (2)
app/actions/get-hooks-config.ts (1)

200-214: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicate custom-policy-paths fallback logic across two files. Both sites independently compute config.customPoliciesPaths ?? (config.customPoliciesPath ? [config.customPoliciesPath] : []) to normalize legacy/plural custom policy paths from a HooksConfig.

  • app/actions/get-hooks-config.ts#L200-L214: extract this fallback into a shared exported helper (e.g. resolveCustomPoliciesPaths(config) in src/hooks/hooks-config.ts) and call it here.
  • src/hooks/manager.ts#L700-L721: use the same shared helper instead of re-deriving explicitPaths inline.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/actions/get-hooks-config.ts` around lines 200 - 214, The custom-policy
path normalization is duplicated across the hooks configuration loader and
manager. Add and export a shared resolveCustomPoliciesPaths helper in
src/hooks/hooks-config.ts, then update app/actions/get-hooks-config.ts lines
200-214 and src/hooks/manager.ts lines 700-721 to use it instead of their inline
fallback or explicitPaths derivation.
__tests__/e2e/hooks/custom-hooks.e2e.test.ts (1)

19-32: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Make the first policy observable.

The first file only returns allow() with no visible effect. A regression that loads only secondPath still passes, so this does not verify that both files—or their order—are honored. Make the second policy depend on an observable effect from the first, or add assertions that distinguish both registrations.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@__tests__/e2e/hooks/custom-hooks.e2e.test.ts` around lines 19 - 32, The test
“loads multiple explicit custom policy files” does not verify that firstPath is
loaded or ordered before secondPath because the first policy’s allow result is
unobservable. Update the fixture policies and assertions so the second policy’s
behavior depends on an observable effect from the first, or otherwise assert
distinct evidence that both registrations execute in the expected order.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Line 6: Update the single-line feature entry in CHANGELOG.md to append the
actual pull request number in parentheses, preserving the existing description
and single-line format.

In `@docs/custom-policies.mdx`:
- Around line 75-80: Update the custom-policy documentation examples and
accompanying text to use plural terminology for multiple explicit entries:
change “the custom policies path” to “custom policies paths” and “file” to
“files” around the --custom uninstall and explicit --custom descriptions, while
preserving the existing command behavior and ordering explanation.

In `@src/hooks/custom-hooks-loader.ts`:
- Around line 233-242: Normalize every custom policy path with resolve before
checking loadedPaths or comparing against convention paths, including paths
already marked absolute. Update the path construction in the custom policy
loading loop, then use that normalized value consistently for existsSync,
deduplication, and loadSingleFile.
- Around line 231-242: Fix the per-event loading flow around loadSingleFile so
Bun reliably republishes custom hooks after policy-registry is cleared, without
relying on the ?v= cache-busting query or loadSequence; alternatively, remove
the fresh-loading guarantee and update the documented contract. Apply the loader
change at src/hooks/custom-hooks-loader.ts:231-242 and align the corresponding
statement at docs/custom-policies.mdx:82 with the implemented behavior.

In `@src/hooks/manager.ts`:
- Around line 215-260: Update the custom policy path condition in the
surrounding configuration-update flow so an empty customPoliciesPath array is
treated as absent rather than entering the write and validation branch. Preserve
processing for non-empty arrays and string values, while avoiding deletion of
the legacy field or writing an empty customPoliciesPaths value when no paths are
supplied.

---

Nitpick comments:
In `@__tests__/e2e/hooks/custom-hooks.e2e.test.ts`:
- Around line 19-32: The test “loads multiple explicit custom policy files” does
not verify that firstPath is loaded or ordered before secondPath because the
first policy’s allow result is unobservable. Update the fixture policies and
assertions so the second policy’s behavior depends on an observable effect from
the first, or otherwise assert distinct evidence that both registrations execute
in the expected order.

In `@app/actions/get-hooks-config.ts`:
- Around line 200-214: The custom-policy path normalization is duplicated across
the hooks configuration loader and manager. Add and export a shared
resolveCustomPoliciesPaths helper in src/hooks/hooks-config.ts, then update
app/actions/get-hooks-config.ts lines 200-214 and src/hooks/manager.ts lines
700-721 to use it instead of their inline fallback or explicitPaths derivation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 79975120-a4ec-49f8-ba89-b4d12aa657a9

📥 Commits

Reviewing files that changed from the base of the PR and between ed998a6 and 2801ac6.

📒 Files selected for processing (13)
  • CHANGELOG.md
  • __tests__/e2e/hooks/custom-hooks.e2e.test.ts
  • __tests__/hooks/hooks-config.test.ts
  • __tests__/hooks/manager.test.ts
  • app/actions/get-hooks-config.ts
  • app/policies/hooks-client.tsx
  • bin/failproofai.mjs
  • docs/custom-policies.mdx
  • src/hooks/custom-hooks-loader.ts
  • src/hooks/handler.ts
  • src/hooks/hooks-config.ts
  • src/hooks/manager.ts
  • src/hooks/policy-types.ts

Comment thread CHANGELOG.md Outdated
Comment thread docs/custom-policies.mdx
Comment thread src/hooks/custom-hooks-loader.ts
Comment thread src/hooks/custom-hooks-loader.ts
Comment thread src/hooks/manager.ts
chhhee10 and others added 3 commits July 29, 2026 20:43
Installing with --custom replaced the configured set, so `-c a` then `-c b`
left only `b` — and nothing was printed to say `a` had stopped applying. The
policies just went inert. Repeated flags in ONE command worked; across commands
the earlier ones vanished, which is how a user hit it within minutes.

Now additive, mirroring enabledPolicies directly above it in the same function:
union with what is configured, deduped, order preserved. `replace` (passed by
the configure wizard) still makes the given set authoritative, exactly as it
does for enabled policies — so the wizard can still remove, and
`policies --uninstall -c` still clears everything.

Carried-over paths are filtered by existence first: a file deleted after it was
configured must not make every future install exit(1) on the strict validation
below. Only paths added by THIS invocation are validated; earlier ones were
validated when they were added.

Four tests, verified failing without the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
__tests__/e2e/hooks/custom-hooks.e2e.test.ts (1)

19-80: 🩺 Stability & Availability | 🔵 Trivial

Complete required clean-container and full-CI validation.

The PR summary says the Docker validation stalled and repository-wide failures remain. Re-run the required clean-container E2E validation and ensure all required CI jobs pass before pushing. As per coding guidelines, “After every push, wait for all CI jobs to finish and fix any failure” and “Validate every non-trivial change end-to-end in a clean Docker container that mimics a real installation.”

Also applies to: 461-498

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@__tests__/e2e/hooks/custom-hooks.e2e.test.ts` around lines 19 - 80, Complete
the required clean-container end-to-end validation for the custom policy
scenarios covered by the tests around runHook, then rerun the full CI suite and
resolve every repository-wide failure before pushing. After each push, wait for
all CI jobs to finish and verify they pass.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/actions/get-hooks-config.ts`:
- Line 180: Align dashboard policy state with runtime scope merging: in
app/actions/get-hooks-config.ts:180, derive enabled/disabled status from the
merged configuration used by src/hooks/handler.ts rather than a single raw
config. In app/actions/update-hooks-config.ts:32-42, add scope-aware mutation
and provenance handling so enabling a policy removes the effective disable
across the relevant scope(s), not merely from the selected config copy.
- Around line 213-219: Update the per-path processing in the
parsedCustomPolicies Promise.all within PoliciesTab configuration loading so
errors from parseCustomPoliciesFromFile are caught for each explicit policy path
and that path returns an empty policy list. Preserve successful parsing, policy
ID generation, and disabled-state handling, allowing other files to load and
customPolicies to be flattened normally.

---

Nitpick comments:
In `@__tests__/e2e/hooks/custom-hooks.e2e.test.ts`:
- Around line 19-80: Complete the required clean-container end-to-end validation
for the custom policy scenarios covered by the tests around runHook, then rerun
the full CI suite and resolve every repository-wide failure before pushing.
After each push, wait for all CI jobs to finish and verify they pass.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2682531b-442f-480c-a792-011ca6ba0a0a

📥 Commits

Reviewing files that changed from the base of the PR and between 432ac38 and c8c1429.

📒 Files selected for processing (14)
  • CHANGELOG.md
  • __tests__/actions/get-hooks-config.test.ts
  • __tests__/actions/update-hooks-config.test.ts
  • __tests__/e2e/hooks/custom-hooks.e2e.test.ts
  • __tests__/hooks/hooks-config.test.ts
  • app/actions/get-hooks-config.ts
  • app/actions/update-hooks-config.ts
  • app/policies/hooks-client.tsx
  • docs/configuration.mdx
  • docs/custom-policies.mdx
  • src/hooks/custom-hooks-loader.ts
  • src/hooks/handler.ts
  • src/hooks/hooks-config.ts
  • src/hooks/policy-types.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • CHANGELOG.md
  • src/hooks/hooks-config.ts
  • src/hooks/policy-types.ts
  • docs/custom-policies.mdx

Comment thread app/actions/get-hooks-config.ts
Comment thread app/actions/get-hooks-config.ts Outdated
nk-ag
nk-ag previously approved these changes Jul 29, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/actions/get-hooks-config.ts`:
- Around line 213-228: Deduplicate resolved custom policy paths before parsing
in the configuration flow around configuredCustomPolicyPaths and
resolvedCustomPaths. Use a Set-based approach that canonicalizes equivalent
paths and preserves first-seen order, so each file is parsed once while existing
policy ID and disabled-state handling remains unchanged. Add a regression test
covering equivalent entries such as rules.mjs and ./rules.mjs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 10a86ec5-7c47-4d10-ab45-e89dd01b72df

📥 Commits

Reviewing files that changed from the base of the PR and between c8c1429 and 8d0c306.

📒 Files selected for processing (11)
  • CHANGELOG.md
  • __tests__/actions/get-hooks-config.test.ts
  • __tests__/actions/update-hooks-config.test.ts
  • __tests__/e2e/hooks/custom-hooks.e2e.test.ts
  • __tests__/hooks/list-convention-column.test.ts
  • __tests__/hooks/manager.test.ts
  • app/actions/get-hooks-config.ts
  • app/actions/update-hooks-config.ts
  • app/policies/hooks-client.tsx
  • src/hooks/hooks-config.ts
  • src/hooks/manager.ts
🚧 Files skipped from review as they are similar to previous changes (6)
  • tests/actions/update-hooks-config.test.ts
  • app/actions/update-hooks-config.ts
  • tests/e2e/hooks/custom-hooks.e2e.test.ts
  • tests/hooks/manager.test.ts
  • app/policies/hooks-client.tsx
  • src/hooks/manager.ts

Comment thread app/actions/get-hooks-config.ts
@nk-ag
nk-ag merged commit 3e9c0df into main Jul 29, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants