feat: json.patch builtin with Rego set support + partial-rule multi-body fix (picks up #442)

[Vitalii Tverdokhlib (vitaliytv)]

Summary

Picks up #442 (json.patch builtin) by Mats Willemsen (@ma-ts), which stalled on review feedback. Addresses both requests from Anand Krishnamoorthi (@anakrish)'s review:

1. Avoid the double round-trip through to_json_str()/from_str() — Value already implements Serialize/Deserialize directly.
2. Register v0/jsonpatch and v1/jsonpatch in tests/opa.passing so OPA's own jsonpatch compliance suite runs in CI.

Doing (2) surfaced a real gap: OPA's json.patch operates on the Rego value directly and special-cases set (a set member is addressed by value — there's no JSON equivalent; see OPA's internal/edittree). The json-patch crate only understands plain JSON, so every set-typed case (add/remove/move on a Rego {"a","b","c"}) silently degraded to array-index semantics and returned Undefined instead of the expected result.

Given that, this PR replaces the json-patch/jsonptr/thiserror dependency with a native implementation over regorus::Value, mirroring OPA's own semantics (github.com/open-policy-agent/opa v1/topdown/json.go + internal/edittree/edittree.go @ v1.2.0):

• Path parsing: leading / is optional (OPA-specific relaxation of RFC6901), array-form paths carry raw (unescaped, non-string-only) segments.
• object: key lookup. array: numeric index / "-" append. set: lookup and insert by value equality.
• add/remove/replace/move/copy/test are composed from two primitives (functional insert/remove) — same decomposition as OPA's EditTree-based applyPatches.
• Any patch-application failure yields Undefined unconditionally, matching builtinJSONPatch, which never hard-errors on a bad patch regardless of strict-builtin-errors.

Second commit: a real (unrelated) interpreter bug found along the way

Wiring v0/jsonpatch into opa.passing initially still failed on OPA's own json_patch_tests batch-comparison rule, for a reason that had nothing to do with json.patch itself. Minimal repro (no jsonpatch involved):

package minrepro
import rego.v1

items := {"a": {"err": "x"}, "b": {"ok": 1}}

passed[k] = t if {
  t := items[k]
  not t.err
} else = t if {
  t := items[k]
  t.err
}

Expected {"a": {...}, "b": {...}}, got only {"b": {...}}.

Root cause in eval_rule_bodies (src/interpreter.rs): the multi-body loop breaks as soon as one body produces a value. That's correct else semantics for a complete rule/function (first matching body wins), but for a partial (object/set) rule, different bodies can legitimately contribute different keys — once body 1 matched something, body 2 (which owns a different key) was never even attempted, silently dropping it. Fix: for partial rules only (ctx.is_set || ctx.key_expr.is_some()), evaluate every body and carry the accumulator (Context::value / Context::rule_value) forward between them instead of discarding it — both are needed; either alone still loses results. Complete rules and functions keep the original first-body-wins behavior untouched.

Known follow-up, out of scope here: for old-style stacked bodies with no else keyword, a body recovered by this fix (that isn't the first) still binds the wrong output value when it doesn't explicitly restate the assignment, because RuleBody::assign is None for those and the parser never threads the rule head's output expression into them. I attempted to fix that at parse time (cloning the head's RuleAssign into each stacked body), but that reuses one Expr node's eidx across multiple RuleBodys and trips an eidx-uniqueness invariant used by the loop-hoisting table elsewhere in the compiler ("expression with eidx N already exists") — a materially bigger, riskier change. Not needed for the regression this PR chases, since the OPA batch-comparison rule only depends on key presence (not passed_cases[k]), not the bound value. Happy to open a separate issue/PR for it if useful.

RVM has an analogous, already-tracked gap for multi-body partial-object rules (#665, still open) — tests/opa.rs now skips the RVM cross-check for jsonpatch/json_patch_tests specifically, same pattern already used elsewhere in that file for other known RVM gaps.

Test plan

• cargo clippy --all-targets --all-features -- -D warnings (incl. ffi/java/python/wasm/ruby bindings) — clean
• cargo fmt --check — clean
• Interpreter test fixture (tests/interpreter/cases/builtins/objects/json.patch.yaml, from feat: add json patch support #442, all 6 RFC6902 ops) — 128/128 passing
• OPA's own jsonpatch compliance suite (cargo test --test opa -- v0/jsonpatch v1/jsonpatch): 7/7 both — both now registered in tests/opa.passing
• Full tests/opa.passing regression run (everything the suite already covers, ~2875 cases across both commits) — 0 failures

Attribution

First commit is Mats Willemsen (@ma-ts)'s original patch, cherry-picked as-is (rebased onto current main) to preserve authorship. Second and third commits are the rework and the interpreter fix described above.

[Vitalii Tverdokhlib (vitaliytv)]

Vitalii Tverdokhlib (@vitaliytv) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

• (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

• (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

Contribution License Agreement

This Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”), and conveys certain license rights to Microsoft Corporation and its affiliates (“Microsoft”) for Your contributions to Microsoft open source projects. This Agreement is effective as of the latest signature date below.

1. Definitions.
   “Code” means the computer software code, whether in human-readable or machine-executable form,
   that is delivered by You to Microsoft under this Agreement.
   “Project” means any of the projects owned or managed by Microsoft and offered under a license
   approved by the Open Source Initiative (www.opensource.org).
   “Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
   Project, including but not limited to communication on electronic mailing lists, source code control
   systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
   discussing and improving that Project, but excluding communication that is conspicuously marked or
   otherwise designated in writing by You as “Not a Submission.”
   “Submission” means the Code and any other copyrightable material Submitted by You, including any
   associated comments and documentation.
2. Your Submission. You must agree to the terms of this Agreement before making a Submission to any
   Project. This Agreement covers any and all Submissions that You, now or in the future (except as
   described in Section 4 below), Submit to any Project.
3. Originality of Work. You represent that each of Your Submissions is entirely Your original work.
   Should You wish to Submit materials that are not Your original work, You may Submit them separately
   to the Project if You (a) retain all copyright and license information that was in the materials as You
   received them, (b) in the description accompanying Your Submission, include the phrase “Submission
   containing materials of a third party:” followed by the names of the third party and any licenses or other
   restrictions of which You are aware, and (c) follow any other instructions in the Project’s written
   guidelines concerning Submissions.
4. Your Employer. References to “employer” in this Agreement include Your employer or anyone else
   for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
   Submission is made in the course of Your work for an employer or Your employer has intellectual
   property rights in Your Submission by contract or applicable law, You must secure permission from Your
   employer to make the Submission before signing this Agreement. In that case, the term “You” in this
   Agreement will refer to You and the employer collectively. If You change employers in the future and
   desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
   and secure permission from the new employer before Submitting those Submissions.
5. Licenses.

• Copyright License. You grant Microsoft, and those who receive the Submission directly or
  indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license in the
  Submission to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute
  the Submission and such derivative works, and to sublicense any or all of the foregoing rights to third
  parties.
• Patent License. You grant Microsoft, and those who receive the Submission directly or
  indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license under
  Your patent claims that are necessarily infringed by the Submission or the combination of the
  Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
  import or otherwise dispose of the Submission alone or with the Project.
• Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
  No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
  granted by implication, exhaustion, estoppel or otherwise.

6. Representations and Warranties. You represent that You are legally entitled to grant the above
   licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
   have disclosed under Section 3). You represent that You have secured permission from Your employer to
   make the Submission in cases where Your Submission is made in the course of Your work for Your
   employer or Your employer has intellectual property rights in Your Submission by contract or applicable
   law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
   have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
   You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
   REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
   EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
   PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
   NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
7. Notice to Microsoft. You agree to notify Microsoft in writing of any facts or circumstances of which
   You later become aware that would make Your representations in this Agreement inaccurate in any
   respect.
8. Information about Submissions. You agree that contributions to Projects and information about
   contributions may be maintained indefinitely and disclosed publicly, including Your name and other
   information that You submit with Your Submission.
9. Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
   the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
   Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
   exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
   defenses of lack of personal jurisdiction and forum non-conveniens.
10. Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
    supersedes any and all prior agreements, understandings or communications, written or oral, between
    the parties relating to the subject matter hereof. This Agreement may be assigned by Microsoft.

@microsoft-github-policy-service agree company="Nitra"

[Copilot]

Pull request overview

Adds first-class json.patch support aligned with OPA semantics (including Rego set behavior) and fixes an interpreter correctness bug where multi-body partial rules could drop keys. It also expands CI coverage by registering OPA’s jsonpatch compliance suites.

Changes:

• Implement json.patch natively over regorus::Value (feature-gated) and add an interpreter fixture for the builtin.
• Fix interpreter eval_rule_bodies to accumulate results across bodies for partial (object/set) rules.
• Register v0/jsonpatch and v1/jsonpatch in tests/opa.passing and skip RVM validation for the known multi-body partial-object gap.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/opa.rs	Skips RVM cross-check for OPA’s jsonpatch/json_patch_tests due to known RVM limitation.
tests/opa.passing	Enables OPA jsonpatch compliance suites in CI (v0/jsonpatch, v1/jsonpatch).
tests/interpreter/cases/builtins/objects/json.patch.yaml	Adds interpreter fixture coverage for json.patch (currently object/array-focused).
src/interpreter.rs	Fixes partial-rule multi-body evaluation to accumulate contributions across bodies.
src/builtins/objects.rs	Registers and implements feature-gated json.patch over Value with OPA-like path/collection semantics.
Cargo.toml	Introduces jsonpatch feature and includes it in full-opa.

[Anand Krishnamoorthi (anakrish)]

Vitalii Tverdokhlib (@vitaliytv) Thank you for taking this on. Can you address the test failure and see if the copilot review comments make sense? Overall, the PR looks good to me.

[Vitalii Tverdokhlib (vitaliytv)]

Vitalii Tverdokhlib (@vitaliytv) Thank you for taking this on. Can you address the test failure and see if the copilot review comments make sense? Overall, the PR looks good to me.

Thank you! done

[Anand Krishnamoorthi (anakrish)]

Automated review: inline findings from independent repository code-review and deep-review skill runs. Each issue was retained after an adversarial verification pass.

[Anand Krishnamoorthi (anakrish)]

Vitalii Tverdokhlib (@vitaliytv) I did one more review focusing on corner cases. After those are addressed, it should be good to go. Thanks!

[Vitalii Tverdokhlib (vitaliytv)]

Vitalii Tverdokhlib (Vitalii Tverdokhlib (@vitaliytv)) I did one more review focusing on corner cases. After those are addressed, it should be good to go. Thanks!

Thank you! Done.

[Anand Krishnamoorthi (anakrish)]

Review summary — Approve (mergeable), with tracked follow-ups

I reviewed this PR using the repo's code-review and deep-review skills (multi-agent, diff + source), and independently validated behavior by building and running the targeted suites.

Verification (local, jsonpatch feature):

• json.patch unit tests + interpreter fixture (128 cases) — ✅
• RVM partial_object_rules.yaml, memory-limit propagation test — ✅
• OPA v0/v1 jsonpatch conformance — ✅ 14/14 (incl. RVM cross-check)
• Full opa.passing regression — ✅ 2875 / 0

The json.patch native EditTree (with OPA-compatible set-by-value semantics), the LimitError-propagation-vs-Undefined-on-malformed-patch handling, and the interpreter/RVM multi-body partial-rule fix are all correct for normal policy evaluation. No reachable allow/deny bug found. 👍

I've filed the non-blocking follow-ups as #781 — the plan is to merge this PR and address them in a follow-up PR, which is a reasonable approach since none affects correctness of default evaluation.

Findings (all tracked in #781)

#	Severity	Location	Summary
1	Med (dual-path)	rvm/vm/rules.rs rule_frame_after_success	Suspendable RVM path still skips later successful bodies of partial rules — run-to-completion was fixed, suspendable was not. Not reachable via default eval_rule (suspendable mode is test-only today), so not a production bug, but the paths now diverge.
2	Med	rvm/program/types.rs else_bodies (#[serde(default)])	Deserialized/hand-built RuleInfo with absent markers defaults to false (= "not else"), so partial else bodies could run as independent bodies. Validate shape at load.
3	Low	builtins/json_patch.rs parse_path/array_index	Leading-/ relaxation is intentional (matches OPA) ✅. But invalid ~ escapes, signed/leading-zero indices (+1/-0), and collapsed //a are accepted more leniently than RFC6902 — can yield a wrong non-Undefined result on malformed input.
4	Low	builtins/json_patch.rs apply	Full EditTree built before op-shape validation, so a malformed patch on a huge target can surface LimitError before the documented Undefined.
5	Med (semver)	rvm/program/types.rs pub else_bodies	New required public field breaks external struct-literal construction. Fine pre-1.0; consider #[non_exhaustive]/constructor.
6	Low (schema)	ast.rs RuleBody.is_else + engine.rs get_ast_as_json	New serialized AST field while version stays 1. Bump/document, or skip_serializing_if.

Suggested before merge (optional, low effort)

The two items most worth doing before RVM suspendable/serialized-program modes are relied upon are #1 (suspendable parity) and #2 (else_bodies metadata safety). Everything else is comfortably deferrable to #781.

Nice work — the native rewrite over regorus::Value with real set support is a clear improvement over the json-patch crate approach, and the accompanying partial-rule fix is principled and well-tested.

[Anand Krishnamoorthi (anakrish)]

Vitalii Tverdokhlib (@vitaliytv) thank you for your contribution! I will go ahead an merge this PR. I've created an issue to track some outstanding issues that the deep-review found.