Skip to content

Enable strict json check in api compare#6969

Closed
sudo-shashank wants to merge 3 commits intomainfrom
shashank/enable-strict-json
Closed

Enable strict json check in api compare#6969
sudo-shashank wants to merge 3 commits intomainfrom
shashank/enable-strict-json

Conversation

@sudo-shashank
Copy link
Copy Markdown
Contributor

@sudo-shashank sudo-shashank commented Apr 27, 2026

Summary of changes

Changes introduced in this pull request:

  • Set FOREST_STRICT_JSON=1 for all api compare check and fix the unknown field error found in some RPC response.

Reference issue to close (if applicable)

Closes #5635

Other information and links

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

  • I have read and agree to the CONTRIBUTING document.
  • I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

  • New Features

    • Extended execution trace data with IPLD operation details and execution logs.
    • Added optional agent information to version endpoint responses.
    • Network parameters now include genesis timestamp.
    • Fork upgrade parameters include new upgrade height field.
    • Introduced new IPLD trace data schemas and message CID serialization.
  • Tests

    • Enhanced API comparison tests with strict JSON validation.
    • Updated test fixtures for execution traces and signed messages.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 27, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 52f2ccd3-8333-4d8d-aa8d-3b85ae9a9676

📥 Commits

Reviewing files that changed from the base of the PR and between 073a2c8 and bd9b661.

⛔ Files ignored due to path filters (2)
  • src/rpc/snapshots/forest__rpc__tests__rpc__v0.snap is excluded by !**/*.snap
  • src/rpc/snapshots/forest__rpc__tests__rpc__v1.snap is excluded by !**/*.snap
📒 Files selected for processing (1)
  • src/lotus_json/signed_message.rs

Walkthrough

This PR extends the RPC API with additional operational tracking and configuration fields. It adds IPLD operation details and execution logs to ExecutionTrace, introduces GenesisTimestamp to NetworkParams, adds UpgradeXxHeight to ForkUpgradeParams, includes an optional Agent field in PublicVersion, and adds a CID field to MessageLotusJson. It also enables strict JSON deserialization for API comparison tests.

Changes

Cohort / File(s) Summary
OpenRPC Schema Specifications
docs/openrpc-specs/v0.json, docs/openrpc-specs/v1.json
Extended ExecutionTrace with IpldOps and Logs fields; added GenesisTimestamp to NetworkParams (required); added UpgradeXxHeight to ForkUpgradeParams (required); added optional Agent to PublicVersion; introduced new TraceIpld and TraceIpldOp schemas.
RPC Type Definitions
src/rpc/methods/state/types.rs
Introduced TraceIpldOp enum (with Get, Put, Unknown variants) and TraceIpld struct for IPLD operation tracking; extended ExecutionTrace with optional logs and ipld_ops fields; customized PartialEq for ExecutionTrace to exclude these new fields from equality comparisons.
RPC Method Implementations
src/rpc/methods/common.rs, src/rpc/methods/state.rs, src/rpc/methods/eth.rs
Added optional agent: Option<String> field to PublicVersion in Filecoin.Version handler; added genesis_timestamp: u64 to NetworkParams sourced from genesis block; replaced commented upgrade_xxx_height with concrete upgrade_xx_height field in ForkUpgradeParams; updated test helper to initialize logs and ipld_ops as empty vectors.
State Management
src/state_manager/utils.rs
Updated trace parsing to explicitly populate logs and ipld_ops with empty vectors when constructing ExecutionTrace.
Lotus JSON Serialization
src/lotus_json/message.rs, src/lotus_json/signed_message.rs
Added optional cid: Option<Cid> field to MessageLotusJson serialized as "CID"; updated conversion logic to populate cid on serialization and ignore it on deserialization; updated test snapshot to include "CID" attribute.
Testing Configuration
scripts/tests/api_compare/docker-compose.yml
Added FOREST_STRICT_JSON=1 environment variable to api-compare, api-compare-gateway, and api-compare-offline containers for strict JSON handling.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • LesnyRumcajs
  • akaladarshi
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Title check ⚠️ Warning The PR title 'Enable strict json check in api compare' refers to a real aspect of the changes but is misleading about the main scope. While the docker-compose update enables strict JSON checking, the primary work involves fixing RPC response schemas and deserialization. Revise title to reflect the main objective: 'Fix RPC response schemas for strict JSON deserialization' or 'Add missing fields to RPC responses for strict JSON validation'.
Docstring Coverage ⚠️ Warning Docstring coverage is 77.78% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed The PR addresses both criteria from issue #5635: implements strict deserialization (FOREST_STRICT_JSON=1 enables it) and fixes offences by adding missing fields to RPC response schemas (ExecutionTrace, ForkUpgradeParams, NetworkParams, PublicVersion, MessageLotusJson).
Out of Scope Changes check ✅ Passed The PR includes all in-scope changes: docker-compose updates for strict JSON, RPC response schema fixes, struct field additions, and test adjustments. All changes directly support the #5635 objective of strict JSON deserialization.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch shashank/enable-strict-json
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch shashank/enable-strict-json

Comment @coderabbitai help to get the list of available commands and usage tips.

@sudo-shashank sudo-shashank added the RPC requires calibnet RPC checks to run on CI label Apr 27, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lotus_json/message.rs (1)

108-120: ⚠️ Potential issue | 🟠 Major

Don’t discard CID if these responses still need semantic parity checks.

from_lotus_json now drops the parsed CID, so RpcTest::identity / RpcTest::validate only compare the message body fields for Message responses. That means ChainGetMessage and GasEstimateMessageGas can serialize a wrong embedded CID and still pass the new strict-json compare as long as the field is syntactically valid. Please keep the CID in a compare-only type, or add an explicit raw-JSON CID assertion for these endpoints.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/lotus_json/message.rs` around lines 108 - 120, The from_lotus_json
implementation is dropping the parsed CID (cid: _) which lets responses with
incorrect embedded CIDs pass semantic parity checks; update from_lotus_json in
message.rs to preserve the parsed CID (e.g., bind cid and store it on the
returned compare-type or on Message) instead of discarding it, or alternatively
add an explicit raw-JSON CID assertion in the RpcTest identity/validate code
paths for the Message responses (notably ChainGetMessage and
GasEstimateMessageGas) so the CID is included in comparisons; locate the
from_lotus_json function and the LotusJson/Message compare-type and either
propagate the cid into that type or add a CID check in RpcTest::identity /
RpcTest::validate.
🧹 Nitpick comments (1)
src/rpc/methods/state.rs (1)

3234-3234: Replace the raw upgrade sentinel with a named constant.

Line 3284 uses a magic value for upgrade_xx_height. Please extract it to a named constant with a short rationale to reduce accidental drift.

♻️ Suggested refactor
+const UPGRADE_XX_HEIGHT_PLACEHOLDER: ChainEpoch = 999_999_999_999_999;
+
 impl TryFrom<&ChainConfig> for ForkUpgradeParams {
@@
-            upgrade_xx_height: 999_999_999_999_999,
+            upgrade_xx_height: UPGRADE_XX_HEIGHT_PLACEHOLDER,

Also applies to: 3284-3284

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/rpc/methods/state.rs` at line 3234, Introduce a named constant (e.g.
UPGRADE_XX_HEIGHT: ChainEpoch = <magic_value>) and replace the raw magic literal
used for upgrade_xx_height with that constant; update the declaration/assignment
involving upgrade_xx_height and any other occurrences to reference
UPGRADE_XX_HEIGHT, and add a brief doc comment explaining the rationale
(prevents accidental drift and centralizes the sentinel for upgrade XX). Ensure
references to the symbol upgrade_xx_height in functions/structs in this module
now use UPGRADE_XX_HEIGHT so the sentinel is defined once and clearly
documented.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@src/lotus_json/message.rs`:
- Around line 108-120: The from_lotus_json implementation is dropping the parsed
CID (cid: _) which lets responses with incorrect embedded CIDs pass semantic
parity checks; update from_lotus_json in message.rs to preserve the parsed CID
(e.g., bind cid and store it on the returned compare-type or on Message) instead
of discarding it, or alternatively add an explicit raw-JSON CID assertion in the
RpcTest identity/validate code paths for the Message responses (notably
ChainGetMessage and GasEstimateMessageGas) so the CID is included in
comparisons; locate the from_lotus_json function and the LotusJson/Message
compare-type and either propagate the cid into that type or add a CID check in
RpcTest::identity / RpcTest::validate.

---

Nitpick comments:
In `@src/rpc/methods/state.rs`:
- Line 3234: Introduce a named constant (e.g. UPGRADE_XX_HEIGHT: ChainEpoch =
<magic_value>) and replace the raw magic literal used for upgrade_xx_height with
that constant; update the declaration/assignment involving upgrade_xx_height and
any other occurrences to reference UPGRADE_XX_HEIGHT, and add a brief doc
comment explaining the rationale (prevents accidental drift and centralizes the
sentinel for upgrade XX). Ensure references to the symbol upgrade_xx_height in
functions/structs in this module now use UPGRADE_XX_HEIGHT so the sentinel is
defined once and clearly documented.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 5026415e-3cba-4e3d-b814-81fb089f7e5a

📥 Commits

Reviewing files that changed from the base of the PR and between 65c39a0 and 068e96a.

⛔ Files ignored due to path filters (2)
  • src/rpc/snapshots/forest__rpc__tests__rpc__v0.snap is excluded by !**/*.snap
  • src/rpc/snapshots/forest__rpc__tests__rpc__v1.snap is excluded by !**/*.snap
📒 Files selected for processing (15)
  • docs/openrpc-specs/v0.json
  • docs/openrpc-specs/v1.json
  • scripts/tests/api_compare/docker-compose.yml
  • src/lotus_json/message.rs
  • src/lotus_json/signed_message.rs
  • src/rpc/methods/chain.rs
  • src/rpc/methods/common.rs
  • src/rpc/methods/eth.rs
  • src/rpc/methods/gas.rs
  • src/rpc/methods/state.rs
  • src/rpc/methods/state/types.rs
  • src/state_manager/utils.rs
  • src/tool/subcommands/api_cmd/api_compare_tests.rs
  • src/tool/subcommands/api_cmd/stateful_tests.rs
  • src/wallet/subcommands/wallet_cmd.rs

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/rpc/methods/state.rs (1)

3234-3284: Replace the hard-coded upgrade sentinel with a named constant.

Line 3284 uses a raw sentinel epoch. A named constant will make intent clearer and reduce drift risk in future edits.

♻️ Suggested small cleanup
+const UPGRADE_XX_SENTINEL_HEIGHT: ChainEpoch = 999_999_999_999_999;
...
-            upgrade_xx_height: 999_999_999_999_999,
+            upgrade_xx_height: UPGRADE_XX_SENTINEL_HEIGHT,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/rpc/methods/state.rs` around lines 3234 - 3284, The hard-coded sentinel
999_999_999_999_999 assigned to ForkUpgradeParams::upgrade_xx_height in the
TryFrom<&ChainConfig> impl should be replaced with a named constant (e.g.,
UPGRADE_SENTINEL or MAX_SENTINEL_EPOCH) to clarify intent and avoid future
drift; define the constant near the top of the module (or in the crate's
constants module) and use that constant in the try_from conversion for
upgrade_xx_height so all references are explicit and easy to update.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/rpc/methods/state.rs`:
- Around line 3234-3284: The hard-coded sentinel 999_999_999_999_999 assigned to
ForkUpgradeParams::upgrade_xx_height in the TryFrom<&ChainConfig> impl should be
replaced with a named constant (e.g., UPGRADE_SENTINEL or MAX_SENTINEL_EPOCH) to
clarify intent and avoid future drift; define the constant near the top of the
module (or in the crate's constants module) and use that constant in the
try_from conversion for upgrade_xx_height so all references are explicit and
easy to update.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: eff58c4a-d341-4268-9bb4-ee3402754126

📥 Commits

Reviewing files that changed from the base of the PR and between 068e96a and 0b44d82.

⛔ Files ignored due to path filters (2)
  • src/rpc/snapshots/forest__rpc__tests__rpc__v0.snap is excluded by !**/*.snap
  • src/rpc/snapshots/forest__rpc__tests__rpc__v1.snap is excluded by !**/*.snap
📒 Files selected for processing (8)
  • docs/openrpc-specs/v0.json
  • docs/openrpc-specs/v1.json
  • scripts/tests/api_compare/docker-compose.yml
  • src/rpc/methods/common.rs
  • src/rpc/methods/eth.rs
  • src/rpc/methods/state.rs
  • src/rpc/methods/state/types.rs
  • src/state_manager/utils.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • src/rpc/methods/eth.rs
  • scripts/tests/api_compare/docker-compose.yml
  • src/rpc/methods/common.rs
  • src/state_manager/utils.rs

@sudo-shashank sudo-shashank marked this pull request as ready for review April 28, 2026 00:57
@sudo-shashank sudo-shashank requested a review from a team as a code owner April 28, 2026 00:57
@sudo-shashank sudo-shashank requested review from LesnyRumcajs and akaladarshi and removed request for a team April 28, 2026 00:57
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
2029 3 2026 0
View the top 3 failed test(s) by shortest run time
forest-filecoin::lotus_json::all_snapshots
Stack Traces | 0.041s run time
thread 'lotus_json::all_snapshots' (45774) panicked at src/lotus_json/mod.rs:307:5:
assertion failed: `(left == right)`: snapshot failed for forest::message::signed_message::SignedMessage

Diff < left / right > :
<{"CID":{"/":"bafy2bzaced3xdk2uf6azekyxgcttujvy3fzyeqmibtpjf2fxcpfdx2zcx4s3g"},"Message":{"CID":{"/":"bafy2bzaced3xdk2uf6azekyxgcttujvy3fzyeqmibtpjf2fxcpfdx2zcx4s3g"},"From":"f00","GasFeeCap":"0","GasLimit":0,"GasPremium":"0","Method":0,"Nonce":0,"Params":null,"To":"f00","Value":"0","Version":0},"Signature":{"Data":"aGVsbG8gd29ybGQh","Type":2}}
>{"CID":{"/":"bafy2bzaced3xdk2uf6azekyxgcttujvy3fzyeqmibtpjf2fxcpfdx2zcx4s3g"},"Message":{"From":"f00","GasFeeCap":"0","GasLimit":0,"GasPremium":"0","Method":0,"Nonce":0,"Params":null,"To":"f00","Value":"0","Version":0},"Signature":{"Data":"aGVsbG8gd29ybGQh","Type":2}}


note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::tests::openrpc_v0
Stack Traces | 0.064s run time
thread 'rpc::tests::openrpc_v0' (46694) panicked at .../home/runner/.cargo.../registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.47.2/src/runtime.rs:719:13:
snapshot assertion for 'rpc/v0' failed in line 822
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
forest-filecoin::rpc::tests::openrpc_v1
Stack Traces | 0.067s run time
thread 'rpc::tests::openrpc_v1' (46695) panicked at .../home/runner/.cargo.../registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.47.2/src/runtime.rs:719:13:
snapshot assertion for 'rpc/v1' failed in line 822
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

RPC requires calibnet RPC checks to run on CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deny unknown fields on RPC response deserialization

1 participant