Skip to content

refactor(tests): auto-convert array-api-strict modules#5528

Merged
wanghan-iapcm merged 1 commit into
deepmodeling:masterfrom
njzjz:array-api-strict-reg
Jun 14, 2026
Merged

refactor(tests): auto-convert array-api-strict modules#5528
wanghan-iapcm merged 1 commit into
deepmodeling:masterfrom
njzjz:array-api-strict-reg

Conversation

@njzjz

@njzjz njzjz commented Jun 13, 2026

Copy link
Copy Markdown
Member

Summary:

  • add a dpmodel-to-array-api-strict wrapper registry with lazy module registration
  • replace repeated array_api_strict descriptor/fitting/utils setattr conversions with shared conversion logic
  • convert residual data lists during append/assignment for DPA2/DPA3 strict wrappers

Tests:

  • ruff check .
  • ruff format --check .
  • pytest source/tests/consistent/descriptor/test_dpa3.py -k array_api_strict -q
  • pytest source/tests/consistent/descriptor/test_dpa2.py -k array_api_strict -q
  • pytest source/tests/consistent/descriptor -k array_api_strict -q
  • pytest source/tests/consistent/fitting -k array_api_strict -q

Summary by CodeRabbit

  • New Features

    • Introduced centralized conversion mechanism for standardizing array handling across model objects with registry-based mappings and lazy module imports.
    • Added support for five additional descriptor types.
    • Enhanced list-value conversion with optional preservation of original list structure.
  • Refactor

    • Simplified descriptor and fitting network classes by consolidating array conversion logic into a unified decorator-based approach.
    • Converted dynamically generated network classes to explicit class definitions for improved clarity.

@github-advanced-security github-advanced-security AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR refactors the array_api_strict testing module to centralize object conversion logic. Instead of individual __setattr__ overrides in descriptor, fitting, and utility classes, a registry-based decorator system (@array_api_strict_module) and conversion functions automatically wrap dpmodel objects into strict array-api types during attribute assignment.

Changes

Central conversion framework

Layer / File(s) Summary
Array conversion function updates and module state
source/tests/array_api_strict/common.py
to_array_api_strict_array return type is clarified and docstring updated; module-level registry, lazy registration tracking, and ArrayAPIList (a list subclass that auto-converts items) are added.
Conversion detection and lookup utilities
source/tests/array_api_strict/common.py
Public register_dpmodel_mapping API, internal predicates for object classification, lazy registration loader _ensure_registrations, and try_convert_module for registry lookups are implemented.
Central conversion entry points and attribute setters
source/tests/array_api_strict/common.py
List conversion helper _try_convert_list, core entry point convert_array_api_strict_value, attribute setter array_api_strict_setattr (with special handling for data-list attributes), and the array_api_strict_module decorator that wires classes and auto-registers dpmodel mappings are added.

Descriptor module simplification and centralization

Layer / File(s) Summary
Descriptor namespace expansion and DPA1 base refactoring
source/tests/array_api_strict/descriptor/__init__.py, source/tests/array_api_strict/descriptor/dpa1.py
Five new descriptor classes (DPA2, DPA3, SeAttenV2, SeT, SeTTebd) are added to namespace exports; DPA1 module classes are decorated with @array_api_strict_module and all custom __setattr__ implementations are removed, becoming thin pass subclasses.
DPA2 and DPA3 descriptor simplification
source/tests/array_api_strict/descriptor/dpa2.py, source/tests/array_api_strict/descriptor/dpa3.py
DPA2 and DPA3 descriptors are simplified from coercion/deserialization implementations to minimal @array_api_strict_module-decorated pass subclasses inheriting from their DP counterparts.
SE descriptor variant refactoring
source/tests/array_api_strict/descriptor/se_atten_v2.py, source/tests/array_api_strict/descriptor/se_e2_a.py, source/tests/array_api_strict/descriptor/se_e2_r.py, source/tests/array_api_strict/descriptor/se_t.py, source/tests/array_api_strict/descriptor/se_t_tebd.py
SE variants (SeAttenV2, SeA, SeR, SeT, SeTTebd) remove setattr overrides and adopt @array_api_strict_module decoration; SeAttenV2 adds dpmodel mapping registration for serialize/deserialize round-trips.
Hybrid and specialized descriptor refactoring
source/tests/array_api_strict/descriptor/hybrid.py, source/tests/array_api_strict/descriptor/repflows.py, source/tests/array_api_strict/descriptor/repformers.py
Hybrid, repflows, and repformers modules remove custom setattr implementations and apply @array_api_strict_module decoration; RepflowLayer and RepformerLayer add _array_api_strict_data_list_attrs declarations for residual list attributes.

Utility modules simplification

Layer / File(s) Summary
Exclude mask utility simplification
source/tests/array_api_strict/utils/exclude_mask.py
AtomExcludeMask and PairExcludeMask remove setattr attribute conversion and are decorated with @array_api_strict_module.
Network utility refactoring with explicit classes and registration
source/tests/array_api_strict/utils/network.py
NativeNet, EmbeddingNet, FittingNet are refactored from factory-produced module-level assignments to explicit @array_api_strict_module-decorated classes; Identity class is added; NativeLayer retains specific attribute coercion (w, b, idt); dpmodel mappings are registered for all network types via serialize/deserialize round-trips.
Type embedding utility simplification
source/tests/array_api_strict/utils/type_embed.py
TypeEmbedNet removes setattr deserialization logic and is decorated with @array_api_strict_module.

Fitting module simplification

Layer / File(s) Summary
Fitting namespace export and class simplification
source/tests/array_api_strict/fitting/__init__.py, source/tests/array_api_strict/fitting/fitting.py
Five fitting network classes (EnergyFittingNet, PropertyFittingNet, DOSFittingNet, DipoleFittingNet, PolarFittingNet) are re-exported in __init__.py; fitting.py removes the setattr_for_general_fitting helper and all setattr overrides, replacing them with @array_api_strict_module-decorated pass subclasses.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • deepmodeling/deepmd-kit#5204: Both PRs establish registry-driven dpmodel-to-wrapper conversion infrastructure and centralized setattr/wrapping flows for automatic strict-type conversion.

Suggested labels

enhancement, Python

Suggested reviewers

  • wanghan-iapcm
  • iProzd
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 21.05% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'refactor(tests): auto-convert array-api-strict modules' accurately summarizes the main change: centralizing array-api-strict conversion logic through an auto-conversion system.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
source/tests/array_api_strict/common.py (1)

160-167: 💤 Low value

Minor: double conversion when keep_converting=True.

When converted is an ArrayAPIList, calling converted.append(converted_item) invokes the overridden append that calls convert_array_api_strict_value again on an already-converted item. The conversion is idempotent so this is correct, but wasteful.

♻️ Optional: bypass the override for initial population
 def _try_convert_list(value: list[Any], *, keep_converting: bool = False) -> list[Any]:
     converted = ArrayAPIList() if keep_converting else []
     changed = keep_converting
     for item in value:
         converted_item = convert_array_api_strict_value(item)
-        converted.append(converted_item)
+        list.append(converted, converted_item)
         changed = changed or converted_item is not item
     return converted if changed else value
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@source/tests/array_api_strict/common.py` around lines 160 - 167, The loop in
_try_convert_list uses converted.append on an ArrayAPIList when
keep_converting=True, triggering ArrayAPIList.append which re-calls
convert_array_api_strict_value and causes a redundant conversion; fix by
collecting converted items into a temporary plain list (e.g., temp = []) inside
the loop (use temp.append(converted_item) and update changed there), and after
the loop, if keep_converting is True build the ArrayAPIList once from that temp
(converted = ArrayAPIList(temp)) so the overridden append is not invoked for
each item; leave the return logic (return converted if changed else value)
unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@source/tests/array_api_strict/common.py`:
- Around line 160-167: The loop in _try_convert_list uses converted.append on an
ArrayAPIList when keep_converting=True, triggering ArrayAPIList.append which
re-calls convert_array_api_strict_value and causes a redundant conversion; fix
by collecting converted items into a temporary plain list (e.g., temp = [])
inside the loop (use temp.append(converted_item) and update changed there), and
after the loop, if keep_converting is True build the ArrayAPIList once from that
temp (converted = ArrayAPIList(temp)) so the overridden append is not invoked
for each item; leave the return logic (return converted if changed else value)
unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: c3e8b7bc-bafd-4e48-8dee-69eb6913cd3f

📥 Commits

Reviewing files that changed from the base of the PR and between 5d94bd6 and c558c14.

📒 Files selected for processing (18)
  • source/tests/array_api_strict/common.py
  • source/tests/array_api_strict/descriptor/__init__.py
  • source/tests/array_api_strict/descriptor/dpa1.py
  • source/tests/array_api_strict/descriptor/dpa2.py
  • source/tests/array_api_strict/descriptor/dpa3.py
  • source/tests/array_api_strict/descriptor/hybrid.py
  • source/tests/array_api_strict/descriptor/repflows.py
  • source/tests/array_api_strict/descriptor/repformers.py
  • source/tests/array_api_strict/descriptor/se_atten_v2.py
  • source/tests/array_api_strict/descriptor/se_e2_a.py
  • source/tests/array_api_strict/descriptor/se_e2_r.py
  • source/tests/array_api_strict/descriptor/se_t.py
  • source/tests/array_api_strict/descriptor/se_t_tebd.py
  • source/tests/array_api_strict/fitting/__init__.py
  • source/tests/array_api_strict/fitting/fitting.py
  • source/tests/array_api_strict/utils/exclude_mask.py
  • source/tests/array_api_strict/utils/network.py
  • source/tests/array_api_strict/utils/type_embed.py

@codecov

codecov Bot commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.19%. Comparing base (5d94bd6) to head (c558c14).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5528      +/-   ##
==========================================
- Coverage   82.19%   82.19%   -0.01%     
==========================================
  Files         891      891              
  Lines      101599   101599              
  Branches     4242     4242              
==========================================
- Hits        83507    83506       -1     
  Misses      16789    16789              
- Partials     1303     1304       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@njzjz njzjz requested a review from wanghan-iapcm June 13, 2026 16:41
@wanghan-iapcm wanghan-iapcm added this pull request to the merge queue Jun 14, 2026
Merged via the queue into deepmodeling:master with commit 87d8557 Jun 14, 2026
73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants