Canonicalize tests to @safetestset for module isolation#64
Closed
ChrisRackauckas-Claude wants to merge 1 commit into
Closed
Canonicalize tests to @safetestset for module isolation#64ChrisRackauckas-Claude wants to merge 1 commit into
ChrisRackauckas-Claude wants to merge 1 commit into
Conversation
Convert each independent test unit to run in its own module via @safetestset, matching the canonical SciML structure (isolation between tests + world-age safety). Core (runtests.jl): - The 4 inline @testset units (Version bumping, Project file handling, Repository processing, Basic functionality legacy) become inline @safetestset blocks, each carrying the using lines it needs (OrgMaintenanceScripts, Test, plus TOML where used). - Each per-file unit (formatting, min_version_fixer, version_check_finder, invalidation_analysis, import_timing_analysis, explicit_imports_fixer, documentation_cleanup) is now @safetestset "Name" begin include("x.jl") end. The included files are made self-contained: stale "already loaded by runtests.jl" comments replaced with explicit using lines (OrgMaintenanceScripts, Test, plus Dates/TOML where the body uses now()/DateTime or TOML.print). These previously relied on Main-leaked imports from runtests.jl. - The commented-out multiprocess_testing include is preserved (still disabled), now shown in @safetestset form. QA (qa.jl): the two plain @testset units (Aqua, JET) extracted into qa_aqua.jl / qa_jet.jl and wrapped as @safetestset ... include(...). The include() form is required for the JET unit because @test_opt is a package macro that must be resolved after `using JET` runs (top-level statements in the included file macroexpand one at a time). @test_broken lines preserved. Deps: SafeTestsets added to Project.toml [extras]/[targets].test and [compat] SafeTestsets = "0.1, 1"; same UUID + compat added to test/qa/Project.toml (QA runs in its own activated env). using SafeTestsets added at top of runtests.jl. GROUP-dispatch ladder, run path, assertions, and test counts unchanged. Verified locally: GROUP=Core Pkg.test passes (Version bumping 5, Project file handling 6, Repository processing 8, Basic functionality 2, Formatting 9, Min Version Fixer 25, Version Check Finder 23, Invalidation 36, Import Timing 56, Explicit Imports 10, Documentation Cleanup 41). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Superseded by the v1.2 folder conversion on sciml-testing-rollout (#63). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Converts each independent test unit to run in its own module via
@safetestset, matching the canonical SciML test structure (isolation between tests + world-age safety). Behavior-preserving: same tests run, same assertions, same GROUP dispatch.Core (
test/runtests.jl)The Core group was a single monolithic
@testset "OrgMaintenanceScripts.jl"wrapping 4 inline@testsets plus 7include(...)calls, all relying on imports leaked intoMainfrom the top ofruntests.jl.@safetestsetblocks, each carrying its ownusinglines (OrgMaintenanceScripts,Test, plusTOMLwhere used).@safetestset "Name" begin include("x.jl") end. The included files are made self-contained: the stale# Note: ... already loaded by runtests.jlcomments are replaced with explicitusinglines. Two files additionally needusing Dates(they callnow()/DateTimeat top level), and one needsusing TOML(callsTOML.print). These previously worked only viaMain-leaked imports.multiprocess_testinginclude is preserved (still disabled), shown in@safetestsetform for consistency.QA (
test/qa.jl)The two plain
@testsetunits (Aqua, JET) are extracted intoqa_aqua.jl/qa_jet.jland wrapped as@safetestset ... include(...). Theinclude()form is required for the JET unit because@test_optis a package macro that must resolve afterusing JETruns (top-level statements in the included file macroexpand one at a time, unlike an inline@safetestsetbody).@test_brokenlines preserved.Deps
SafeTestsetsadded toProject.toml[extras]/[targets].testand[compat] SafeTestsets = "0.1, 1"; same UUID + compat added totest/qa/Project.toml(QA runs in its own activated env).using SafeTestsetsadded at the top ofruntests.jl.Verification
Ran locally on Julia 1.11:
GROUP=Core Pkg.test()passes. All 11 converted units pass:Version bumping 5/5, Project file handling 6/6, Repository processing 8/8, Basic functionality (legacy) 2/2, Formatting Functions 9/9, Minimum Version Fixer 25/25, Version Check Finder 23/23, Invalidation Analysis 36/36, Import Timing Analysis 56/56, Explicit Imports Fixer 10/10, Documentation Cleanup 41/41.
(The "Analysis failed" log lines from the invalidation/import-timing tests are pre-existing expected behavior: those tests deliberately run analysis on a mock package and assert the failure path returns
-1— unrelated to this change.)Ignore until reviewed by @ChrisRackauckas.