[FEAT]: Deprecate the rampart_sinks fixture in favor of the pytest_rampart_sinks hook#105
Open
nina-msft wants to merge 6 commits into
Open
[FEAT]: Deprecate the rampart_sinks fixture in favor of the pytest_rampart_sinks hook#105nina-msft wants to merge 6 commits into
rampart_sinks fixture in favor of the pytest_rampart_sinks hook#105nina-msft wants to merge 6 commits into
Conversation
nina-msft
marked this pull request as ready for review
July 7, 2026 23:17
nina-msft
force-pushed
the
dev/nina-msft/deprecate-sinks-fixture
branch
from
July 7, 2026 23:35
6de45a8 to
1c373d9
Compare
nina-msft
commented
Jul 7, 2026
spencrr
reviewed
Jul 8, 2026
spencrr
left a comment
Contributor
There was a problem hiding this comment.
Looks good! First quick pass:
- print_deprecation_message -> log deprecation warning maybe?
- teats have a lot of mocks -> let’s use pytester and capture warnings that way?
nina-msft
force-pushed
the
dev/nina-msft/deprecate-sinks-fixture
branch
from
July 8, 2026 00:10
1c373d9 to
cd89f7b
Compare
Contributor
Author
|
@spencrr I just updated the test name to be |
The pytest_rampart_sinks hook is the supported sink registration API. Resolving the legacy rampart_sinks fixture - in the single-process bootstrap and in controller-side discovery - now emits a DeprecationWarning pointing users to the hook, scheduled for removal in 1.0.0. Adds rampart/common/deprecation.py (mirrors PyRIT's print_deprecation_message), wires it into both fixture-consumption paths, documents the deprecation in the pytest-integration and xdist guides, and adds unit + pytester coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nina-msft
force-pushed
the
dev/nina-msft/deprecate-sinks-fixture
branch
from
July 8, 2026 00:52
cd89f7b to
3ea79a3
Compare
spencrr
reviewed
Jul 8, 2026
nina-msft
enabled auto-merge (squash)
July 10, 2026 23:52
bashirpartovi
left a comment
Contributor
There was a problem hiding this comment.
This is great @nina-msft . However, there are a few places in the documentation that need to be updated:
- Quickstart doc, we need to update Step 4:
concepts/overview.mddescribes the feature as "Report sinks: Structured output via the rampart_sinks fixture" see pytest-integrationci-integration.mdshows a full@pytest.fixture … def rampart_sinks()code sample as the main example (it does have a downstream !!! tip pointing at the hook for xdist, so it's the least bad of the 3, but the headline example is still the fixture).
nina-msft
disabled auto-merge
July 15, 2026 17:13
Update the five docs that still presented the deprecated rampart_sinks fixture as the primary path (quickstart Step 4, concepts overview, ci-integration headline example, usage index nav, results-and-reporting wiring) to lead with the pytest_rampart_sinks hook and frame the fixture as a deprecated fallback. Fixes anchor links to #pytest_rampart_sinks-hook and drops the now-unused pytest/ReportSink imports from the CI sample. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80e88044-7224-4c36-b9ac-3873c4c3d2ae
Resolve conflict in tests/unit/pytest_plugin/test_xdist_aggregation.py by adopting upstream's configured_pytester fixture (asyncio ini isolation) and layering this branch's fixture-deprecation additions on top: - _LIST_CONFTEST module-level list-form sinks conftest - TestSinkFixtureDeprecation covering single-process warn, xdist controller warn, and list-form no-warn cases Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80e88044-7224-4c36-b9ac-3873c4c3d2ae
The merged-in ruff DOC ruleset flags a ``Returns: None`` section on a ``-> None`` function (docstring-extraneous-returns). Remove it so ``ruff check`` passes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80e88044-7224-4c36-b9ac-3873c4c3d2ae
Deprecating the `rampart_sinks` fixture (not the module-level list form) means the list form must stay silent. That contract was guarded only by the slow `pytester` subprocess test `test_controller_list_form_does_not_warn_under_xdist`, whose discovery and report-count assertions already overlap `test_finds_list_rampart_sinks` and `test_xdist_emits_single_consolidated_report`. Move the one unique assertion (list form emits no deprecation warning) into fast, in-process tests in `TestSinkDeprecationWarning`, adding a symmetric "fixture form warns" assertion that no unit test previously covered. Drop the subprocess test and its now-unused `_LIST_CONFTEST`. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80e88044-7224-4c36-b9ac-3873c4c3d2ae
The slow xdist aggregation tests re-verify verdict/threshold/rendering logic that is already fully unit-tested in test_plugin.py. Trim the redundant subprocess permutations while keeping the genuine integration guards, and backfill the two verdict cases that were only covered end-to-end so no coverage is lost. - Fold test_population_statistics_over_full_set into test_xdist_emits_single_consolidated_report (one fewer -n 2 run). - Drop the three loadgroup trial-group verdict permutations; keep the safety-critical fails_when_any_unsafe_under_load end-to-end guard. - Add below-threshold-FAIL and all-safe-PASS model tests to TestSessionRecordTrialGroup and a PASSED-line test to TestTrialGroupRendering. Removes ~4 slow subprocess runs (~90s each) with equivalent fast in-process coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 80e88044-7224-4c36-b9ac-3873c4c3d2ae
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #73. Now that the
pytest_rampart_sinkshook is the supportedsink-registration API, this PR begins deprecating the legacy
rampart_sinksfixture.
Resolving the fixture now emits a
DeprecationWarning(removal target:0.3.0)in both fixture-consumption paths:
_rampart_sink_bootstrap, after the fixture resolves._resolve_sink_candidate, when it unwraps a genuine@pytest.fixture-formrampart_sinks(not module-level lists or plaincallables).
The hook stays authoritative and the fixture remains a single-process fallback,
so nothing breaks for existing users — they just get a migration nudge.
Changes
rampart/common/deprecation.py—emit_deprecation_warning(...),mirroring PyRIT's helper (qualified-name resolution +
DeprecationWarningatstacklevel=3).plugin.pyand_xdist.pyfixture-resolution paths.usage/pytest-integration.mdandusage/xdist.md.tests/unit/common/test_deprecation.py(unit) and aTestSinkFixtureDeprecationclass in the slow aggregation suite that drivesreal subprocess pytest runs via
pytester— single-process warn,xdist-controller warn, and list-form no-warn — replacing the previous
MagicMock-based warn/no-warn tests.Breaking changes
None. The fixture continues to work as a single-process fallback; it only emits
a
DeprecationWarningnow.Validation
uv run ruff check .→ all checks pass;ruff format --checkclean.uv run ty check→ all checks passed.uv run pytest tests/unit/common/test_deprecation.py tests/unit/pytest_plugin/test_xdist.py tests/unit/pytest_plugin/test_plugin.py→ 145 passed.uv run pytest tests/unit/pytest_plugin/test_xdist_aggregation.py -m slow→ 16 passed (includes thepytester-basedTestSinkFixtureDeprecationclass).Notes
removed_in="0.3.0"is the removal milestone for the fixture — deprecated inthe
0.1line, kept through0.2, removed in0.3(two minor versions, perdocs/contributing/release-process.md).@bashirpartovi.
Checklist
pre-commit run --all-filespasses