Pre-release validation: executed notebook outputs + validation fixes (post #2157)#2187
Draft
romanlutz wants to merge 10 commits into
Draft
Pre-release validation: executed notebook outputs + validation fixes (post #2157)#2187romanlutz wants to merge 10 commits into
romanlutz wants to merge 10 commits into
Conversation
Keep scenario DTOs typed for CLI output, migrate the custom-target example to first_message, and remove empty per-objective attacks when reconstructing sampled scenario resumes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Repair Python E2E target propagation and notebook examples, correct the Responses vision payload, and isolate and harden frontend E2E validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Repair scanner isolation, provider resource cleanup, Hugging Face downloads, Windows-safe output, and executable notebook examples discovered by exhaustive post-merge validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Use the newly configured adversarial_chat target in the benchmark end-to-end smoke test after confirming registration and a clean 7/7 full scenario run. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Remove the generic scorer lifecycle method and threshold-wrapper delegation while retaining Azure Content Safety's explicit resource cleanup. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Store the audited execution results for all runnable documentation notebooks so output cells can be reviewed directly in the branch changeset. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Retain Realtime WebSocket cleanup as a public API while moving ordinary OpenAI teardown behind a private hook and removing validation-only cleanup calls from documentation notebooks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
…lidation branch Resolved notebook conflicts in doc/code/converters/1_text_to_text_converters and doc/code/executor/1_single_turn by regenerating executed outputs from the merged .py sources. Verified affected tests (110 passed), full pre-commit, and notebook cell-output audit (0 errors). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This is a draft validation/review branch, not a clean mergeable change. It exists so the executed notebook outputs and validation repairs can be reviewed in a diff. It contains committed notebook outputs and several
Merge origin/maincommits (latest main: #2178 FlipAttack→fliptechnique, #2158 harm-category standardization), so it is not intended to be merged as-is.What's in here
--start-serverport; Windows CP1252 encoding-safe output paths.snapshot_download+ direct local load (failures no longer swallowed).AzureContentFilterScorerclose clients/credentials idempotently (no generic scorer-cleanup contract added, per review).origin/mainmerged (MAINT: Migrate FlipAttack to a core 'flip' attack technique #2178, FEAT Overlay harm-category standardization across dataset loaders #2158, and prior) with the two conflicted notebooks regenerated from their merged.pysources.Validation evidence (high level)
uv sync --all-extras; affected post-merge groups 110 (MAINT: Migrate FlipAttack to a core 'flip' attack technique #2178) + 2,266 (FEAT Overlay harm-category standardization across dataset loaders #2158 datasets/models) passed.tywith all extras.Full evidence-based report with per-notebook/per-cell questionable outputs is in the session artifacts (
files/validation-report.md).Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com
Copilot-Session: 52d2518d-ebe5-4888-8189-6146b6503448