Skip to content

docs(tutorials): pass@k evaluation and severity scoring for red-team results (#30, #31) - #55

Open
opitaru-sys wants to merge 1 commit into
GenAI-Security-Project:mainfrom
opitaru-sys:tutorial-eval-framework-passk
Open

docs(tutorials): pass@k evaluation and severity scoring for red-team results (#30, #31)#55
opitaru-sys wants to merge 1 commit into
GenAI-Security-Project:mainfrom
opitaru-sys:tutorial-eval-framework-passk

Conversation

@opitaru-sys

Copy link
Copy Markdown

Follows up on my comment in #30. This adds a self-contained tutorial plus a small standard-library harness that gives red-team results a consistent evaluation and severity layer, toward #30 and relating to #31.

The core idea is statistical honesty about stochastic guards: a single success out of one attempt tells you very little, so the harness scores attacks as pass@k with a Wilson confidence interval, enforces an N-reroll rule before anything is called reproduced, reports zero-success runs as a bounded null rather than "secure", and only assigns severity to findings that actually reproduced. Records carry optional OWASP LLM Top 10 and MITRE ATLAS tags, and the report aggregates by both.

Contents: the tutorial README, the harness (redteam_eval.py, CLI included), a JSON Schema for input records, a worked example dataset covering confirmed / bounded-null / inconclusive findings, and a 22-test suite runnable with python test_redteam_eval.py. Standard library only, Python 3.10+. isort and black run per CONTRIBUTING.

One layout note: existing tutorials are flat .md files, but this one carries code, so it uses a subfolder the way exploitation/ and sandboxes/ already do. Happy to flatten or restructure if you prefer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant