FEAT Add dedicated LlamaGuard scorer

[immu4989]

Fixes #1830.

Summary

Adds a dedicated LlamaGuardScorer built on the scorer-composition API introduced by #2125, #2146, and #2157. The bundled default targets meta-llama/Llama-Guard-3-8B and its S1-S14 policy. The scorer sends Llama Guard's native plain-text classification request and parses its native safe / unsafe\nS1,S6 response without adding a generic SelfAskTrueFalseScorer hook.

• Adds explicit LlamaGuardMessageRole.USER and .AGENT request framing, defaulting to Agent for model-response classification.
• Keeps the PyRIT scoring objective on the resulting Score without inserting it into the Llama Guard conversation.
• Adds versioned Llama Guard 3 8B prompt and MLCommons policy assets under pyrit/datasets/score/llamaguard/.
• Adds LlamaGuardPolicy and LlamaGuardCategory value types for validated custom taxonomies and optional category descriptions.
• Binds strict response parsing to the configured policy, rejecting malformed output, duplicate categories, and categories outside that policy.
• Sends no system prompt, JSON response format, or response schema to the classifier endpoint.
• Replaces the architecture refactor's test-only Llama Guard pilot with production scorer, policy, parser, and transport coverage.

Usage

from pyrit.score import LlamaGuardScorer

scorer = LlamaGuardScorer(chat_target=llamaguard_endpoint)
scores = await scorer.score_text_async("Model response to classify")

User-input classification is available with message_role=LlamaGuardMessageRole.USER. Custom policies can be loaded with LlamaGuardPolicy.from_yaml(...) and passed through the policy argument. The 1B model uses S1-S13 rather than the bundled 8B S1-S14 policy and therefore requires a matching custom policy.

Validation

• Complete scorer unit suite: 1,378 passed, 16 skipped.
• Changed-file pre-commit hooks, including Ruff and ty: passed.

The endpoint round trip is covered with a mocked PromptTarget; this has not been exercised against a live Llama Guard deployment.

[romanlutz]

I don't have a llama-guard deployment and can't test this. Can you confirm that you did test it?

[immu4989]

I don't have a llama-guard deployment and can't test this. Can you confirm that you did test it?

The PR has not been exercised against a live LlamaGuard endpoint. The tests cover the plumbing in two places:

1. The pure-parser tests in tests/unit/score/test_llamaguard_parser.py validate the contract against every documented LlamaGuard output shape (safe, Safe (mixed case), whitespace, unsafe with single, multiple, missing, and empty category lines, plus empty input, refusal strings, and a malformed verdict).
2. The integration tests feed the same canonical output shapes through a mocked PromptChatTarget into SelfAskTrueFalseScorer with response_parser=parse_llamaguard_response, asserting the full Score object lands correctly with violated_categories in metadata.

What is not covered: the actual LlamaGuard-3-8B chat template rendering and the round-trip through a real endpoint. If a live smoke test against Together/Groq/Fireworks would help you merge with confidence, I am happy to run one and paste the transcript here. I held off on writing that as a unit-suite test because it would require a configured API key and would not be reproducible in CI.

[immu4989]

Friendly bump on this. Conflicts resolved twice now and merge-state is clean apart from the four conversations I've marked addressed. Happy to apply any further changes if you'd like, or close if direction has shifted.

[immu4989]

Closing this. The response_parser hook here was the right shape for the immediate LlamaGuard need, but the scorer-architecture series (#2125, #2146) replaces it with a cleaner CallableResponseHandler, and I would rather not leave a stale hook in the base class competing with that direction.

Glad parse_llamaguard_response is carried into the pilot in #2146. Once that lands I would like to follow up with ShieldGemma and WildGuard handlers on the same CallableResponseHandler pattern, which is the natural next step for the safety-classifier coverage this PR was aiming at.

Thanks for the review time on this one.

[romanlutz]

I actually think this is a great contribution and inspired me to do #2125 #2146 #2157 and now we can revive this! I hope you don't mind...

[immu4989]

I actually think this is a great contribution and inspired me to do #2125 #2146 #2157 and now we can revive this! I hope you don't mind...

Not at all, I am glad it was useful, and happy to revive it. Seeing the series land, the composition approach is clearly the better home for this than the response_parser hook this PR originally added.

Since #2125 through #2157 removed that hook, reviving this means reworking the PR onto the new API rather than merging as is. Concretely: drop the old response_parser plumbing, and ship the LlamaGuard piece as a parse_llamaguard_response helper plus the canonical Llama-Guard-3 system prompt, wired through CallableResponseHandler. No new scorer class, consistent with the series. The parser and prompt carry over almost directly from what was already here.

One consolidation note: a few hours before you reopened this, I filed #2172 proposing exactly that, the LlamaGuard parser and prompt on CallableResponseHandler, with a question about whether you wanted it in-tree. Your reopen answers that. I am happy to do the work here and have this PR close #2172, or keep #2172 as the design thread if you prefer. Whichever you would rather track.

I will start the rework unless you would rather shape the approach first.

[romanlutz]

@immu4989 i was curious to see if it would go as cleanly as I thought. See the current PR state. Is this in line with your thinking?

[immu4989]

Yes, this is exactly it. I pulled the branch and ran it: the two new test files pass (16), the full tests/unit/score suite is green (1379), and ty and ruff are clean. The shape is what I had in mind: parse_llamaguard_response and LLAMAGUARD_SYSTEM_PROMPT_PATH as the reusable pieces, the bundled question and system-prompt YAML, TrueFalseQuestionPaths.LLAMAGUARD, and the pilot replaced by real parser and composition tests. No new scorer class, composed through CallableResponseHandler.

One small optional thing from reading the parser: the metadata keys differ by verdict. The safe branch returns {"raw_classifier_output": ...} while the unsafe branch adds violated_categories, so a consumer doing score_metadata["violated_categories"] would KeyError on a safe score. If a stable schema across verdicts is worth it, the safe branch could return violated_categories: "" as well. Happy to push that small change to the branch if you would like it, otherwise it is fine as is.

On consolidation: this supersedes #2172, which I filed proposing this same work a few hours before you reopened here, so this PR can close it. I will add a pointer there.

I would also be glad to follow up with ShieldGemma and WildGuard on the same handler once this lands.

[romanlutz]

I suspect it's better to have a proper scorer vs what's in the PR right now. I'll just push now so that you can see what that looks like but I can remove that commit, too.

[immu4989]

I think the dedicated scorer is the better call, and it does not conflict with the composition direction. It still composes CallableResponseHandler with parse_llamaguard_response under the hood and goes through _run_llm_scoring_async, so it is a domain scorer sitting on top of the composition primitives rather than a return to the old plumbing-in-a-subclass pattern. I would keep the commit.

What the dedicated class buys over raw composition is the LlamaGuard-specific structure a generic scorer plus parser cannot express cleanly: LlamaGuardPolicy for enabling or customizing the S categories, and LlamaGuardMessageRole for classifying a User versus an Agent turn. Those are real parts of how LlamaGuard is used, and having them as first-class constructor arguments is a much better experience than asking every user to hand render the policy into a prompt string. The parser validating returned codes against the active policy is a nice touch too.

Ran it here: the new test_llamaguard_scorer, test_llamaguard_policy, and test_llamaguard_parser all pass (30), the full tests/unit/score suite is green (1394), and ty and ruff are clean.

One follow on this shape makes easy: ShieldGemma and WildGuard could each be the same pattern, a small policy plus parser plus dedicated scorer, once this lands. Happy to take those.

[romanlutz]

One follow on this shape makes easy: ShieldGemma and WildGuard could each be the same pattern, a small policy plus parser plus dedicated scorer, once this lands. Happy to take those.

Yes that's exactly what I was thinking!

[romanlutz]

I made a few more minor changes. @immu4989 do you happen to have a deployment to test this?