-
Notifications
You must be signed in to change notification settings - Fork 6
feat(ai): add pyoaev support for AI adversarial exposure validation (#295) #296
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SamuelHassine
wants to merge
4
commits into
main
Choose a base branch
from
feature/295-ai-adversarial-exposure-validation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+219
−1
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
94c262d
feat(ai): add pyoaev support for AI adversarial exposure validation (…
SamuelHassine 3202c21
feat(ai): add Artificial Intelligence security domain (#295)
SamuelHassine 8ff0bf3
test(ai): cover AI SDK primitives and fix expectations return type (#…
SamuelHassine 86147e6
refactor(ai): build AI expectations path with a single f-string (#295)
SamuelHassine File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| from pyoaev.base import RESTManager, RESTObject | ||
| from pyoaev.mixins import CreateMixin, DeleteMixin, GetMixin, ListMixin, UpdateMixin | ||
| from pyoaev.utils import RequiredOptional | ||
|
|
||
|
|
||
| class AiTarget(RESTObject): | ||
| _id_attr = "asset_id" | ||
|
|
||
|
|
||
| class AiTargetManager( | ||
| GetMixin, ListMixin, CreateMixin, UpdateMixin, DeleteMixin, RESTManager | ||
| ): | ||
| """Manage AI Target assets (LLM endpoints / AI agents under adversarial test).""" | ||
|
|
||
| _path = "/ai_targets" | ||
| _obj_cls = AiTarget | ||
| _create_attrs = RequiredOptional( | ||
| required=("asset_name", "ai_target_provider"), | ||
| optional=( | ||
| "asset_description", | ||
| "asset_tags", | ||
| "asset_external_reference", | ||
| "ai_target_endpoint", | ||
| "ai_target_model", | ||
| "ai_target_modality", | ||
| "ai_target_system_prompt", | ||
| "ai_target_api_key_variable", | ||
| "ai_target_configuration", | ||
| ), | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| """Deterministic per-inject canary marker shared by the AI red-team injector and the AI defense | ||
| collectors. | ||
|
|
||
| The marker is derived purely from the inject (and optional agent) id, so the injector that sends the | ||
| attack and the collector that validates an AI defense response compute the same value independently, | ||
| without the platform having to store it. It is emitted by the injector (request header + in-prompt | ||
| token) and matched by collectors against guardrail / firewall logs. | ||
| """ | ||
|
|
||
| import hashlib | ||
|
|
||
|
|
||
| def build_marker(inject_id: str, agent_id: str = "") -> str: | ||
| seed = f"{inject_id}:{agent_id}".encode("utf-8") | ||
| return "oaev" + hashlib.sha256(seed).hexdigest()[:16] | ||
|
SamuelHassine marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| from unittest import TestCase, main, mock | ||
|
|
||
|
|
||
| def mock_response(*args, **kwargs): | ||
| class MockResponse: | ||
| def __init__(self): | ||
| self.status_code = 200 | ||
| self.history = None | ||
| self.content = None | ||
| self.headers = {"Content-Type": "application/json"} | ||
|
|
||
| def json(self): | ||
| return {} | ||
|
|
||
| return MockResponse() | ||
|
|
||
|
|
||
| class TestAiTargetManager(TestCase): | ||
| @mock.patch("requests.Session.request", side_effect=mock_response) | ||
| def test_create_posts_to_ai_targets(self, mock_request): | ||
| from pyoaev import OpenAEV | ||
|
|
||
| api_client = OpenAEV("url", "token") | ||
| data = { | ||
| "asset_name": "OpenAI guardrail", | ||
| "ai_target_provider": "openai", | ||
| "ai_target_endpoint": "https://api.openai.com/v1", | ||
| } | ||
|
|
||
| api_client.ai_target.create(data=data) | ||
|
|
||
| _, kwargs = mock_request.call_args | ||
| self.assertEqual(kwargs["method"], "post") | ||
| self.assertEqual(kwargs["url"], "url/api/ai_targets") | ||
| self.assertEqual(kwargs["json"], data) | ||
|
|
||
| @mock.patch("requests.Session.request", side_effect=mock_response) | ||
| def test_get_requests_single_ai_target(self, mock_request): | ||
| from pyoaev import OpenAEV | ||
|
|
||
| api_client = OpenAEV("url", "token") | ||
|
|
||
| api_client.ai_target.get("asset-123") | ||
|
|
||
| _, kwargs = mock_request.call_args | ||
| self.assertEqual(kwargs["method"], "get") | ||
| self.assertEqual(kwargs["url"], "url/api/ai_targets/asset-123") | ||
|
|
||
| @mock.patch("requests.Session.request", side_effect=mock_response) | ||
| def test_update_puts_to_ai_target(self, mock_request): | ||
| from pyoaev import OpenAEV | ||
|
|
||
| api_client = OpenAEV("url", "token") | ||
| new_data = {"asset_description": "updated"} | ||
|
|
||
| api_client.ai_target.update("asset-123", new_data=new_data) | ||
|
|
||
| _, kwargs = mock_request.call_args | ||
| self.assertEqual(kwargs["method"], "put") | ||
| self.assertEqual(kwargs["url"], "url/api/ai_targets/asset-123") | ||
| self.assertEqual(kwargs["json"], new_data) | ||
|
|
||
| @mock.patch("requests.Session.request", side_effect=mock_response) | ||
| def test_delete_calls_delete_on_ai_target(self, mock_request): | ||
| from pyoaev import OpenAEV | ||
|
|
||
| api_client = OpenAEV("url", "token") | ||
|
|
||
| api_client.ai_target.delete("asset-123") | ||
|
|
||
| _, kwargs = mock_request.call_args | ||
| self.assertEqual(kwargs["method"], "delete") | ||
| self.assertEqual(kwargs["url"], "url/api/ai_targets/asset-123") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| import unittest | ||
|
|
||
| from pyoaev.signatures.ai_marker import build_marker | ||
|
|
||
|
|
||
| class TestBuildMarker(unittest.TestCase): | ||
| def test_marker_has_expected_prefix_and_length(self): | ||
| marker = build_marker("inject-1", "agent-1") | ||
|
|
||
| self.assertTrue(marker.startswith("oaev")) | ||
| # "oaev" prefix (4 chars) + 16 hex chars from the sha256 digest. | ||
| self.assertEqual(len(marker), 20) | ||
| self.assertTrue(all(c in "0123456789abcdef" for c in marker[4:])) | ||
|
|
||
| def test_marker_is_deterministic_for_same_inputs(self): | ||
| self.assertEqual( | ||
| build_marker("inject-1", "agent-1"), | ||
| build_marker("inject-1", "agent-1"), | ||
| ) | ||
|
|
||
| def test_marker_differs_for_different_inputs(self): | ||
| self.assertNotEqual( | ||
| build_marker("inject-1", "agent-1"), | ||
| build_marker("inject-2", "agent-1"), | ||
| ) | ||
| self.assertNotEqual( | ||
| build_marker("inject-1", "agent-1"), | ||
| build_marker("inject-1", "agent-2"), | ||
| ) | ||
|
|
||
| def test_agent_id_defaults_to_empty(self): | ||
| self.assertEqual(build_marker("inject-1"), build_marker("inject-1", "")) | ||
|
|
||
| def test_marker_value_is_stable_across_runs(self): | ||
| # Lock the exact value so the injector and collectors (potentially in | ||
| # other languages) stay byte-for-byte compatible. | ||
| self.assertEqual(build_marker("inject-1", "agent-1"), "oaev6457d87cba0698ab") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import unittest | ||
|
|
||
| from pyoaev.signatures.signature_type import SignatureType | ||
| from pyoaev.signatures.types import MatchTypes, SignatureTypes | ||
|
|
||
|
|
||
| class TestAiSignatureTypes(unittest.TestCase): | ||
| def test_ai_signature_type_values(self): | ||
| self.assertEqual( | ||
| SignatureTypes.SIG_TYPE_AI_REQUEST_MARKER.value, "ai_request_marker" | ||
| ) | ||
| self.assertEqual( | ||
| SignatureTypes.SIG_TYPE_AI_TARGET_ENDPOINT.value, "ai_target_endpoint" | ||
| ) | ||
|
|
||
| def test_ai_request_marker_usable_in_signature_type(self): | ||
| signature_type = SignatureType( | ||
| label=SignatureTypes.SIG_TYPE_AI_REQUEST_MARKER, | ||
| match_type=MatchTypes.MATCH_TYPE_SIMPLE, | ||
| ) | ||
|
|
||
| struct = signature_type.make_struct_for_matching(data="oaevdeadbeef") | ||
|
|
||
| self.assertEqual(struct.get("type"), MatchTypes.MATCH_TYPE_SIMPLE.value) | ||
| self.assertEqual(struct.get("data"), "oaevdeadbeef") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.