MAINT: Fold GCG into pyrit/executor/promptgen, remove auxiliary_attacks

.devcontainer/devcontainer.json

@@ -47,7 +47,7 @@
           "**/.pytest_cache/**": true,
           "**/build/**": true,
           "**/dist/**": true,
-          "**/pyrit/auxiliary_attacks/gcg/attack/**": true,
+          "**/pyrit/executor/promptgen/gcg/attack/**": true,
           "**/doc/**": true,
           "**/.ty_cache/**": true,
           "**/frontend/node_modules/**": true,

.pre-commit-config.yaml

@@ -91,5 +91,5 @@ repos:
         entry: uv run --link-mode=copy ty check
         language: system
         files: ^pyrit/
-        exclude: ^pyrit/auxiliary_attacks/
+        exclude: ^pyrit/executor/promptgen/gcg/
         types: [python]

MANIFEST.in

@@ -8,5 +8,5 @@ recursive-include pyrit *.png
 recursive-include pyrit *.wav
 recursive-include pyrit *.mp4
 recursive-include pyrit *.md
-include pyrit/auxiliary_attacks/gcg/src/Dockerfile
+include pyrit/executor/promptgen/gcg/src/Dockerfile
 recursive-include pyrit/backend/frontend *

doc/code/auxiliary_attacks/0_auxiliary_attacks.ipynb → doc/code/executor/gcg/0_gcg.ipynb

@@ -7,7 +7,7 @@
     "lines_to_next_cell": 0
    },
    "source": [
-    "# Auxiliary Attacks"
+    "# Using GCG Adversarial Suffixes"
    ]
   },
   {
@@ -17,17 +17,7 @@
     "lines_to_next_cell": 0
    },
    "source": [
-    "> ⚠️ **Experimental module.** `pyrit.auxiliary_attacks` is experimental: its\n",
-    "> APIs may change in any release without a deprecation cycle. Importing the\n",
-    "> package (or any submodule) emits a `pyrit.exceptions.ExperimentalWarning`.\n",
-    "> Pin pyrit to a specific version if you depend on it. To silence the\n",
-    "> warning:\n",
-    ">\n",
-    "> ```python\n",
-    "> import warnings\n",
-    "> from pyrit.exceptions import ExperimentalWarning\n",
-    "> warnings.filterwarnings(\"ignore\", category=ExperimentalWarning)\n",
-    "> ```"
+    "The greedy coordinate gradient (GCG) [@zou2023gcg] algorithm generates adversarial suffixes that can be appended to a base prompt to jailbreak a language model. PyRIT provides an Azure Machine Learning (AML) pipeline for generating these suffixes."
    ]
   },
   {
@@ -36,25 +26,13 @@
    "metadata": {
     "lines_to_next_cell": 0
    },
-   "source": [
-    "Auxiliary attacks cover a variety of techniques that do not fit into the core PyRIT functionality.\n",
-    "\n",
-    "These attack pipelines may be useful to run before orchestrating other attacks. For example, we provide an Azure Machine Learning (AML) pipeline for generating suffixes using the greedy coordinate gradient (GCG) [@zou2023gcg] algorithm."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3",
-   "metadata": {
-    "lines_to_next_cell": 0
-   },
    "source": [
     "## GCG Suffixes"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "4",
+   "id": "3",
    "metadata": {
     "lines_to_next_cell": 0
    },
@@ -67,7 +45,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "5",
+   "id": "4",
    "metadata": {},
    "outputs": [
     {
@@ -167,7 +145,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6",
+   "id": "5",
    "metadata": {},
    "source": [
     "Next, let's apply a GCG suffix trained on Phi-3-mini to the base prompt using the `SuffixAppendConverter`."
@@ -176,7 +154,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "7",
+   "id": "6",
    "metadata": {},
    "outputs": [
     {
@@ -264,7 +242,8 @@
  ],
  "metadata": {
   "jupytext": {
-   "cell_metadata_filter": "-all"
+   "cell_metadata_filter": "-all",
+   "main_language": "python"
   },
   "language_info": {
    "codemirror_mode": {

doc/code/auxiliary_attacks/0_auxiliary_attacks.py → doc/code/executor/gcg/0_gcg.py

@@ -9,23 +9,9 @@
 #       jupytext_version: 1.17.3
 # ---
 # %% [markdown]
-# # Auxiliary Attacks
+# # Using GCG Adversarial Suffixes
 # %% [markdown]
-# > ⚠️ **Experimental module.** `pyrit.auxiliary_attacks` is experimental: its
-# > APIs may change in any release without a deprecation cycle. Importing the
-# > package (or any submodule) emits a `pyrit.exceptions.ExperimentalWarning`.
-# > Pin pyrit to a specific version if you depend on it. To silence the
-# > warning:
-# >
-# > ```python
-# > import warnings
-# > from pyrit.exceptions import ExperimentalWarning
-# > warnings.filterwarnings("ignore", category=ExperimentalWarning)
-# > ```
-# %% [markdown]
-# Auxiliary attacks cover a variety of techniques that do not fit into the core PyRIT functionality.
-#
-# These attack pipelines may be useful to run before orchestrating other attacks. For example, we provide an Azure Machine Learning (AML) pipeline for generating suffixes using the greedy coordinate gradient (GCG) [@zou2023gcg] algorithm.
+# The greedy coordinate gradient (GCG) [@zou2023gcg] algorithm generates adversarial suffixes that can be appended to a base prompt to jailbreak a language model. PyRIT provides an Azure Machine Learning (AML) pipeline for generating these suffixes.
 # %% [markdown]
 # ## GCG Suffixes
 # %% [markdown]

doc/code/auxiliary_attacks/1_gcg_azure_ml.ipynb → doc/code/executor/gcg/1_gcg_azure_ml.ipynb

@@ -12,23 +12,6 @@
    "cell_type": "markdown",
    "id": "1",
    "metadata": {},
-   "source": [
-    "> ⚠️ **Experimental module.** `pyrit.auxiliary_attacks` is experimental: its\n",
-    "> APIs may change in any release without a deprecation cycle. Importing the\n",
-    "> package below emits a `pyrit.exceptions.ExperimentalWarning`. Pin pyrit to\n",
-    "> a specific version if you depend on it. To silence the warning:\n",
-    ">\n",
-    "> ```python\n",
-    "> import warnings\n",
-    "> from pyrit.exceptions import ExperimentalWarning\n",
-    "> warnings.filterwarnings(\"ignore\", category=ExperimentalWarning)\n",
-    "> ```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2",
-   "metadata": {},
    "source": [
     "This notebook shows how to generate GCG [@zou2023gcg] suffixes using Azure Machine Learning (AML), which consists of three main steps:\n",
     "1. Connect to an Azure Machine Learning (AML) workspace.\n",
@@ -38,15 +21,15 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3",
+   "id": "2",
    "metadata": {},
    "source": [
     "## Connect to Azure Machine Learning Workspace"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "4",
+   "id": "3",
    "metadata": {},
    "source": [
     "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning (AML), providing a centralized place to work with all the artifacts you create when using AML. In this section, we will connect to the workspace in which the job will be run.\n",
@@ -57,7 +40,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "5",
+   "id": "4",
    "metadata": {},
    "outputs": [
     {
@@ -86,7 +69,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6",
+   "id": "5",
    "metadata": {},
    "source": [
     "The Azure ML SDK emits a fair amount of telemetry to stderr that looks\n",
@@ -101,7 +84,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "7",
+   "id": "6",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -115,7 +98,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "8",
+   "id": "7",
    "metadata": {},
    "outputs": [
     {
@@ -135,32 +118,32 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9",
+   "id": "8",
    "metadata": {},
    "source": [
     "## Create AML Environment"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "10",
+   "id": "9",
    "metadata": {},
    "source": [
     "To install the dependencies needed to run GCG, we create an AML environment from a\n",
-    "[Dockerfile](../../../pyrit/auxiliary_attacks/gcg/src/Dockerfile). The Dockerfile uses\n",
+    "[Dockerfile](https://github.com/microsoft/PyRIT/blob/main/pyrit/executor/promptgen/gcg/src/Dockerfile). The Dockerfile uses\n",
     "an NVIDIA CUDA base image with Python 3.11 and installs PyRIT with the `gcg` extra."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "11",
+   "id": "10",
    "metadata": {},
    "outputs": [
     {
      "data": {
       "text/plain": [
-       "Environment({'arm_type': 'environment_version', 'latest_version': None, 'image': None, 'intellectual_property': None, 'is_anonymous': False, 'auto_increment_version': False, 'auto_delete_setting': None, 'name': 'pyrit-gcg', 'description': 'PyRIT GCG environment: CUDA 12.1 + Python 3.11 + pip install -e .[gcg]', 'tags': {'Owner': 'unknown'}, 'properties': {'azureml.labels': 'latest'}, 'print_as_yaml': False, 'id': '/subscriptions/db1ba766-2ca3-42c6-a19a-0f0d43134a8c/resourceGroups/gcg-romanlutz/providers/Microsoft.MachineLearningServices/workspaces/gcg-romanlutz/environments/pyrit-gcg/versions/17', 'Resource__source_path': '', 'base_path': './git/copilot-worktrees/PyRIT/romanlutz-upgraded-barnacle/doc/code/auxiliary_attacks', 'creation_context': <azure.ai.ml.entities._system_data.SystemData object at 0x000002BBB8E0F770>, 'serialize': <msrest.serialization.Serializer object at 0x000002BBB9129320>, 'version': '17', 'conda_file': None, 'build': <azure.ai.ml.entities._assets.environment.BuildContext object at 0x000002BBB915FB10>, 'inference_config': None, 'os_type': 'Linux', 'conda_file_path': None, 'path': None, 'datastore': None, 'upload_hash': None, 'translated_conda_file': None})"
+       "Environment({'arm_type': 'environment_version', 'latest_version': None, 'image': None, 'intellectual_property': None, 'is_anonymous': False, 'auto_increment_version': False, 'auto_delete_setting': None, 'name': 'pyrit-gcg', 'description': 'PyRIT GCG environment: CUDA 12.1 + Python 3.11 + pip install -e .[gcg]', 'tags': {'Owner': 'unknown'}, 'properties': {'azureml.labels': 'latest'}, 'print_as_yaml': False, 'id': '/subscriptions/db1ba766-2ca3-42c6-a19a-0f0d43134a8c/resourceGroups/gcg-romanlutz/providers/Microsoft.MachineLearningServices/workspaces/gcg-romanlutz/environments/pyrit-gcg/versions/17', 'Resource__source_path': '', 'base_path': './git/copilot-worktrees/PyRIT/romanlutz-upgraded-barnacle/doc/code/executor/gcg', 'creation_context': <azure.ai.ml.entities._system_data.SystemData object at 0x000002BBB8E0F770>, 'serialize': <msrest.serialization.Serializer object at 0x000002BBB9129320>, 'version': '17', 'conda_file': None, 'build': <azure.ai.ml.entities._assets.environment.BuildContext object at 0x000002BBB915FB10>, 'inference_config': None, 'os_type': 'Linux', 'conda_file_path': None, 'path': None, 'datastore': None, 'upload_hash': None, 'translated_conda_file': None})"
       ]
      },
      "execution_count": null,
@@ -180,7 +163,7 @@
     "env_docker_context = Environment(\n",
     "    build=BuildContext(\n",
     "        path=Path(HOME_PATH),\n",
-    "        dockerfile_path=\"pyrit/auxiliary_attacks/gcg/src/Dockerfile\",\n",
+    "        dockerfile_path=\"pyrit/executor/promptgen/gcg/src/Dockerfile\",\n",
     "    ),\n",
     "    name=\"pyrit-gcg\",\n",
     "    description=\"PyRIT GCG environment: CUDA 12.1 + Python 3.11 + pip install -e .[gcg]\",\n",
@@ -192,27 +175,27 @@
   },
   {
    "cell_type": "markdown",
-   "id": "12",
+   "id": "11",
    "metadata": {},
    "source": [
     "## Submit Training Job to AML"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "13",
+   "id": "12",
    "metadata": {},
    "source": [
     "Finally, we configure the command to run the GCG algorithm. The entry point is\n",
-    "[`pyrit.auxiliary_attacks.gcg.experiments.run`](../../../pyrit/auxiliary_attacks/gcg/experiments/run.py),\n",
+    "[`pyrit.executor.promptgen.gcg.experiments.run`](https://github.com/microsoft/PyRIT/blob/main/pyrit/executor/promptgen/gcg/experiments/run.py),\n",
     "invoked as a module so the uploaded code snapshot takes priority over the\n",
     "Docker-installed package (Python's `-m` flag puts the cwd at the front of `sys.path`).\n",
     "\n",
     "The new public API takes a typed ``GCGConfig`` (strategy) and a separate\n",
     "``GCGDataConfig`` (CSV paths/counts). We build both locally with whatever\n",
     "overrides we want, serialize each into a JSON file the AML job can read as\n",
     "an input, and ship those paths through the job command. Defaults come from\n",
-    "the dataclasses in ``pyrit.auxiliary_attacks.gcg.config``; goals and targets\n",
+    "the dataclasses in ``pyrit.executor.promptgen.gcg.config``; goals and targets\n",
     "flow into ``GCGGenerator.execute_async`` at runtime, not through the config.\n",
     "\n",
     "We also have to specify a GPU compute target. In our experience, a GPU instance with\n",
@@ -226,22 +209,13 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "14",
+   "id": "13",
    "metadata": {},
-   "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "./AppData/Local/Temp/ipykernel_95196/4292719205.py:3: ExperimentalWarning: pyrit.auxiliary_attacks is experimental: APIs may change in any release without a deprecation cycle. Pin pyrit to a specific version if you depend on this module. To silence: warnings.filterwarnings('ignore', category=pyrit.exceptions.ExperimentalWarning).\n",
-      "  from pyrit.auxiliary_attacks.gcg import (\n"
-     ]
-    }
-   ],
+   "outputs": [],
    "source": [
     "import tempfile\n",
     "\n",
-    "from pyrit.auxiliary_attacks.gcg import (\n",
+    "from pyrit.executor.promptgen.gcg import (\n",
     "    GCGAlgorithmConfig,\n",
     "    GCGConfig,\n",
     "    GCGDataConfig,\n",
@@ -271,7 +245,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "15",
+   "id": "14",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -280,7 +254,7 @@
     "job = command(\n",
     "    code=Path(HOME_PATH),\n",
     "    command=(\n",
-    "        \"python -m pyrit.auxiliary_attacks.gcg.experiments.run\"\n",
+    "        \"python -m pyrit.executor.promptgen.gcg.experiments.run\"\n",
     "        \" --config ${{inputs.config}}\"\n",
     "        \" --data ${{inputs.data}}\"\n",
     "        \" --output-dir ${{outputs.results}}\"\n",
@@ -302,7 +276,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "16",
+   "id": "15",
    "metadata": {},
    "outputs": [
     {
@@ -373,7 +347,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "17",
+   "id": "16",
    "metadata": {},
    "source": [
     "## Wait for the Job to Complete and Inspect the Generated Suffix\n",
@@ -394,7 +368,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "18",
+   "id": "17",
    "metadata": {},
    "outputs": [
     {
@@ -490,7 +464,8 @@
  ],
  "metadata": {
   "jupytext": {
-   "cell_metadata_filter": "-all"
+   "cell_metadata_filter": "-all",
+   "main_language": "python"
   },
   "language_info": {
    "codemirror_mode": {