Centralize Socket validation and update roadmap completion records

.github/workflows/validate-socket-metadata.yml

@@ -1,4 +1,4 @@
-name: Validate Socket Metadata
+name: Validate Socket
 
 on:
   push:
@@ -22,5 +22,5 @@ jobs:
       - name: Sync root maintainer environment
         run: uv sync --dev
 
-      - name: Validate root marketplace wiring
-        run: uv run scripts/validate_socket_metadata.py
+      - name: Run Socket compatibility validation
+        run: uv run scripts/validate_socket.py --profile compatibility

AGENTS.md

@@ -95,7 +95,7 @@ uv sync --dev
 ### Validation
 
 ```bash
-uv run scripts/validate_socket_metadata.py
+uv run scripts/validate_socket.py --profile compatibility
 ```
 
 ### Optional Project Commands

CONTRIBUTING.md

@@ -31,7 +31,7 @@ Use the root repository for work about:
 - repo-root marketplace wiring in [`.agents/plugins/marketplace.json`](./.agents/plugins/marketplace.json)
 - root maintainer docs under [`docs/`](./docs/)
 - root policies in [README.md](./README.md), [AGENTS.md](./AGENTS.md), and [ROADMAP.md](./ROADMAP.md)
-- root validation and CI such as [`scripts/validate_socket_metadata.py`](./scripts/validate_socket_metadata.py) and [`.github/workflows/validate-socket-metadata.yml`](./.github/workflows/validate-socket-metadata.yml)
+- root validation and CI such as [`scripts/validate_socket.py`](./scripts/validate_socket.py) and [`.github/workflows/validate-socket-metadata.yml`](./.github/workflows/validate-socket-metadata.yml)
 - coordinated child-skill guidance that needs one consistent policy across multiple monorepo-owned plugin or skills repositories
 
 If the change is really about one child repository's own skills, packaging, tests, or release flow, start in that child repository's docs and workflow instead of treating `socket` as a generic catch-all.
@@ -97,14 +97,16 @@ The root validation path does not require application secrets. If your change in
 - the `uv` dev environment is synced
 - the root marketplace file is valid JSON
 - the root packaged plugin paths still point at real installable plugin surfaces
-- the root validator completes successfully
+- the core Socket validation profile completes successfully
 
-You can verify that baseline with:
+Use the compatibility profile for ordinary root changes. It runs the core marketplace, skill-metadata, root-test, type, and lint checks plus the checked-in Hermes and Claude compatibility checks:
 
 ```bash
-uv run scripts/validate_socket_metadata.py
+uv run scripts/validate_socket.py --profile compatibility
 ```
 
+Use `--profile full` when the change affects child validation or behavior; it runs each participating child suite once from its owning project. The `--profile release --version X.Y.Z` profile additionally runs the read-only release-ready gate and is for reviewed `main` only.
+
 Every new or materially changed Socket plugin, skill, or MCP declaration needs
 an explicit Hermes compatibility outcome in the same pass. When that outcome
 changes the checked-in Hermes skill tap, regenerate and validate it:
@@ -250,9 +252,7 @@ Root baseline validation:
 
 ```bash
 uv sync --dev
-uv run mypy
-uv run scripts/validate_socket_metadata.py
-uv run scripts/validate_claude_compatibility.py
+uv run scripts/validate_socket.py --profile compatibility
 ```
 
 When the change intentionally bumps released version numbers across the superproject, inventory or update the maintained manifest surfaces with:

docs/maintainers/release-modes.md

@@ -80,14 +80,12 @@ After a version bump lands on `main`, run the executable pre-tag gate:
 scripts/release.sh release-ready X.Y.Z
 ```
 
-`release-ready` verifies version, clean-main, tag, and subtree accounting. When
-the release touches the Hermes skill tap, run its explicit content validator
-alongside the root marketplace validator before this gate:
+`release-ready` verifies version, clean-main, tag, and subtree accounting. Run
+the consolidated full profile before this gate; it covers root, compatibility,
+and participating child validation without duplicating child suite ownership:
 
 ```bash
-uv run scripts/validate_socket_metadata.py
-uv run scripts/validate_hermes_compatibility.py
-uv run scripts/validate_claude_compatibility.py
+uv run scripts/validate_socket.py --profile full
 ```
 
 Then capture evidence from that exact commit and generate the release-note
@@ -143,9 +141,7 @@ Before opening or merging the `socket` release PR:
 Before tagging `socket`:
 
 - confirm the subtree policy table above was followed
-- run `uv run scripts/validate_socket_metadata.py`
-- when the release touches the Claude marketplace, Claude inventory, or Claude MCP adapters, also run `uv run scripts/validate_claude_compatibility.py`
-- when the release touches the Hermes skill tap, also run `uv run scripts/validate_hermes_compatibility.py`
+- run `uv run scripts/validate_socket.py --profile full`
 - confirm local `main` is fast-forwarded to `origin/main`
 - run `scripts/release.sh release-ready X.Y.Z`
 - run `scripts/release.sh release-evidence`

docs/maintainers/tvos-skills-plan.md

@@ -7,8 +7,8 @@ general SwiftUI, AVFoundation, or model-runtime skills into tvOS catch-alls.
 
 ## Status
 
-Implemented on the `docs/tvos-skills-plan` release branch for Socket 9.23.0;
-the protected-main publication steps remain pending.
+Completed and released in Socket v9.23.0. This document remains as the
+implementation and evidence record for the two shipped workflows.
 
 ## Decision
 

plugins/model-lab-skills/skills/ablate-refusal-representations/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Ablate Refusal Representations"
   short_description: "Research refusal-direction ablation with explicit controls."
-  default_prompt: "Design and evaluate refusal-representation ablation as controlled model-internals research."
+  default_prompt: "Use $ablate-refusal-representations to design and evaluate refusal-representation ablation as controlled model-internals research."

plugins/model-lab-skills/skills/benchmark-model-runtime/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Benchmark Model Runtime"
   short_description: "Benchmark model runtimes fairly across deployment targets."
-  default_prompt: "Benchmark model runtimes using controlled artifacts, workloads, thermal state, memory, latency, and energy evidence."
+  default_prompt: "Use $benchmark-model-runtime to benchmark model runtimes using controlled artifacts, workloads, thermal state, memory, latency, and energy evidence."

plugins/model-lab-skills/skills/choose-apple-model-runtime/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Choose Apple Model Runtime"
   short_description: "Route Apple model authoring, conversion, and deployment choices."
-  default_prompt: "Choose among Core AI, Core ML, MLX, ExecuTorch, and Foundation Models for an Apple model workflow."
+  default_prompt: "Use $choose-apple-model-runtime to choose among Core AI, Core ML, MLX, ExecuTorch, and Foundation Models for an Apple model workflow."

plugins/model-lab-skills/skills/choose-model-lab-workflow/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Choose Model Lab Workflow"
   short_description: "Route model training, research, evaluation, and runtime tasks."
-  default_prompt: "Choose the appropriate Model Lab workflow and explain the evidence boundary."
+  default_prompt: "Use $choose-model-lab-workflow to choose the appropriate Model Lab workflow and explain the evidence boundary."

plugins/model-lab-skills/skills/compare-model-checkpoints/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Compare Model Checkpoints"
   short_description: "Compare model checkpoints without hiding tradeoffs."
-  default_prompt: "Compare model checkpoints across quality, behavior, provenance, size, and runtime evidence."
+  default_prompt: "Use $compare-model-checkpoints to compare model checkpoints across quality, behavior, provenance, size, and runtime evidence."

plugins/model-lab-skills/skills/design-model-experiment/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Design Model Experiment"
   short_description: "Define a reproducible model experiment before spending compute."
-  default_prompt: "Design a reproducible model experiment with hypotheses, controls, budgets, provenance, and stop conditions."
+  default_prompt: "Use $design-model-experiment to design a reproducible model experiment with hypotheses, controls, budgets, provenance, and stop conditions."

plugins/model-lab-skills/skills/evaluate-jailbreak-resilience/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Evaluate Jailbreak Resilience"
   short_description: "Evaluate authorized jailbreak and prompt-injection resilience."
-  default_prompt: "Build an authorized jailbreak-resilience evaluation with bounded targets, metrics, and disclosure-safe evidence."
+  default_prompt: "Use $evaluate-jailbreak-resilience to build an authorized jailbreak-resilience evaluation with bounded targets, metrics, and disclosure-safe evidence."

plugins/model-lab-skills/skills/evaluate-language-model/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Evaluate Language Model"
   short_description: "Build repeatable capability, behavior, and safety evaluations."
-  default_prompt: "Design and run a repeatable language-model evaluation with scoped metrics and regression evidence."
+  default_prompt: "Use $evaluate-language-model to design and run a repeatable language-model evaluation with scoped metrics and regression evidence."

plugins/model-lab-skills/skills/evaluate-tool-calling-model/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Evaluate Tool-Calling Model"
   short_description: "Evaluate tool selection, arguments, execution, and recovery."
-  default_prompt: "Evaluate a tool-calling model or harness across selection, schema, execution, recovery, and side-effect controls."
+  default_prompt: "Use $evaluate-tool-calling-model to evaluate a tool-calling model or harness across selection, schema, execution, recovery, and side-effect controls."

plugins/model-lab-skills/skills/fine-tune-language-model/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Fine-Tune Language Model"
   short_description: "Plan and run reproducible adapter or full fine-tuning."
-  default_prompt: "Choose and execute a reproducible language-model fine-tuning workflow with explicit resource and validation gates."
+  default_prompt: "Use $fine-tune-language-model to choose and execute a reproducible language-model fine-tuning workflow with explicit resource and validation gates."

plugins/model-lab-skills/skills/prepare-language-model-dataset/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Prepare Language Model Dataset"
   short_description: "Prepare auditable training and evaluation datasets."
-  default_prompt: "Prepare and document a language-model dataset with splits, provenance, quality checks, and leakage controls."
+  default_prompt: "Use $prepare-language-model-dataset to prepare and document a language-model dataset with splits, provenance, quality checks, and leakage controls."

plugins/model-lab-skills/skills/research-model-representations/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Research Model Representations"
   short_description: "Design causal representation and activation research."
-  default_prompt: "Design a reproducible language-model representation study with causal controls and interpretable evidence."
+  default_prompt: "Use $research-model-representations to design a reproducible language-model representation study with causal controls and interpretable evidence."

plugins/model-lab-skills/skills/steer-language-model-behavior/agents/openai.yaml

@@ -1,4 +1,4 @@
 interface:
   display_name: "Steer Language Model Behavior"
   short_description: "Implement and evaluate controlled model behavior steering."
-  default_prompt: "Choose, implement, and evaluate a model steering intervention without conflating behavior change with understanding."
+  default_prompt: "Use $steer-language-model-behavior to choose, implement, and evaluate a model steering intervention without conflating behavior change with understanding."

pyproject.toml

@@ -17,9 +17,6 @@ dev = [
 [tool.pytest.ini_options]
 testpaths = [
   "tests",
-  "plugins/agent-portability-skills/skills/bootstrap-skills-plugin-repo/tests",
-  "plugins/agent-portability-skills/skills/sync-skills-repo-guidance/tests",
-  "plugins/repository-skills/skills/maintain-project-repo/tests",
   "plugins/model-lab-skills/skills/design-model-experiment/scripts",
   "plugins/model-lab-skills/skills/evaluate-language-model/scripts",
   "plugins/model-lab-skills/skills/compare-model-checkpoints/scripts",
@@ -29,10 +26,6 @@ testpaths = [
 files = [
   "scripts",
   "tests",
-  "plugins/agent-portability-skills/skills/bootstrap-skills-plugin-repo/scripts",
-  "plugins/agent-portability-skills/skills/bootstrap-skills-plugin-repo/tests",
-  "plugins/agent-portability-skills/skills/sync-skills-repo-guidance/scripts",
-  "plugins/agent-portability-skills/skills/sync-skills-repo-guidance/tests",
   "plugins/repository-skills/skills/maintain-project-repo/scripts",
   "plugins/repository-skills/skills/maintain-project-repo/tests",
   "plugins/model-lab-skills/skills/design-model-experiment/scripts",

scripts/release_version.py

@@ -147,7 +147,7 @@ def utc_timestamp() -> str:
 
 
 def run_local_marketplace_smoke(root: Path) -> dict[str, Any]:
-    with tempfile.TemporaryDirectory(prefix="socket-codex-home.", dir="/private/tmp") as temp_dir:
+    with tempfile.TemporaryDirectory(prefix="socket-codex-home.") as temp_dir:
         codex_home = Path(temp_dir)
         command_env = os.environ.copy()
         command_env["CODEX_HOME"] = str(codex_home)
@@ -660,7 +660,7 @@ def release_notes(
         "## Migration/upgrade notes\n\n"
         "- Run `codex plugin marketplace upgrade socket` to refresh a local Codex install.\n\n"
         "## Verification performed\n\n"
-        "- Ran `uv run scripts/validate_socket_metadata.py`.\n"
+        "- Ran `uv run scripts/validate_socket.py --profile full`.\n"
         "- Ran `scripts/release.sh release-ready "
         f"{version}`.\n"
         f"{render_evidence_summary(evidence)}"
@@ -830,8 +830,8 @@ def render_patch_refresh(root: Path, targets: list[VersionTarget], *, allow_unme
         for path in unchanged_files:
             print(f"- {path}")
 
-    print("Validating root marketplace metadata...")
-    run_command(root, ["uv", "run", "scripts/validate_socket_metadata.py"])
+    print("Running full Socket validation...")
+    run_command(root, ["uv", "run", "scripts/validate_socket.py", "--profile", "full"])
     print("Committing patch version bump...")
     run_git(root, ["add", *changed_files])
     run_git(root, ["commit", "-m", f"release: bump socket patch to {desired_version}"])