Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .claude/commands/gm-babysit.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: Check on all tracked tasks for an immediate status update

Check on all tasks you're currently tracking. Use this for an immediate status update.

Note: Task events normally arrive automatically via the taskyou channel. This command is for a manual spot-check when you want an immediate snapshot.

First, load the project configuration:
```bash
source ./config.env
Expand All @@ -29,8 +31,6 @@ source ./config.env
- **Blocked**: Explain what's blocking it. Suggest next steps (retry, send input, review output).
- **Still processing**: Note it's still running — no action needed unless it's been unusually long.

5. **Re-launch the background notification watcher** if there are tasks still in progress and no background agent is currently watching.

6. **If all tracked tasks are done**, let the user know there's nothing left to monitor.
5. **If all tracked tasks are done**, let the user know there's nothing left to monitor.

Keep updates brief — one line per task.
89 changes: 87 additions & 2 deletions .claude/commands/gm-doctor.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,91 @@ If `config.env` is missing, report FAIL — the plugin commands won't work witho

---

### Check 7: Security Audit
### Check 7: Task Event Channel

Check that the Claude Code channel for push-based task notifications is set up and working.

**Steps:**

1. Check if the channel files exist in the GM directory:
```bash
test -f "$LOCAL_PROJECT_DIR/channel/taskyou-channel.ts" && echo "CHANNEL_EXISTS" || echo "CHANNEL_MISSING"
test -f "$LOCAL_PROJECT_DIR/.mcp.json" && echo "MCP_JSON_EXISTS" || echo "MCP_JSON_MISSING"
test -d "$LOCAL_PROJECT_DIR/channel/node_modules" && echo "DEPS_INSTALLED" || echo "DEPS_MISSING"
```

2. **If channel files are missing**, deploy them from the plugin templates:

a. Find the TaskYou-OS plugin directory (or repo checkout):
```bash
TASKYOU_OS_DIR=""
if [ -f "./templates/channel/taskyou-channel.ts.tmpl" ]; then
TASKYOU_OS_DIR="."
else
PLUGIN_DIR=$(python3 -c "import json; d=json.load(open('$HOME/.claude/plugins/installed_plugins.json')); entries=d.get('plugins',{}).get('taskyou-os@taskyou-os',[]); print(entries[0]['installPath'] if entries else '')" 2>/dev/null)
if [ -n "$PLUGIN_DIR" ] && [ -f "$PLUGIN_DIR/templates/channel/taskyou-channel.ts.tmpl" ]; then
TASKYOU_OS_DIR="$PLUGIN_DIR"
fi
fi
```

b. If templates are available, render and deploy them. Use `config.env` to substitute variables:
```bash
source "$LOCAL_PROJECT_DIR/config.env"
mkdir -p "$LOCAL_PROJECT_DIR/channel"
```
- Read `$TASKYOU_OS_DIR/templates/channel/taskyou-channel.ts.tmpl`, substitute `{{SERVER_HOST}}` and `{{SERVER_HOME}}` with values from config.env, write to `$LOCAL_PROJECT_DIR/channel/taskyou-channel.ts`
- Read `$TASKYOU_OS_DIR/templates/channel/package.json.tmpl`, write to `$LOCAL_PROJECT_DIR/channel/package.json`
- Read `$TASKYOU_OS_DIR/templates/mcp.json.tmpl`, write to `$LOCAL_PROJECT_DIR/.mcp.json`

c. Install dependencies:
```bash
cd "$LOCAL_PROJECT_DIR/channel" && bun install --silent
```

d. Report WARN: "Deployed task event channel. Restart Claude Code to activate."

3. **If channel files exist but deps are missing**, install them:
```bash
cd "$LOCAL_PROJECT_DIR/channel" && bun install --silent
```
Report WARN: "Installed missing channel dependencies."

4. **If channel exists, check for drift** — compare deployed channel against the plugin template (same approach as nono drift detection in Check 9). If the template is newer, update the deployed file and report WARN.

5. **Check the shell alias** includes `--dangerously-load-development-channels server:taskyou`:
```bash
grep "$GM_ALIAS" ~/.zshrc 2>/dev/null || grep "$GM_ALIAS" ~/.bashrc 2>/dev/null
```
- If the alias exists but doesn't include `--dangerously-load-development-channels server:taskyou`, report WARN and show the user the updated alias line they should use:
```
alias <GM_ALIAS>='cd <LOCAL_PROJECT_DIR> && CLAUDE_CONFIG_DIR=<CONFIG_DIR> claude --dangerously-load-development-channels server:taskyou'
```
- If the alias already includes the flag, report PASS.

6. **Check the CLAUDE.md** has the channel-based monitoring section (not the old background-agent approach):
```bash
grep -c "Task event channel" "$LOCAL_PROJECT_DIR/CLAUDE.md"
grep -c "background monitoring agent" "$LOCAL_PROJECT_DIR/CLAUDE.md"
```
- If it has "background monitoring agent" but not "Task event channel", the CLAUDE.md needs updating. Render the Task Tracking section from the template and show the user the diff, offering to update it.

7. **Runtime self-check** — confirm the channel actually boots and completes the MCP handshake (catches a broken render, a missing/corrupt dep, or a bad bun). The smoke test ships next to the channel:
```bash
test -f "$LOCAL_PROJECT_DIR/channel/smoke-test.ts" && \
( cd "$LOCAL_PROJECT_DIR/channel" && timeout 30 bun run smoke-test.ts )
```
- Exit 0 (ends with `PASSED`): report PASS "channel boots + handshakes."
- Non-zero or `FAILED`: report WARN with the smoke-test output — the channel is registered but won't push events until fixed.
- If `smoke-test.ts` is absent (GM predates this), copy it from `templates/channel/smoke-test.ts` in the plugin and re-run.

**If all channel files exist, deps installed, alias correct, and the smoke test passes:** Report PASS with "Task event channel active."
**If deployed or fixed anything:** Report WARN with summary.
**If templates not found:** Report FAIL with "Channel templates not found. Update the TaskYou-OS plugin first."

---

### Check 8: Security Audit

Run the server-side security audit script to check credentials, permissions, and exposed services.

Expand Down Expand Up @@ -340,7 +424,7 @@ ssh -o ConnectTimeout=5 "$SERVER_HOST" '$HOME/.local/bin/audit.sh' 2>/dev/null

---

## Check 8: Credential Isolation (nono)
## Check 9: Credential Isolation (nono)

This check verifies if nono is set up, and if not, strongly recommends it. Always run this check regardless of whether credentials are currently configured.

Expand Down Expand Up @@ -441,6 +525,7 @@ TaskYou-OS Doctor
Daemon mode PASS/WARN/FAIL
Executor health PASS/WARN/FAIL
GM templates PASS/WARN/FAIL
Task event channel PASS/WARN/FAIL
Security audit PASS/WARN/FAIL
Credential isolation PASS/WARN
─────────────────────────────────
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/qa.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: QA harness

# Validates the generated GM system end-to-end: renders the real setup.sh into a
# throwaway $HOME, smoke-tests the channel MCP server, and drives a real hook all
# the way to a channel push. Hermetic — nothing touches the runner's home beyond
# the sandbox dir. ty is not installed; its assertions are skipped (CI mode).

on:
pull_request:
paths:
- "setup.sh"
- "templates/channel/**"
- "templates/hooks/**"
- "templates/mcp.json.tmpl"
- "qa/**"
- ".github/workflows/qa.yml"
push:
branches: [main]
workflow_dispatch:

jobs:
qa:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # run-qa.sh creates a worktree of HEAD
- uses: oven-sh/setup-bun@v2
- name: Run QA harness
run: bash qa/run-qa.sh
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ This is the TaskYouOS template repo. It generates fully working AI agent managem

- Templates are the source of truth. Never edit generated output — edit the template.
- Test changes by running `./setup.sh local /tmp/test-project` with a test config.env.
- For channel/hook/setup changes, run the hermetic QA harness: `qa/run-qa.sh` (sandboxed `$HOME`, never touches your real ty/GMs). See `qa/README.md`.
- If you have an existing GM at `~/Projects/gms/<name>/`, use it as a reference implementation.

## Adding New Templates
Expand Down
38 changes: 38 additions & 0 deletions qa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# QA harness

Hermetic, non-destructive end-to-end tests for the GM system that `setup.sh`
generates. It runs the **real** `setup.sh` against a throwaway `$HOME` and a git
worktree of a branch, then exercises the task-event channel all the way through.

It never touches your real `~/.local/share/task`, `~/Library/Application Support/task`,
`~/.gitconfig`, your GMs, or your running daemon — everything keys off a sandbox
`$HOME`, which is removed on exit.

## Run

```bash
qa/run-qa.sh # tests the current checkout (HEAD)
qa/run-qa.sh pr-31 # tests a specific local branch/ref
```

Requires `bun` and `python3`. `ty` is optional — when it's absent (CI), the ty
project-registration assertions are skipped; render + smoke + notification
checks still run.

## What it covers

1. **`setup.sh local`** — renders the GM + channel; asserts files, deps, alias
flag, no unresolved `{{placeholders}}`.
2. **Channel smoke test** — MCP handshake, capabilities, and tool list
(`templates/channel/smoke-test.ts`).
3. **`setup.sh server` (local mode)** — hooks land in the OS-correct dir
(the macOS `~/Library/Application Support` fix), `notifications.jsonl` is
created, and the project registers in the *sandbox* ty.
4. **Notification e2e** — fires the real `task.completed` hook and asserts the
channel pushes a matching `notifications/claude/channel` event. Two modes:
- `steady` — pre-existing backlog is **not** replayed, next event delivered.
- `cold` — first event after an **empty-file** start **is** delivered
(regression test for the cold-start fix).

CI runs this on every PR touching `setup.sh`, `templates/channel`,
`templates/hooks`, or `qa/` (see `.github/workflows/qa.yml`).
160 changes: 160 additions & 0 deletions qa/channel-notify-test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
#!/usr/bin/env bun
// Channel notification end-to-end test.
//
// Unlike smoke-test.ts (handshake + capabilities only), this exercises the full
// chain the feature actually depends on:
//
// rendered hook → notifications.jsonl → channel runner (local/SSH) → push
//
// It spawns the *rendered* channel server, does the MCP handshake, then fires the
// real rendered task.completed hook and asserts the channel pushes a matching
// `notifications/claude/channel` event back out over stdio.
//
// Driven by env vars (set by run-qa.sh):
// CHANNEL_TS absolute path to the rendered taskyou-channel.ts
// NOTIF_FILE absolute path to notifications.jsonl (== {{SERVER_HOME}}/notifications.jsonl)
// HOOK_SCRIPT absolute path to the rendered task.completed hook
// MODE "steady" (default): start with a pre-existing line, assert the
// new event is delivered and the pre-existing backlog is NOT replayed.
// "cold": start from an EMPTY file, assert the very first event
// is still delivered (regression test for the cold-start fix).
// POLL_WAIT_MS optional override for how long we wait for a poll cycle (default 14000)
//
// Exit 0 = PASS, non-zero = FAIL.

import { spawn, spawnSync } from "child_process";
import { appendFileSync, existsSync } from "fs";

const CHANNEL_TS = process.env.CHANNEL_TS!;
const NOTIF_FILE = process.env.NOTIF_FILE!;
const HOOK_SCRIPT = process.env.HOOK_SCRIPT!;
const POLL_WAIT_MS = parseInt(process.env.POLL_WAIT_MS || "14000", 10);
const MODE = process.env.MODE === "cold" ? "cold" : "steady";

const TASK_ID = "qa-9999";

function die(msg: string): never {
console.log(`✗ ${msg}`);
console.log("\nFAILED");
process.exit(1);
}

for (const [k, v] of Object.entries({ CHANNEL_TS, NOTIF_FILE, HOOK_SCRIPT })) {
if (!v || !existsSync(v)) die(`${k} missing or not found: ${v}`);
}

// 1. steady mode: seed one pre-existing line, so we can assert the channel skips
// the startup backlog (must NOT replay it) yet still delivers the next event.
// cold mode: leave the file empty, so we assert the very first event written
// after startup IS delivered (the bug the `initialized` fix closes).
if (MODE === "steady") {
appendFileSync(
NOTIF_FILE,
JSON.stringify({
event: "completed",
task_id: "qa-seed",
title: "seed (must not be replayed)",
project: "qa-test",
timestamp: new Date(0).toISOString(),
}) + "\n"
);
}

// 2. Spawn the rendered channel server.
const proc = spawn("bun", ["run", CHANNEL_TS], {
stdio: ["pipe", "pipe", "pipe"],
});

let pushed: any = null;
let sawSeedReplay = false;
let buf = "";

proc.stdout.on("data", (d: Buffer) => {
buf += d.toString();
const lines = buf.split("\n");
buf = lines.pop() || "";
for (const line of lines) {
if (!line.trim()) continue;
let msg: any;
try {
msg = JSON.parse(line);
} catch {
continue;
}
if (msg.method === "notifications/claude/channel") {
const meta = msg.params?.meta || {};
if (meta.task_id === "qa-seed") sawSeedReplay = true;
if (meta.task_id === TASK_ID) pushed = msg;
}
}
});

let stderr = "";
proc.stderr.on("data", (d: Buffer) => (stderr += d.toString()));

const fail = (msg: string) => {
console.log(`✗ ${msg}`);
if (stderr.trim()) console.log("channel stderr:", stderr.trim());
console.log("\nFAILED");
proc.kill();
process.exit(1);
};

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

(async () => {
// 3. MCP initialize handshake.
proc.stdin.write(
JSON.stringify({
jsonrpc: "2.0",
id: 1,
method: "initialize",
params: {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "channel-notify-test", version: "0.0.1" },
},
}) + "\n"
);

// 4. Let the startup poll record its position (lastLineCount = 1 from the seed).
await sleep(2500);

// 5. Fire the REAL rendered hook — appends a new line to notifications.jsonl.
const r = spawnSync("bash", [HOOK_SCRIPT], {
env: {
...process.env,
TASK_ID,
TASK_TITLE: 'QA notify "end to end"', // embedded quotes test JSON escaping
TASK_PROJECT: "qa-test",
},
encoding: "utf8",
});
if (r.status !== 0) fail(`hook exited ${r.status}: ${r.stderr}`);

// 6. Wait for the next poll cycle to detect + push the new line.
await sleep(POLL_WAIT_MS);

if (MODE === "steady" && sawSeedReplay)
fail("channel REPLAYED the pre-existing seed line on startup (should not)");
if (!pushed)
fail(
MODE === "cold"
? `cold start: first event (${TASK_ID}) was swallowed — the initialized fix is missing`
: `no channel push for task ${TASK_ID} within ${POLL_WAIT_MS}ms`
);

const content = pushed.params?.content || "";
if (!content.includes(TASK_ID)) fail("pushed event content missing task id");
if (!content.includes("end to end")) fail("pushed event lost the hook title (JSON escaping?)");

console.log(`✓ [${MODE}] channel pushed completed event for ${TASK_ID}`);
if (MODE === "steady") console.log(`✓ did not replay pre-existing notifications on startup`);
if (MODE === "cold") console.log(`✓ first event after empty-file start was delivered (cold-start fix)`);
console.log(`✓ title with embedded quotes survived hook→channel`);
console.log("\nPASSED");
proc.kill();
process.exit(0);
})();

setTimeout(() => fail("overall test timeout"), POLL_WAIT_MS + 12000);
Loading
Loading