Skip to content

Studio Code: run a WordPress language server (wp-lsp) alongside the agent - #4618

Draft
draganescu wants to merge 1 commit into
trunkfrom
add-wp-lsp-agent-support
Draft

Studio Code: run a WordPress language server (wp-lsp) alongside the agent#4618
draganescu wants to merge 1 commit into
trunkfrom
add-wp-lsp-agent-support

Conversation

@draganescu

Copy link
Copy Markdown
Contributor

Related issues

  • None yet — this is an exploratory Proof of Concept. If we decide to pursue it, a companion issue laying out the plan should come first (per CONTRIBUTING.md); happy to open one.

How AI was used in this PR

Built end-to-end in a Claude Code session (Fable 5): the model read the wp-lsp README and the agent runtime, proposed the integration architecture, implemented it, and wrote the tests. The integration test it wrote against the real server caught one real bug during development (a TDZ crash in the server pool that would have silently disabled the whole feature). I steered and reviewed the direction; line-by-line human review is still pending — one more reason this is a draft.

Proposed Changes

WordPress is held together by strings. add_action( 'init', 'my_func' ) names a function nothing calls directly; register_post_type( 'book' ) creates a slug that turns up in a query three files away; a block's identity is spread across block.json, PHP, and JS. None of that is visible to grep — and grep is what Studio Code does today when it needs to understand a site's code.

This PR runs wp-lsp, a WordPress language server, next to the agent — one server per site, on Studio's bundled PHP, shipped with the CLI the same way phpMyAdmin is. The agent gets two things out of it:

  1. An Lsp tool for exact answers instead of text matches: every callback attached to a hook (in priority order, including [ $this, 'method' ] ones), where a post type slug is registered, which files make up a block, where a script handle is enqueued.
  2. Automatic diagnostics after every PHP edit. When the agent writes add_action( 'ini', … ), the warning — Unknown hook 'ini'. Did you mean 'init'? — lands directly in the Edit result, and the agent corrects itself in the same turn, before the user (or a screenshot loop) ever sees the bug. Same for wrong callback argument counts, deprecated hooks, and text-domain mismatches.

Where the hopes are:

  • Fewer grep→read→grep loops on "trace this identifier" work — fewer tokens and turns, and correct answers where grep is structurally blind.
  • WordPress-specific bugs caught at edit time instead of shipping inside generated sites.
  • All of this is meant to be counted, not assumed: docs/design-docs/wp-lsp-agent.md describes concrete ways to measure the impact — A/B evals through the existing promptfoo runner (the STUDIO_WP_LSP_PATH override doubles as a clean off-switch), grading finished sites with wp-lsp itself as an independent defect counter, and proposed Tracks counters including a "self-correction rate" (diagnostics reported vs. resolved within the session).

Trade-offs and behavior notes:

  • A PHP edit now waits up to ~3s for diagnostics (~10s on the first edit of a site, while the initial index runs). Counted against the wins in the measurement doc, not ignored.
  • Zero new npm dependencies — the LSP client is ~200 lines of hand-rolled JSON-RPC framing.
  • Local sites only; when the wp-lsp archive or a PHP binary is missing, everything degrades to exactly today's behavior (no tool, no prompt section, no waits).

Testing Instructions

Setup

  1. npm install (postinstall downloads wp-lsp v0.2.0 into wp-files/).
  2. npm run cli:build
  3. From the repo root: node apps/cli/dist/cli/main.mjs

Watch the agent catch its own bug

  1. Pick or create a local site in the chat.
  2. Say: Add add_action( 'ini', 'demo_setup' ); and an empty demo_setup() function to my theme's functions.php, spelled exactly like that.
  3. Watch the Edit result: it comes back with Unknown hook 'ini'. Did you mean 'init'? and the agent fixes it in the same turn.

Use the Lsp tool

  1. Say: Use the Lsp tool to list everything that runs on wp_head in this site.
  2. Expect resolved callbacks with file:line locations, not grep output.
  3. Then try without naming the tool — where is the post post type registered? — to see it reach for Lsp on its own.

Confirm the fallback is clean

  1. mkdir -p /tmp/empty && STUDIO_WP_LSP_PATH=/tmp/empty node apps/cli/dist/cli/main.mjs
  2. Confirm the Lsp tool is not offered and PHP edits gain no diagnostics block — behavior is identical to trunk.

Automated tests

  1. npm test -- apps/cli/ai/lsp/tests/ — 41 tests; the integration test drives the real wp-lsp binary over stdio (needs php on PATH, auto-skips without it).
  2. npm run typecheck

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?

🤖 Generated with Claude Code

…pool, post-edit PHP diagnostics

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@draganescu

Copy link
Copy Markdown
Contributor Author

This is an idea which I am not sure if worth exploring - grep is fine and fast.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant