open_workspace recursively scans the full workspace and can exceed MCP client timeouts

[Ahmed-Hindy]

Summary

open_workspace can block long enough for an MCP client to time out when the requested workspace is a directory containing many independent projects.

The delay comes from eager discovery of nested AGENTS.md and CLAUDE.md files. This is local filesystem work in DevSpace, not an authentication or tunnel failure.

Current behavior

WorkspaceRegistry.createWorkspaceContext() awaits findAvailableAgentsFiles(root) before open_workspace responds.

findAvailableAgentsFiles() calls walkWorkspace(), which recursively and serially traverses every descendant directory except a small fixed skip list. It visits every file entry even though it only needs to identify two instruction-file names. There is no depth, directory-count, or time bound.

Reproduction

1. Configure an allowed root that contains many project directories and their build/source trees.
2. Call open_workspace on that parent directory instead of on one project.
3. Observe that the MCP request can take longer than the client transport timeout.

On one Windows reproduction with roughly 70 direct project folders, the server completed open_workspace after about 153 seconds. The MCP client reported a network failure even though the server later logged a successful HTTP 200 response. A single repository opened in about 55 ms; the parent directory still took several seconds after filesystem warming.

Expected behavior

Opening a workspace should be bounded by the workspace's immediate setup work. Nested instruction discovery should not make the initial MCP request depend on the size of the complete descendant tree.

Suggested fix: lazy context loading

1. During open_workspace, load only root-level and global instruction files.
2. Remove the full descendant walk from the synchronous open path.
3. Before a path-aware tool operates on a file or directory, inspect only that target's ancestor directories up to the workspace root for AGENTS.md / CLAUDE.md.
4. Return any newly encountered instruction files with that tool response and cache them for the workspace.
5. Apply the same ancestor-context check to shell calls using their explicit working directory.

This preserves nested instruction semantics without silently dropping instructions. A scan timeout alone would avoid the latency but could cause the model to miss applicable instructions.

Acceptance tests

• Opening a workspace with a large generated/project collection completes within a small fixed budget.
• Root-level instructions are still returned by open_workspace.
• A nested instruction file is returned before a tool operates within its directory.
• Previously loaded instruction files are not returned repeatedly.
• Context lookup cannot escape the workspace root through parent traversal or links.