Restructure modules into workflow groupings (step 1 of 2)

[micheleRP]

Predecessor: PR #38 reorganized nav.adoc only (job-to-be-done labels overlaid on existing files). This PR is the larger physical restructure: it moves files into new modules so the directory layout matches the new IA. Step 2 (out of scope here) will rename pages to be verb-oriented and surface the conversational labels at the URL level.

Context

The team likes the prototype's IA framing (Connect data & tools, Monitor & debug, Control & govern, Routing & LLM settings via AI Gateway) and wants to adopt those workflow groupings, but compromised for the first release to stay consistent with the docs and cloud-docs repos:

• Keep get-started/ as a module — both docs and cloud-docs use this name; preserves several file paths and matches sibling-repo convention.
• No separate Concepts module — docs and cloud-docs don't isolate concepts that way. Concept pages live alongside the workflow they describe.
• Tutorials live with their workflow — agents/tutorials/* becomes connect/tutorials/.

This is step 1 of a two-step rollout. Step 1 moves files into new module directories and keeps the original filenames intact wherever possible. Step 2 (separate PR, later) will rename pages to align with verb-oriented labels.

Preview page

Overview

New module layout

modules/
  get-started/    overview, BYOC quickstart, AI Gateway quickstart, AI Agent quickstart
  connect/        MCP servers, agent build, BYOA register, IDE integrations, tutorials, agent concepts
  monitor/        transcripts, BYOA telemetry, custom traces, logs, metrics, observability concepts, troubleshoot
  control/        governance dashboard, guardrails (overview/rules/violations/types), budgets, permissions
  gateway/        AI Gateway overview, architecture, configure-provider, Bedrock setup, aggregation, connect-agent
  reference/      rpk-install, rpk command tree, glossary
  ROOT/           nav.adoc + cross-module partials
  shared/         shared images

8 modules total (was 10). The retired ones are agents, ai-gateway, governance, observability, integrations, mcp — their content redistributes across the new workflow modules. rpk-install.adoc moved into reference/ (was get-started/) so it doesn't compete for attention with the quickstart pages.

New nav.adoc

* Quickstarts
** xref:get-started:adp-overview.adoc[Overview]
** xref:get-started:byoc-quickstart.adoc[ADP Quickstart]
** xref:get-started:gateway-quickstart.adoc[AI Gateway Quickstart]
** xref:get-started:quickstart.adoc[AI Agent Quickstart]

* Connect data & tools
** xref:connect:managed/managed-catalog.adoc[Plug in any app, database, or tool]
** xref:connect:create-agent.adoc[Turn your data source into an agent]
** xref:connect:create-server.adoc[Build a tool server for your own data]
** xref:connect:register-remote.adoc[Connect a tool server you host yourself]
** xref:connect:user-delegated-oauth.adoc[Let agents act as the signed-in user]
** xref:connect:byoa-register.adoc[Register your own agent (BYOA)]
** xref:connect:claude-code.adoc[Claude Code]
** xref:connect:remote-mcp-clients.adoc[Remote MCP clients]

* Monitor & debug
** xref:monitor:transcripts.adoc[See what your agent did]
** xref:monitor:troubleshoot-ai-agents.adoc[Investigate a broken run]
** xref:monitor:metrics.adoc[Check speed, cost, and errors]
** xref:control:guardrails/violations.adoc[Review blocked requests]
** xref:monitor:byoa-telemetry.adoc[Send telemetry from agents you host]

* Control & govern
** xref:control:dashboard/overview.adoc[See all your agents in one place]
** xref:control:guardrails/overview.adoc[Fix agents calling things they shouldn't]
** xref:control:guardrails/create-guardrail.adoc[Set safety rules for all agents]
** xref:control:budgets.adoc[Set spending limits]
** xref:control:permissions-overview.adoc[Control who can do what]

* Routing & LLM settings
** xref:gateway:overview.adoc[How the gateway works]
** xref:gateway:configure-provider.adoc[Configure LLM provider]
*** xref:gateway:bedrock-setup.adoc[Set up AWS Bedrock]

* Settings reference
** xref:control:permissions-reference.adoc[Roles and permissions matrix]
** xref:control:guardrails/types-reference.adoc[Safety rule providers]
** xref:reference:rpk-install.adoc[Install rpk]
** xref:reference:rpk/index.adoc[rpk command reference]
** xref:reference:glossary.adoc[Glossary]

The Concepts section from PR #38's nav is removed; concept pages are still in the build (under their workflow modules) but don't get their own nav section.

What changed (per commit, in order)

1. Drop stub and index-only pages — 24 files removed; each was 3–6 lines (mostly section indexes and TODO-only stubs). Notable: governance/guardrails/cost-tracking.adoc is dropped because it's eng-blocked per the ADP GA Readiness Plan Track A.
2. Move quickstarts into get-started/ — agents/quickstart.adoc and ai-gateway/gateway-quickstart.adoc join the existing adp-overview.adoc and byoc-quickstart.adoc.
3. Create connect/ module — folds agents/, mcp/, integrations/ content (create-agent, BYOA, MCP server creation/registration/OAuth, managed catalog, IDE integrations, tutorials, agent concepts). Two forced renames due to filename collisions: agents/overview.adoc → agents-overview.adoc; mcp/overview.adoc → mcp-overview.adoc.
4. Create monitor/ module — replaces observability/, plus pulls in agents/troubleshoot/troubleshoot-ai-agents.adoc and agents/monitor.adoc.
5. Create control/ module — renames governance/ to control/. Preserves dashboard/ and guardrails/ subdirs.
6. Create gateway/ module — renames ai-gateway/ to gateway/. Preserves admin/ and builders/ subdirs.
7. Move partials into consuming modules — ROOT/partials/integrations/* → connect/partials/; observability partials → monitor/partials/. Deduplicates 14 redundant partials.
8. Rewrite xrefs and includes — 519 replacements across 72 files. Exception rewrites first (forced renames and the quickstart moves), then bulk module-prefix swaps (xref:agents: → xref:connect:, etc.).
9. Add :page-aliases: entries — 59 moved pages each carry an alias to the old <module>:<path>.adoc so existing URLs (bookmarks, blog links, support tickets) keep resolving. Existing aliases (e.g. to cloud-data-platform:ai-agents:*) are preserved.
10. Rewrite nav.adoc — drops the old 132-line noun-based shape; adopts the workflow grouping with conversational xref:[Label] overrides.
11. Remove empty old module dirs and orphan duplicates — modules agents/, mcp/, ai-gateway/, observability/, governance/, integrations/ deleted now that everything's moved.
12. Fix broken xrefs surfaced by the first build — six bare xref:troubleshoot/troubleshoot-ai-agents.adoc refs (intra-module before the move) now use the explicit xref:monitor: prefix; refs to dropped section-index pages updated or removed.
13. Move rpk-install into reference/ — frames it as reference material rather than a quickstart prerequisite. rpk ai is still beta this release, so we don't want to surface rpk installation as a headline path.

Move summary

124 files changed: 538 insertions, 9671 deletions (most of the deletions are git counting renames as add+delete in line counts, but they're true R renames in git history — git blame is preserved).

• 59 page renames (cross-repo aliases preserved; no new in-repo aliases added — see Backwards compatibility)
• 24 stub pages dropped
• 14 redundant partials deduplicated
• 519 xref/include rewrites
• 6 pre-existing aliases (5 cross-repo, 1 in-repo) preserved unchanged

Backwards compatibility

The new workflow-module URLs (/connect/, /monitor/, /control/, /gateway/) and the prior noun-module URLs (/agents/, /mcp/, /ai-gateway/, etc.) have never shipped publicly — this is a pre-launch restructure. So we don't add new :page-aliases: entries for in-repo path changes; nothing externally links those paths.

The five pre-existing aliases pointing at cloud-docs paths (cloud-data-platform:ai-agents:* and develop:*) are preserved unchanged, because cloud-docs's published content still links to those paths. One pre-existing in-repo alias (reference:rpk/rpk-profile.adoc on the rpk-profile single-source stub) is also kept because three rpk-cloud pages link via the shorter path.

Open questions

Captured here so the team can override before this lands:

1. connect/agents-overview.adoc + connect/mcp-overview.adoc — two overview.adoc files collided in connect/. Step 2 idea: merge into a single connect/overview.adoc (or drop both if redundant once we look at them together).
2. Tutorials nesting — connect/pages/tutorials/ keeps the tutorials together as a sub-tree. If later we want a top-level Tutorials section, the move is easy.
3. Concept pages — currently scattered (connect/concepts.adoc, monitor/concepts.adoc, connect/a2a-concepts.adoc, etc.). Step 2 can decide whether to surface them in a nav Concepts section or merge into workflow intros.

What's out of scope (step 2 work)

• Filename rewrites for verb orientation (e.g. create-server.adoc → build-tool-server.adoc). The conversational nav labels currently live as xref:[Label] overrides; step 2 makes the URL match the label.
• Filling gaps flagged in PR Reorganize ADP nav around job-to-be-done workflows #38 (Agent configuration fields, Tool server configuration fields, Gateway configuration, Activity data format, Pick which LLM handles each request, Investigate a slow run).
• agents-overview.adoc + mcp-overview.adoc merge into a single connect/overview.adoc.

Test plan

• npm run build exits clean (only pre-existing rpk-ai table warnings, unchanged)
• Every nav entry resolves to a real page (17 spot-checked: all 200)
• Pre-existing cross-repo aliases (cloud-data-platform:ai-agents:*) preserved on the original carrier pages
• All file moves recorded as R in git log (blame history preserved)
• Cross-browser nav spot-check (Chrome / Firefox / Safari)
• Verify cross-repo refs from cloud-docs still resolve once that repo's #561 lands

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for redpanda-agentic-data-plane ready!

Name	Link
🔨 Latest commit	f810c49
🔍 Latest deploy log	https://app.netlify.com/projects/redpanda-agentic-data-plane/deploys/6a18bf31d9818900089b9c2d
😎 Deploy Preview	https://deploy-preview-39--redpanda-agentic-data-plane.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Feediver1]

Be sure to update your navtree entries--this is just a first pass/attempt:

Connect Data & Tools (use our navtree top level capitalization/init caps)
Plug in any app, database, or tool --> Plug In Apps, Databases, and Tools
Turn your data source into an agent --> Transform Data Sources into Agents
Build a tool server for your own data --> Build a Tool Server for Your Data
Connect a tool server you host yourself --> Connect a Self-Hosted Tool Server
Let agents act as the signed-in user --> Authorize Agents to Act as Users
Register your own agent (BYOA) --> Register Your Own Agents

Monitor & Debug
See what your agent did --> View Agent Activity
Investigate a broken run --> Troubleshoot Failed Runs
Check speed, cost, and errors --> Monitor Performance, Cost, and Errors
Review Blocked Requests --> seems okay
Send telemetry from agents you host --> Share Telemetry from Hosted Agents

Control & Govern
See all your agents in one place --> View Your Agents
Fix agents calling things they shouldn't --> Restrict Agent Tool Access
Set safety rules for all agents --> Set Global Safety Rules
Set spending limits --> Set Spending Limits
Control who can do what --> Manage Access

Routing & LLM Settings
How the gateway works --> Gateway Overview
Configure LLM provider --> Configure LLM Providers

Settings Reference
Roles and permissions matrix --> Roles and Permissions
Safety rule providers --> Safety Rule Providers
Install rpk --> seems okay
rpk command reference -- rpk Command Reference
Glossary --> fine as is