RPCN - Pipelines listing page speed + UX improvements

[SpicyPete]

Rebuild the Redpanda Connect pipelines list for large clusters

The pipeline list was built for a handful of pipelines. On clusters with hundreds it was slow to
render, offered no way to narrow down what you were looking for, and buried broken pipelines
wherever the server happened to return them. This reworks the page around finding one pipeline in
a long list, and fixes the cost of getting that list in the first place.

How it ships

The page's only mount used to be isFeatureFlagEnabled('enableRpcnTiles') && isEmbedded(), and
#2591 removed that flag as dead code — the flag has been off in LaunchDarkly, so this list has
never actually rendered for a user. It now mounts on a capability instead:

• Features.pipelinesApi — i.e. the backend serves redpanda.api.console.v1alpha1.PipelineService.
  Cloud gets the new list whether or not it's embedded, with no flag to flip.
• Self-hosted reports that service unsupported (OSS defaults in endpoint_compatibility.go) and
  keeps the legacy tabs, including the Redpanda Connect install intro.
• Checked ahead of the Kafka Connect spinner deliberately: the list handles a pending Kafka Connect
  probe itself, so waiting there would delay its own multi-page fetch behind an unrelated request —
  and a failed /console/endpoints leaves feature detection pending forever, so falling through to
  the legacy path beats parking the page on a spinner that never resolves.

Kafka Connect stays reachable: the new page renders the same TabKafkaConnect component behind its
own Kafka Connect tab when a cluster is configured, and the /connect-clusters/$clusterName routes
are untouched.

List page

• Status tabs — All / Running / Stopped / Error, each with a live count. Transitional states ride
  with their destination (starting counts as running, stopping as stopped). Counts come from the
  status column's faceted row model, so each tab shows what selecting it would yield under the
  current search and filters.
• Search by name or ID, debounced, matching case-insensitively against both.
• Faceted filters for input, output, and tag, with per-option counts. Clear filters appears
  only when something is actually filtered — the status tabs are views, not filters, so they aren't
  swept up by it — and it tracks the input directly rather than lagging 200ms behind the debounce.
• Sortable Pipeline and Status columns. Status sorts attention-first by default (errors and
  transitions above healthy pipelines, idle at the bottom), so a broken pipeline lands on page 1 of a
  large cluster instead of wherever the server put it.
• Rows are clickable, with the same guard the registry DataTable uses: clicks on portaled
  content (open menus, the delete-confirm backdrop) and on interactive descendants don't navigate,
  and neither does a click that ends a text selection — the pipeline ID is select-all, so one click
  grabs the whole thing for copying. ⌘/Ctrl/Shift-click and middle-click are left to the browser, so
  "open in a new tab" still means that.
• Connector badges collapse duplicates into redpanda ×2 rather than repeating the same logo
  across the column.
• Progressive rendering. The list used to wait for every page before showing anything; it now
  renders as soon as the first page has rows and streams the rest in behind the table, with a
  distinct line for "still loading pages" vs "background refresh failed" — partial data and stale
  data read differently now.
• Search, facets and page survive a trip to the Kafka Connect tab and back (keepMounted — Base UI
  panels unmount by default).
• Dropped the Processors column and the old DataTableFilter stack from this page.

Accessibility

• The status tabs own a panel. They sit outside the table (one table filtered per tab, not four
  panels), so without an aria-controls target a screen reader announced "tab, 1 of 4" with nowhere
  to move into. Each tab now points at the table region, which is labelled by the active tab.
• Async status is announced. "Loading more pipelines" and "Couldn't refresh pipelines" appear
  without user action, so each has an always-mounted sr-only live region (polite for the drain,
  role="alert" for the failure) — the visible lines animate in and out, and a live region only
  announces changes made while it's already in the DOM.
• Row click stays a pointer shortcut with no row tab stop: the name cell is already a real link, so a
  tab stop per row would just duplicate it.

Fetch and render cost

• Page size 500 instead of MAX_PAGE_SIZE (which is 25) — 20× fewer sequential round trips to
  drain. The server does the same work per call at any page size: it lists everything and slices.
• Deduplicate the drain by pipeline ID. The dataplane's keyset page token names the first ID of
  the next page; when that pipeline is deleted mid-drain, a server resolving the token by exact match
  restarts at page one and replays rows we already have.
• Stop draining on any token the drain already requested. Keyset tokens only move forward, so a
  repeat means the server sent us backwards. The first cut only caught an immediate A → A repeat,
  which a A → B → A cycle walks straight past — the drain then alternates forever, adding a page to
  the query cache every round. Now checked against every prior page param, with a test for each shape.
• Memoize the YAML parse per config text. The transform re-ran a full parse for every row on every
  drain step and poll tick; it's now O(new rows), with a bounded cache that evicts its oldest half.
• Row identity keyed on pipeline ID (not row index) and autoResetPageIndex: false, so streaming
  pages don't yank you back to page 1 or repaint a shifted window of rows. Filter and sort changes
  still reset to page 1, and a shrinking row set is clamped before paint.
• Cache the facet icon component per connector name, so logos in an open filter popover don't remount
  and flash on every poll; count the status tabs in a single pass; and memoize the per-row connector
  aggregation so cells don't re-derive it on every keystroke.

[github-actions]

🚨 Registry drift detected

App: frontend · Scope: diff vs origin/master · Files: 31

	Count
⚠️ Outdated registry components	0
🛠 Locally-modified components	1
❓ Unknown to registry	0
🎨 Off-token palette colours	68
🔢 Ad-hoc utility classes	1

Components needing attention

Status	Component	Uses	Detail
🛠 locally-modified	data-table	1×	no tagged release matches installed bytes

🎨 Off-token colours (palette literals)

Use semantic tokens (primary, muted-foreground, border, …) instead of raw palette names.

Class	Uses	Files
indigo-400	14	1
indigo-500	9	1
indigo-300	7	1
indigo-alpha-200	7	1
blue-400	6	1
indigo-600	6	1
indigo-800	6	1
blue-900	5	1
green-500	5	1
indigo-100	5	1
orange-400	5	1
red-600	5	1
blue-500	4	1
blue-alpha-200	4	1
green-400	4	1
indigo-200	4	1
indigo-900	4	1
indigo-alpha-100	4	1
indigo-alpha-300	4	1
orange-500	4	1
red-400	4	1
red-500	4	1
red-alpha-200	4	1
blue-600	3	1
blue-800	3	1
green-300	3	1
green-600	3	1
orange-200	3	1
blue-100	2	1
blue-200	2	1
blue-300	2	1
blue-alpha-100	2	1
blue-alpha-300	2	1
blue-alpha-500	2	1
green-100	2	1
green-50	2	1
green-700	2	1
green-800	2	1
green-900	2	1
indigo-50	2	1
indigo-700	2	1
indigo-alpha-500	2	1
orange-100	2	1
orange-300	2	1
orange-800	2	1
orange-900	2	1
red-200	2	1
red-300	2	1
red-50	2	1
red-alpha-100	2	1
red-alpha-300	2	1
red-alpha-500	2	1
blue-alpha-400	1	1
green-200	1	1
green-alpha-100	1	1
green-alpha-200	1	1
indigo-alpha-400	1	1
orange-50	1	1
orange-600	1	1
orange-700	1	1
orange-950	1	1
purple-500	1	1
purple-600	1	1
red-100	1	1
red-700	1	1
red-800	1	1
red-950	1	1
red-alpha-400	1	1

🔢 Ad-hoc utility classes (arbitrary values)

These bypass the design tokens. Prefer a named scale entry or add a new token.

Class	Category	Uses	Files
rounded-[4px]	🔢 radius	1	1

Generated by lookout audit-changes.

[SpicyPete]

@claude review

[SpicyPete]

@claude review

[github-actions]

Claude finished @SpicyPete's task in 7m 57s —— View job

---

Review: RPCN pipelines listing (#2593)

• Read repo guidelines (frontend/AGENTS.md)
• Review list page + list-utils
• Review useListPipelinesQuery drain logic + tests
• Review registry data-table / badge-group / tabs changes
• Review out-of-scope files (header, data-refresh, theme.css, throughput card)
• Post review

⚠️ I could not run bun run type:check && bun run lint && bun run test — frontend/node_modules is absent in this environment and dependency installs are blocked. Everything below is from reading the diff.

Overall this is strong work: the drain-termination tests (pipeline.test.tsx:130-236) cover exactly the shapes that used to loop, the faceted tab counts are the right mechanism, and the memoized YAML parse plus getRowId + autoResetPageIndex: false address the real render cost. Findings, most important first.

---

1. Registry divergence, and app-wide behavior changes riding along

The audit bot already flags data-table as locally modified (no tagged release matches installed bytes). Beyond drift, three changes in src/components/redpanda-ui/ alter every existing consumer, not just this page:

• data-table.tsx:390-420 — rows now get tabIndex={0} and Enter/Space activation whenever onRow/expandRowByClick is set. That adds one tab stop per row to every table in the app. Note this is the opposite call from the one this PR makes for its own list (list.tsx:855-857: "no tab stop or Enter handler, since the name cell already holds the same link"). Both rationales can't be right for a 500-row table — worth reconciling.
• data-table.tsx:429-435 — the pagination footer moved out of <TableFooter> and now renders whenever pagination is enabled, where before it was gated on displayState === 'data'. Every existing table will show "Page 0 of 0" + "Rows per page" while loading and when empty.
• badge-group.tsx:54 — maxVisible now defaults to 3. Every in-repo call site passes it explicitly so nothing regresses here, but it is a breaking default for the shared component.

Recommendation: land these in ui-registry and re-sync, otherwise the next registry sync silently reverts the row-activation guard, the clamp logic and resolvePageDisplayState. A visual sweep of existing DataTable consumers would also be worth doing before merge.

2. header.tsx:150 — the Debug bundle label may render empty

<RegistryButton render={<Link to="/debug-bundle">Debug bundle</Link>} variant="ghost" />

Button destructures children and passes {renderButtonChildren(children, icon, isLoading)} — i.e. an explicit children: undefined — down to ButtonPrimitive (button.tsx:250-285). Whether the render element's own children survive that depends on Base UI's merge semantics; BreadcrumbLink (the pattern this looks like) works precisely because it never destructures children, so the key is absent rather than present-and-undefined. The registry Button has a first-class path for this that the rest of the codebase uses (license-utils.tsx:393, cluster-health-overview.tsx:92):

<RegistryButton as={Link} to="/debug-bundle" variant="ghost">Debug bundle</RegistryButton>

Please confirm the label actually renders in the non-embedded self-hosted header. Fix this →

3. An aborted drain truncates the list silently

pipeline.tsx:114-126 correctly stops on any previously-requested token, but when it trips, getNextPageParam returns undefined → hasNextPage is false → isLoading clears and no error is set. The page then renders a partial list that is indistinguishable from a complete one: neither status line in list.tsx:873-885 fires. On a cluster where a mid-drain delete triggers a cycle, the user quietly loses pipelines from the list. At minimum log/report it; ideally surface the same "Failed to load all pipelines" line by threading a flag out of the hook.

4. Smart polling refetches every drained page every 2s

list.tsx:595-597 enables enableSmartPolling, and pipeline.tsx:105-113 returns SHORT_POLLING_INTERVAL while any pipeline is transitional. A TanStack infinite query refetch re-fetches all accumulated pages, so a 10k-pipeline cluster (20 pages at 500) issues ~20 sequential requests every 2 seconds for as long as one pipeline is STARTING/STOPPING — and starting a pipeline is exactly when someone is looking at this page. The page-size bump makes this 20× better than before, but it's still the dominant cost on a large cluster. Options: poll the transitional rows individually via getPipeline, scale the interval by page count, or cap with maxPages.

5. configComponentsCache retains YAML for the process lifetime

list.tsx:110-132: module-level Map, keys are full config YAML strings, cap 10 000, never cleared on unmount. At a few KB per config that's tens of MB of retained strings surviving navigation away from the page — and the query cache already holds the live copies, so the cache is only useful for the current view. Suggest sizing it to a small multiple of the page's working set (a few hundred) or clearing it when the list unmounts.

---

Smaller items

• isDeletingPipeline is global. One useDeletePipelineMutation instance is shared by every row (list.tsx:598), so while any delete is in flight every row's menu item and confirm dialog show the deleting state (list.tsx:415, list.tsx:419). Compare against the id being deleted.
• Row-click selection guard is page-wide. window.getSelection()?.toString() (list.tsx:742) blocks navigation on any selection anywhere in the document. Scope it: selection.containsNode(event.currentTarget, true).
• list.test.tsx:257-276 doesn't test what it says. It asserts a plain click on the ID span navigates — but that span is select-all (list.tsx:493), so in a real browser the click creates a selection and hits the guard above. The test passes only because jsdom doesn't implement select-all. Click a non-select-all cell instead, otherwise this test will keep passing even if row navigation breaks for everything except the ID.
• Untested headline behaviors. The transport at list.test.tsx:91-111 serves two pages but nothing asserts the intermediate partial-render state, the "Loading more pipelines" / "Couldn't refresh pipelines" lines, their live regions, or that the page index survives a page arriving mid-drain — those are the changes the PR leads with.
• Pagination bar always visible (list.tsx:887) even at "Page 1 of 1" with 3 rows. Consider hiding when getPageCount() <= 1.
• "Asc / Desc" for the Status column (data-table-column-header.tsx:51-58) reads oddly for a priority order where "Asc" means errors-first. A per-column label override would help.
• Tab counts don't sum to All when a pipeline is UNSPECIFIED (list-utils.ts:71-80) — intentional, but a user seeing 1+1+1 ≠ 4 will read it as a bug.
• Whitespace-only search (list.tsx:722): search.trim() !== ''is false, soClear filters` is hidden while the input still visibly contains text and the user has no button to empty it.
• rounded-[4px] (data-table-faceted-filter.tsx:99) is the ad-hoc utility the audit flags — use a radius token.

Scope and description

• Fonts (theme.css:1-18, plus @fontsource/inter, @fontsource/geist-mono, inter-ui in package.json), the header.tsx debug-bundle tooltip, the data-refresh Chakra→registry port, the throughput-card empty/error states and the audit-workflow tuning are all unrelated to the pipelines list. The font imports in particular add webfont payload to every page in the app and belong in a change where that can be measured on its own. Both bun.lock and yarn.lock are touched — worth confirming that's intended.
• The "How it ships" section doesn't match the code. overview.tsx:145 mounts the new list on isEmbedded(), not Features.pipelinesApi, and that line is already in origin/master (via Fix - Show the new pipeline list in Cloud without feature flag #2594) rather than part of this diff. The pipelinesApi capability is only used for RpConnectTabContent (overview.tsx:100-101) and the docs link. Either the description or the mount needs updating — as written, "Cloud gets the new list whether or not it's embedded" is not what ships.

Nice work

getNextPageParam checking all prior page params with a test per cycle shape, the id-keyed dedup with later-pages-win, the sr-only always-mounted live regions (the reason the animated lines can't announce is stated in the code), getUniqueValues deduping per row so facet counts don't double-count, and resolvePageDisplayState holding loading through a pending clamp — these are all the non-obvious cases, handled and commented.
· branch rpcn/many-many-pipelines-listings