diff --git a/.agents/skills/hyperview-cli/SKILL.md b/.agents/skills/hyperview-cli/SKILL.md index 64971d0..a8331af 100644 --- a/.agents/skills/hyperview-cli/SKILL.md +++ b/.agents/skills/hyperview-cli/SKILL.md @@ -1,6 +1,6 @@ --- name: hyperview-cli -description: Use HyperView's control-plane CLI for hyperview serve, dataset create, workspace create, embeddings compute, layouts compute, browserless paper figure export, runtime jobs, ui layout set, ui selection set, ui panel add/update, extension add, tools run, panel modules, Python tools, and local HyperView extension workflows. +description: Use HyperView's control-plane CLI for hyperview serve, static workspace export, dataset create, workspace create, embeddings compute, layouts compute, browserless paper figure export, runtime jobs, ui layout set, ui selection set, ui panel add/update, extension add, tools run, panel modules, Python tools, and local HyperView extension workflows. license: MIT compatibility: Requires Python 3.10-3.13 and the hyperview CLI (`uv tool install --python 3.12 hyperview`). Runtime-control commands require a running HyperView server. metadata: @@ -30,6 +30,7 @@ HyperView currently supports Python 3.10 through 3.13; `--python 3.12` keeps the - Register a custom embedding provider. - Compute embeddings or layouts without restarting the UI. - Export paper-ready static 3D embedding figures without opening the UI. +- Export read-only static demo bundles with `hyperview export`. - Switch the active workspace, layout, or selection in a running session. - Add, remove, or compose extension-backed panel instances. - Create, install, reload, or use a local extension with Python tools and browser panels. @@ -43,7 +44,8 @@ HyperView currently supports Python 3.10 through 3.13; `--python 3.12` keeps the 5. Submit embedding or layout jobs through the runtime. 6. Use `hyperview ui ...` commands to switch what the live UI shows. 7. Export paper figures with `hyperview figure export` when the user needs screenshots or publication diagrams. -8. For extensions, create an extension folder and install it into the running workspace. +8. Export read-only static demos with `hyperview export --out bundle/`. +9. For extensions, create an extension folder and install it into the running workspace. ## Current model @@ -51,15 +53,18 @@ HyperView currently supports Python 3.10 through 3.13; `--python 3.12` keeps the - Datasets are created separately from workspaces. - The workspace owns the dataset selection. - `ui layout set` changes the active layout and opens the matching built-in scatter panel. -- `ui similarity set` selects an anchor sample and pins the nearest-neighbor context to an explicit layout or space. +- Samples retrieval state lives under the runtime-managed Samples panel state. `ui samples retrieval set-anchor` selects an anchor sample and pins nearest-neighbor context to an explicit layout or space. - Runtime-added panels can be built-in samples panels, typed scatter instances bound to explicit layout keys, or extension-backed panel modules loaded into the host React tree. -- Runtime-added panels use the stable `HyperViewPanelSDK` surface on `window`. +- Runtime control commands use canonical namespaced ids: `workspace.*`, `panel..*`, and `collection.*`. Older command ids remain deprecated aliases for compatibility only. +- Command results carry the runtime snapshot in a `CommandResult` envelope (`ok`, `command`, `result`, `workspace`, `snapshot`, `revision`, `error`). Frontends and agents should apply the returned snapshot instead of doing an immediate `/api/runtime` refetch. +- Runtime-added panels use the stable thin `HyperViewPanelSDK` surface on `window`. - Extensions are repo-local folders with `extension.toml`, optional Python tools, and optional panel modules. -- Extension panels call Python tools through `HyperViewPanelSDK.hooks.useTool()` or `hyperview tools run`. +- Extension panels run control commands through the SDK command client; Python tools are still reachable through `hyperview tools run`. - Extensions define reusable tools/panels; workspace views compose concrete panel instances and layout. In Python launch scripts, register extensions with `session.ui.add_extension(...)` and place panels with `hv.ui.ExtensionPanel(...)`. - In practice, create datasets and workspaces before starting the runtime for that workspace. - `figure export` is browserless and supports 3D layouts only. It reuses the persisted 3D camera for the layout when available, otherwise it chooses a paper-oriented default view. - Paper figure defaults are square, white-background, opaque PNGs with a faint sphere guide and direct labels for small label sets. +- `hyperview export --out bundle/` writes a self-contained static frontend + JSON/API/media bundle. It is intentionally read-only: visitors can browse, select, pan/zoom, inspect panels, and use exported precomputed data, but mutations are disabled with a "Read-only demo — pip install hyperview for the full workbench" notice. Read [references/commands.md](references/commands.md) for command recipes covering datasets, workspaces, providers, embeddings, layouts, paper figures, runtime UI state, selections, and jobs. Read [references/panel-modules.md](references/panel-modules.md) when the task involves authoring a browser panel module. @@ -72,16 +77,24 @@ Read [references/extensions.md](references/extensions.md) when the task involves - Prefer `workspace create --dataset ...` over separate create and dataset-attach calls when setting up a new workspace. - In Python dataset setup code, use public ingestion helpers such as `dataset.add_samples([...])` or `dataset.add_images_dir(...)`. - Prefer built-in providers before registering custom providers. For Hyper3-CLIP, use `dataset.compute_embeddings(model="hyper3-clip-v0.5", provider="hyper-models")`. +- In comparison demos, fail fast when a required model/provider cannot compute embeddings; do not silently substitute the baseline model as a "candidate" fallback. - When a project truly needs a custom Python provider, use `hyperview provider register ...` from the CLI or `hv.register_provider(...)` in Python. - For custom panel code, create an extension under `.hyperview/extensions//`; do not register arbitrary panel module files directly. - For side-by-side embedding comparisons, add typed scatter panels through `hyperview ui panel add --kind scatter --layout-key ... --reference-panel-id ... --direction right`. - Use first-class view layout fields for panel sizing and visibility: `hv.ui.PanelLayout(width=..., min_width=...)` in Python, or `hyperview ui panel resize/move/focus/close/show` from the CLI. Do not pass Dockview-specific sizing through panel props. +- Runtime panel add/update/remove, sizing, placement, focus, visibility, and state commands share the same control path in CLI and Python. Use `hyperview ui panel ...` or `session.ui`/`session.control`; do not call raw panel-control HTTP routes from examples or demos. +- When invoking raw commands, prefer canonical command names: `workspace.panel.add/update/remove/resize/move/focus/close/show`, `workspace.panel.state.get/patch`, `panel.samples.retrieval.*`, `collection.neighbors.create`, and `collection.filter.set`. Treat legacy `ui.*` command ids as deprecated aliases. - To retitle an existing runtime panel or replace its props, use `hyperview ui panel update --panel-id ... --title ... --props-json ...` instead of remove/re-add when preserving panel identity matters. -- For nearest-neighbor comparisons, use `hyperview ui similarity set --sample-id ... --layout-key ...` or panel SDK `commands.showSimilar(...)`; do not infer neighbor space from whichever scatter panel is focused. +- For nearest-neighbor comparisons, use `hyperview ui samples retrieval set-anchor --sample-id ... --layout-key ...` or run `panel.samples.retrieval.set-anchor` / `collection.neighbors.create` through the SDK command client; do not infer neighbor space from whichever scatter panel is focused. +- For collection-backed Samples panel actions, use `hyperview panel samples show-neighbors ...` and `hyperview panel labels filter ...`; read the returned `result.collection_id` and `result.collection`. +- Use `hyperview ui panel state get/patch` or SDK `usePanelState()` when a panel needs durable panel-owned state. Keep durable state under runtime panel state instead of browser local storage or ad hoc events. +- For reset controls in panels, run the relevant runtime command through the SDK command client when both selection and nearest-neighbor context should be cleared. +- Do not use timers or browser storage to wait for panel readiness or guard startup state. Write the intended state through runtime commands or `usePanelState()`. +- When a panel needs to update another runtime panel's documented props, run `workspace.panel.update` through the SDK command client; do not hand-roll panel-control HTTP requests from panel code. - For extensions, prefer `.hyperview/extensions//` in the project root. `hyperview serve` auto-discovers those folders and attaches them to the launched workspace, so they can live in version control with the dataset/project code. - For demos/spaces that launch HyperView from Python, compose panels with `hv.ui.Horizontal`, `hv.ui.Vertical`, `hv.ui.Tabs`, `hv.ui.Grid`, `hv.ui.Scatter`, `hv.ui.Samples`, `hv.ui.ExtensionPanel`, and `hv.ui.PanelLayout`; keep extension manifests focused on reusable panel/tool definitions. - Keep layout orchestration out of panel modules. A panel should not close, hide, or rearrange sibling panels on mount; use `hv.ui.View(...)` or `hyperview ui panel ...` to compose the workspace. -- Treat local `focusPanel` and `closePanel` as transient user-action helpers, not as startup layout machinery. For durable control from a panel, use SDK commands such as `setActivePanel`, `setPanelVisible`, `resizePanel`, and `movePanel`. +- Treat host focus/resize helpers as transient user-action helpers, not as startup layout machinery. For durable control from a panel, run the corresponding `workspace.panel.*` command. - Pass only documented panel props through `hv.ui.ExtensionPanel(..., props=...)`. - Tools can write files under `ctx.extension_storage` and return `ctx.url_for(path)` for panel-renderable artifact URLs. - Put query results, benchmark tables, contact sheets, and other generated artifacts behind extension tools or compact panel props. Do not embed large base64 payloads or generated datasets inside panel JavaScript. @@ -94,6 +107,7 @@ Read [references/extensions.md](references/extensions.md) when the task involves - Treat the workspace as the durable unit. Changing datasets means setting a new workspace dataset, not switching among many datasets inside one workspace. - Prefer panel modules over raw HTML. The panel system no longer relies on iframes. - For paper diagrams, prefer `hyperview figure export` over browser screenshots unless the user explicitly needs exact UI chrome. +- For public, read-only examples-gallery demos, prefer `hyperview export --out bundle/` over a sleeping Python server. - For publication figures, keep the defaults first: `--theme light`, `--guide-style paper`, and `--legend auto`. Use `--show-selection` only when selected samples are meaningful and will be explained in the caption. - The first `uv run hyperview ...` invocation in a session can take 30+ seconds (torch/datasets imports). Allow generous timeouts and avoid sending SIGINT. @@ -101,8 +115,8 @@ Read [references/extensions.md](references/extensions.md) when the task involves The runtime exposes JSON discovery endpoints alongside the CLI. Use them to obtain layout keys, sample IDs, and registered tools/panels for follow-up commands: -- `GET /api/runtime?workspace_id=` — full snapshot. Read `workspace.ui.active_layout_key`, `workspace.ui.selected_ids`, `workspace.ui.custom_panels[*].data.module_src`, and registered `extensions`/`tools`. +- `GET /api/runtime?workspace_id=` — full snapshot. Read `workspace.ui.active_layout_key`, `workspace.ui.selected_ids`, `workspace.ui.panels`, `workspace.ui.custom_panels[*].state`, `workspace.ui.custom_panels[*].data.module_src`, and registered `extensions`/`tools`. - `GET /api/embeddings?workspace_id=` — the active or default layout, including `layout_key`, `geometry`, and sample `ids`. Use the returned `layout_key` for `hyperview ui layout set --layout-key ...` and pick from `ids` for `hyperview ui selection set --ids ...`. - `GET /api/tools` — registered tool URIs (also returned by `hyperview tools list --json`). -Prefer layout metadata over parsing layout-key strings. Use `/api/dataset`, `usePanelLayouts()`, or `dataset.list_layouts()` when filtering by geometry, dimension, model, or space. +Prefer layout metadata over parsing layout-key strings. Use `/api/dataset`, exported runtime snapshots, or `dataset.list_layouts()` when filtering by geometry, dimension, model, or space. diff --git a/.agents/skills/hyperview-cli/references/commands.md b/.agents/skills/hyperview-cli/references/commands.md index 2e0eb08..54cd3b2 100644 --- a/.agents/skills/hyperview-cli/references/commands.md +++ b/.agents/skills/hyperview-cli/references/commands.md @@ -60,6 +60,17 @@ hyperview dataset create cifar10_demo \ --label-key label ``` +Create a multimodal dataset with captions: + +```bash +hyperview dataset create coco_captions_demo \ + --hf-dataset HuggingFaceM4/COCO \ + --split train \ + --image-key image \ + --text-key sentences \ + --samples 500 +``` + Create a persisted dataset from a local image directory: ```bash @@ -204,6 +215,40 @@ hyperview figure export figures/embedding-panel-a.png \ --title "ArcFace spherical embeddings" ``` +## Static Demo Bundles + +Export a read-only, self-contained bundle for a workspace: + +```bash +hyperview export research --out dist/research-demo +``` + +The bundle contains the packaged static frontend, `api/runtime.json`, +`api/dataset.json`, sample shards under `api/samples/`, media and thumbnails, +layout coordinate JSON under `api/embeddings/`, materialized collection items +under `api/collections/`, and extension panel modules under +`api/panels/content/`. + +The generated `index.html` sets `window.__HYPERVIEW_STATIC__ = true`. In this +mode the frontend reads JSON files from the bundle, keeps selection and panel +state changes client-side, and disables mutations with the visible notice +`Read-only demo — pip install hyperview for the full workbench`. + +Python launch/session code can export the same bundle: + +```python +session = hv.launch(dataset, block=False) +session.export("dist/research-demo", workspace_id="research") +``` + +For persisted workspaces, use the top-level API: + +```python +import hyperview as hv + +hv.export_workspace("research", "dist/research-demo") +``` + ## Runtime UI Discover an existing layout key and sample IDs before mutating runtime state: @@ -215,6 +260,17 @@ curl --max-time 2 'http://127.0.0.1:6262/api/embeddings?workspace_id=research' | If no layout exists yet (`active_layout_key` is `null` and `/api/embeddings` returns nothing), create one with `hyperview embeddings compute ... --layout euclidean:2d` (creates embeddings + layout) or `hyperview layouts compute ... --space-key --layout euclidean:2d` (adds a layout to an existing embedding space). +Runtime command ids are namespaced: + +- `workspace.*` for workspace/view/panel placement and panel-owned state storage +- `panel..*` for panel-owned transitions such as Samples retrieval +- `collection.*` for materialized filters, neighbors, and query result sets + +Every control command returns a `CommandResult` envelope with `ok`, `command`, +`result`, `workspace`, `snapshot`, `revision`, and optional `error`. Apply the +returned `snapshot` when present. Do not issue an immediate `/api/runtime` +refetch unless a legacy endpoint did not return a snapshot. + Switch the live UI to a layout and selection: ```bash @@ -224,7 +280,9 @@ hyperview ui selection set --workspace research --ids sample-1,sample-8 `--layout-key` must be an existing layout (use the `layout_key` returned by `/api/embeddings`). When the chosen layout is Euclidean 3D, HyperView opens or focuses the Euclidean 3D scatter panel. -Add a custom panel through an extension: +Add a custom panel through an extension. Panel add/update/remove and panel +layout controls share HyperView's public control command path; prefer these CLI +commands or the matching Python `session.ui` helpers in examples. ```bash hyperview extension add .hyperview/extensions/label-histogram \ @@ -328,6 +386,22 @@ hyperview ui panel close --workspace research --panel-id notes hyperview ui panel show --workspace research --panel-id notes ``` +Read or patch durable panel-owned state: + +```bash +hyperview ui panel state get \ + --workspace research \ + --panel-id samples \ + --json + +hyperview ui panel state patch \ + --workspace research \ + --panel-id samples \ + --state-json '{"settings":{"density":"compact"}}' \ + --expected-revision 0 \ + --json +``` + Remove a runtime panel by id: ```bash @@ -336,20 +410,52 @@ hyperview ui panel remove \ --panel-id hycoclip-poincare ``` -Pin nearest-neighbor results to a specific embedding layout: +Pin nearest-neighbor results to the Samples panel state for a specific embedding layout: ```bash -hyperview ui similarity set \ +hyperview ui samples retrieval set-anchor \ --workspace research \ --sample-id \ --layout-key \ --k 18 + +hyperview ui samples retrieval set-k \ + --workspace research \ + --k 36 + +hyperview ui samples retrieval set-text \ + --workspace research \ + --query "a dog playing in the park" \ + --layout-key \ + --k 18 ``` -Clear the explicit nearest-neighbor context: +Clear the explicit Samples retrieval context: ```bash -hyperview ui similarity clear --workspace research +hyperview ui samples retrieval clear --workspace research +``` + +Use `hyperview ui samples retrieval ...` for compatibility with existing CLI +flows. Raw commands should use the canonical `panel.samples.retrieval.*` +command ids. + +Use panel collection shortcuts when the desired outcome is a Samples panel collection: + +```bash +hyperview panel samples show-neighbors \ + --workspace research \ + --sample-id \ + --space-key \ + --k 18 \ + --json + +hyperview panel labels filter \ + --workspace research \ + --value cat \ + --json + +hyperview panel labels filter --workspace research --clear ``` ## Extensions and Tools diff --git a/.agents/skills/hyperview-cli/references/extensions.md b/.agents/skills/hyperview-cli/references/extensions.md index c153967..e89f756 100644 --- a/.agents/skills/hyperview-cli/references/extensions.md +++ b/.agents/skills/hyperview-cli/references/extensions.md @@ -107,59 +107,48 @@ Panel modules must be browser-loadable JavaScript modules. They export a default const sdk = globalThis.HyperViewPanelSDK; if (!sdk) throw new Error("HyperViewPanelSDK is not available on window."); -const { React, components, hooks } = sdk; -const { Panel, PanelToolbar, PanelToolbarButton } = components; -const { usePanelSelection, useTool } = hooks; +const { React, hooks } = sdk; +const { useSelection, usePanelState } = hooks; export default function SelectionProfilePanel() { - const { selectedIds } = usePanelSelection(); - const profile = useTool("selection_profile.summarize"); + const { selectedIds } = useSelection(); + const { state, patchState } = usePanelState(); const selectionKey = selectedIds.join("|"); React.useEffect(() => { - profile.run({ sample_ids: selectedIds }); - }, [profile.run, selectionKey]); - - const result = profile.result; + patchState({ last_selection: selectedIds }); + }, [patchState, selectionKey]); return React.createElement( - Panel, - { className: "h-full" }, - React.createElement(PanelToolbar, { - items: [ - { id: "selection", label: "Selection", value: String(selectedIds.length) }, - { id: "status", label: "Status", value: profile.loading ? "running" : result ? "ready" : "idle" }, - ], - actions: React.createElement(PanelToolbarButton, { onClick: () => profile.run({ sample_ids: selectedIds }) }, "Refresh"), - }), - React.createElement("pre", { style: { padding: 12, overflow: "auto" } }, JSON.stringify(result, null, 2)) + "main", + { style: { padding: 12, font: "12px system-ui" } }, + React.createElement("div", null, `Selected: ${selectedIds.length}`), + React.createElement("pre", null, JSON.stringify(state, null, 2)) ); } ``` -Available SDK hooks include `usePanelRuntimeState`, `usePanelHostState`, `usePanelDatasetInfo`, `usePanelSamplesView`, `usePanelSelectedSamples`, `usePanelSelection`, `usePanelHover`, `usePanelLayouts`, `usePanelLayoutView`, `usePanelCommands`, `usePanelUiState`, `usePanelClient`, and `useTool`. - -For dataset-wide panel behavior, prefer `usePanelClient().querySamples(...)`, -`aggregateSamples(...)`, `selectSamples(...)`, `getSamplesByIds(...)`, -`searchSimilar(...)`, or an extension tool over scanning a fixed -`listSamples({ limit: ... })` page or hand-building API URLs in the browser. -Sample reads default to `includeThumbnails: false`; use each sample's -`thumbnail_url` for images, and request inline thumbnails only when a panel -explicitly needs base64 data. - -Use `usePanelHostState()` for synchronized host state. Use narrower hooks such as `usePanelSelection()`, -`usePanelSelectedSamples()`, `usePanelHover()`, `usePanelLayouts()`, and -`usePanelLayoutView()` when the panel only needs one part of that state. Use -`usePanelCommands()` for host writes. Selection and active-layout changes -update host state immediately and persist to runtime UI state in the -background by default. Pass `{ persist: true }` only when the caller must wait -for durable runtime state, and pass `{ persist: false }` for local transient UI -changes. +Available SDK hooks are intentionally thin: `useCommandClient`, `usePanelState`, +`useSelection`, `useCollection`, `useSamples`, and `useHostAdapter`. -Do not use browser globals such as `window.dispatchEvent` to synchronize panels, -and use SDK commands for control-plane writes. +For dataset-wide panel behavior, prefer runtime collections and +`useSamples(collectionId)` over scanning a fixed page or hand-building API URLs +in the browser. When a panel needs a new filtered or nearest-neighbor result +set, run `collection.filter.set`, `collection.neighbors.create`, or +`panel.samples.retrieval.*` through `useCommandClient()`. -`useTool(uri)` returns `{ run, result, loading, error, reset }`. Call `run(params)` to invoke the tool; `result` holds the last successful return value, `loading` is true while a call is in flight, and `error` is the last failure message (or `null`). See [panel-modules.md](panel-modules.md#hook-return-shapes) for the full hook return shape table. +Use `useSelection()` for synchronized selection state. Use `usePanelState()` for +panel-owned props/state. Use `useHostAdapter()` only for transient host actions +such as focus; durable layout/state changes should go through `workspace.*` +commands. In static exports, mutating commands are disabled, while selection and +panel state patches remain client-side and ephemeral. + +Do not use browser globals such as `window.dispatchEvent` to synchronize panels, +and use SDK commands for control-plane writes. Use `usePanelState()` for durable +panel-owned state. Run `workspace.panel.update` when a panel needs to update +another runtime panel instance's documented props. Python tools are invoked from +the CLI/API with `hyperview tools run` or by higher-level extension flows; do not +assume a browser `useTool` hook is present in the thin SDK. ## CLI Workflow diff --git a/.agents/skills/hyperview-cli/references/panel-modules.md b/.agents/skills/hyperview-cli/references/panel-modules.md index a27c059..0caf3b0 100644 --- a/.agents/skills/hyperview-cli/references/panel-modules.md +++ b/.agents/skills/hyperview-cli/references/panel-modules.md @@ -32,26 +32,24 @@ A runtime panel module must export either: The module runs in the browser and should use the SDK from `window.HyperViewPanelSDK`. When a Python view provides panel `props`, HyperView passes them to the component as the `props` prop, alongside `panel` and `panelId`; panel code can also read -them with `HyperViewPanelSDK.hooks.usePanelProps()`. +them from `HyperViewPanelSDK.hooks.usePanelState().props`. Minimal example: ```js const sdk = globalThis.HyperViewPanelSDK; -const { React, components, hooks } = sdk; -const { Panel, PanelToolbar } = components; -const { usePanelRuntimeState } = hooks; +const { React, hooks } = sdk; +const { usePanelState, useSamples } = hooks; export default function MyPanel() { - const { runtimeDatasetName } = usePanelRuntimeState(); + const { panelId, props } = usePanelState(); + const { samples, total } = useSamples(props.collection_id); return React.createElement( - Panel, - { className: "h-full" }, - React.createElement(PanelToolbar, { - items: [{ id: "dataset", label: "Dataset", value: runtimeDatasetName || "unknown" }], - }), - React.createElement("div", { style: { padding: 12 } }, "Hello from a panel module") + "main", + { style: { padding: 12, font: "12px system-ui" } }, + React.createElement("div", null, `Panel: ${panelId}`), + React.createElement("div", null, `Samples visible: ${samples.length} / ${total}`) ); } ``` @@ -61,60 +59,39 @@ export default function MyPanel() { Current global SDK fields: - `React` -- `components.Panel` -- `components.PanelHeader` -- `components.PanelTitle` -- `components.PanelToolbar` -- `components.PanelToolbarButton` -- `components.PanelToolbarMenu` -- `hooks.usePanelClient()` -- `hooks.usePanelCommands()` -- `hooks.usePanelDatasetInfo()` -- `hooks.usePanelHostState()` -- `hooks.usePanelHover()` -- `hooks.usePanelInstance()` -- `hooks.usePanelLayouts()` -- `hooks.usePanelLayoutView()` -- `hooks.usePanelProps()` -- `hooks.usePanelRuntimeState()` +- `hooks.useCommandClient()` +- `hooks.usePanelState()` - `hooks.usePanelSamples()` -- `hooks.usePanelSamplesView()` -- `hooks.usePanelSelectedSamples()` -- `hooks.usePanelSelection()` -- `hooks.usePanelUiState()` -- `hooks.useTool(uri)` +- `hooks.useSelection()` +- `hooks.useCollection(collectionId)` +- `hooks.useSamples(collectionId)` +- `hooks.useHostAdapter()` - `createClient(workspaceId)` Important distinction: -- `usePanelSamplesView()` gives access to host-managed collection state and is the best hook for panels that should stay synchronized with the visible HyperView UI. -- `usePanelHostState()` gives read access to the same host state used by built-in panels. -- `usePanelClient()` is an escape hatch for API data reads that are not already exposed through host state or commands. -- `useTool(uri)` calls an installed Python tool registered by an extension and returns `{ loading, result, error, run, reset }`. +- `useCommandClient()` discovers and runs backend-owned control commands. Command results include snapshots; apply them instead of refetching runtime state. +- `usePanelState()` reads concrete panel props/state and patches panel-owned state through `workspace.panel.state.patch`. +- `useSelection()` exposes current selection and selection setters. +- `useCollection(collectionId)` reads runtime collection metadata. `useSamples(collectionId)` materializes `all`/`filter`/`neighbors`/`search` collections through the paged `GET /api/collections/{id}/items` endpoint (call `loadMore()` while `hasMore`); other kinds fall back to the host-loaded sample page. `scores` carries per-sample distances for neighbors/search collections. +- `useHostAdapter()` exposes host-only focus/resize helpers. Use `workspace.panel.*` commands for durable panel layout changes. ### Hook return shapes Current hook return shapes: -- `usePanelSelection()` → `{ selectedIds: string[], selectionSource: SelectionUpdateSource }` -- `usePanelCommands()` → `{ setLabelFilter, setHoveredId, clearLassoSelection, clearSelection(): void, setSelection(ids, { source?, persist?, clearLasso? }): Promise, showSimilar({ sampleId, layoutKey, k?, source?, focus?, persist? }): Promise, setActiveLayout(layoutKey, { persist? }): Promise, setLayoutViewCamera(layoutKey, camera3d): void, setLayoutViewCameraPersisted(layoutKey, camera3d): Promise, setPanelLayout(panelId, { width?, height?, minWidth?, minHeight?, maxWidth?, maxHeight? }), resizePanel(panelId, { width?, height?, minWidth?, minHeight?, maxWidth?, maxHeight? }), movePanel(panelId, { position, referencePanelId?, direction? }), setPanelVisible(panelId, visible), setActivePanel(panelId), focusPanel(panelId): boolean, focusBuiltin(role): boolean, focusPanelByRole(role): boolean, closePanel(panelId): boolean }`. `showSimilar` is layout-scoped: pass the layout key and HyperView resolves the associated embedding space automatically. `persist` accepts `true`, `false`, or `"background"` for selection/layout/similarity commands; omitted/`"background"` updates local state immediately and writes runtime state asynchronously, `true` waits for runtime persistence, and `false` is local-only. -- `usePanelHover()` → `{ hoveredId, setHoveredId(id), clearHover() }` -- `usePanelLayoutView(layoutKey?)` → `{ layoutKey, view, camera3d, setCamera3d(camera3d) }` -- `usePanelLayouts()` → `{ layouts, spaces, get(layoutKey), getSpace(spaceKey), find(query), filter(query) }`; query supports `layoutKey`, `spaceKey`, `geometry`, `modelId`, and `dimension`. -- `usePanelSelectedSamples({ includeThumbnails? })` → `{ selectedIds, samples, loading, error }` -- `usePanelRuntimeState()` → `{ activeWorkspaceId, runtimeDatasetName, activeLayoutKey, activeSimilarityQuery, requestedLayoutKey, workspaces, customPanels, viewRevision, layoutViews }` -- `usePanelHostState()` → grouped low-level host state: `{ instance, runtime, datasetInfo, samples, samplesView, selection, hover, ui, filters, lasso, neighbors }` -- `usePanelProps()` → props supplied by the concrete `hv.ui.ExtensionPanel(...)` instance -- `usePanelUiState()` → `{ sampleGridSize, setSampleGridSize, scatterLabelOverlayMode, setScatterLabelOverlayMode }` -- `usePanelDatasetInfo()` / `usePanelSamples()` / `usePanelSamplesView()` → host-managed dataset and view state -- `useTool(uri)` → `{ loading: boolean, result: TResult | null, error: string | null, run(params?): Promise, reset(): void }` -- `usePanelClient()` → API data client. Prefer host hooks and `usePanelCommands()` for UI state; useful data methods include `querySamples`, `aggregateSamples`, `getSamplesByIds`, `searchSimilar`, and `selectSamples`. +- `useCommandClient()` → `{ listCommands(): Promise, runCommand(command, envelope?): Promise }` +- `usePanelState()` → `{ panel, panelId, props, state, stateRevision, patchState(statePatch, { replaceState?, expectedRevision? }): Promise }` +- `useSelection()` → `{ selectedIds: string[], selectionSource, setSelection(ids): Promise, clearSelection(): Promise }` +- `useCollection(collectionId)` → `RuntimeCollection | null` +- `useSamples(collectionId, { pageSize? })` → `{ collection, samples, scores, total, loading, error, hasMore, loadMore }` +- `useHostAdapter()` → `{ focusPanel(panelId): boolean, resizePanel(panelId, options): Promise }` Sample reads default to `includeThumbnails: false` and return `thumbnail_url` for image rendering. Request inline thumbnails only when the panel specifically needs base64 thumbnail payloads. -To clear the current selection from a panel and enqueue runtime persistence, use `await usePanelCommands().setSelection([])`. Pass `{ persist: true }` when the code must wait for runtime persistence, and pass `{ persist: false }` only for local transient UI changes. +To clear the current selection from a panel, use `await useSelection().clearSelection()`. To create nearest-neighbor or filtered Samples state, run `collection.neighbors.create`, `collection.filter.set`, or `panel.samples.retrieval.*` through `useCommandClient().runCommand(...)`. ## Placement @@ -147,17 +124,15 @@ hyperview ui panel add \ ## Good Practices - Prefer panel modules over HTML or iframe content. -- Use `usePanelSamplesView()` for view-synchronized behavior. -- Use `usePanelCommands()` for host interactions such as label filtering or selection changes. -- Use `usePanelHostState()` when a custom panel needs low-level synchronized host state such as hover, selection, lasso, neighbors, layout views, or active workspace context. -- Use `usePanelLayouts()` for layout/space lookup instead of scanning `datasetInfo.layouts` by hand. -- Use `usePanelSelectedSamples()` for selected sample metadata instead of manually watching selected ids and fetching samples. -- Use `usePanelClient()` only for data that is not already available through the host state. -- Use `usePanelClient().querySamples(...)`, `aggregateSamples(...)`, or `selectSamples(...)` for dataset-wide behavior instead of fixed-limit client scans. -- Use `usePanelClient().searchSimilar(...)` instead of hand-building `/api/search/similar/...` URLs. -- Use `commands.showSimilar({ sampleId, layoutKey })` for nearest-neighbor UI state; pass the layout key and let HyperView resolve the embedding space. -- Use SDK commands or client methods for control-plane writes. -- Use `setActivePanel`, `setPanelVisible`, `resizePanel`, and `movePanel` when a panel needs to durably control panel view state. Use local `focusPanel` and `closePanel` only for transient user actions. +- Use `useSamples(collectionId)` for host-loaded samples and collection-backed display. +- Use `useSelection()` for selection reads and selection changes. +- Use `useCommandClient()` for control-plane writes and command discovery. +- Run `collection.neighbors.create` or `panel.samples.retrieval.set-anchor` for nearest-neighbor UI state; pass the layout key and let HyperView resolve the associated embedding space. +- Use `usePanelState()` for durable panel-owned state. Patch with `expectedRevision` when concurrent edits would lose user work. +- Do not use browser storage, ad hoc events, or timers to coordinate startup state or cross-panel readiness; write durable intent through runtime commands or `usePanelState()`. +- Run `workspace.panel.update` instead of hand-building panel-control HTTP requests from panel modules. +- Use the command client for control-plane writes. +- Use `workspace.panel.focus`, `workspace.panel.show/close`, `workspace.panel.resize`, and `workspace.panel.move` when a panel needs to durably control panel view state. Use host adapters only for transient user actions. - Do not use `window.dispatchEvent` / `window.addEventListener` to synchronize panel state. Keep shared state in the host/runtime model, or keep the interaction inside one owner panel until a public shared-state hook exists. - Do not use `focusPanel` or `closePanel` from mount effects to create the initial workspace layout. Compose startup layout with `hv.ui.View(...)` or CLI panel commands. - Pass only documented panel props. diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a63397c..c533f2b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -30,9 +30,9 @@ jobs: # Build frontend - name: Set up Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v6 with: - node-version: 20 + node-version: 24 cache: npm cache-dependency-path: frontend/package-lock.json @@ -61,7 +61,7 @@ jobs: python -c "import hyperview; print(f'hyperview {hyperview.__version__}')" - name: Upload dist artifacts - uses: actions/upload-artifact@v5 + uses: actions/upload-artifact@v6 with: name: python-package-distributions path: dist/ diff --git a/docs/parity-assessment-2026-07.md b/docs/parity-assessment-2026-07.md new file mode 100644 index 0000000..3d98581 --- /dev/null +++ b/docs/parity-assessment-2026-07.md @@ -0,0 +1,215 @@ +# HyperView: architecture verdict and a path to parity with Encord / FiftyOne / Rerun + +Written 2026-07-04, immediately after the panel/control refactor (PR #25) +landed. Grounded in the current code on `refactor/panel-control-2026-07-cont`, +`docs/architecture.md`, and the July gap assessment. + +## 1. Verdict on the current architecture + +**The control plane is architected correctly — and it is the part none of the +three competitors have.** The things that are usually wrong in a tool at this +stage are right here: + +- **Machine-readable outcomes end to end.** Every mutation goes through a + namespaced command (`workspace.*`, `panel..*`, `collection.*`) and + returns a `CommandResult` carrying a runtime snapshot. An agent can drive + the entire workbench without scraping UI state. FiftyOne's operators come + closest, but they grew out of a plugin system; HyperView's command plane is + the primary interface, not an add-on. +- **Runtime/workspace state as source of truth.** Layouts resolve server-side, + panel state is runtime-owned with revisions, selection is a control-plane + write. The frontend is a renderer, which is why the static export + (`hyperview export`) fell out almost for free — that trick is only possible + because no state of record lives in the browser. +- **Collections as first-class query objects.** `all`/`filter`/`neighbors`/ + `search` collections store the *query*, membership is re-materialized on + read (`GET /api/collections/{id}/items`). This is the correct seed for a + real view/query system: it is FiftyOne's `DatasetView` concept in embryo, + and it is already wired through commands, REST, the panel SDK, and static + export. +- **Explicit extension surfaces.** Provider registry, jobs, panel contracts + single-sourced from Python, extension panels as JS modules loaded through + the same `PanelHost` path as built-ins. The built-in/extension symmetry is + something FiftyOne only reached after years. +- **Representation/index split (contract level).** The right long-term shape + for multi-vector / late-interaction / hybrid search, landed without a + storage migration. + +**Where it is architecturally thin is the data model — and that is the entire +gap to the other three tools.** Everything below the control plane assumes: +one dataset per workspace, one entity set (`samples`), one media type +(image + optional text), one annotation type (a string `label`), metadata as +an untyped dict. `EntityRef` (`dataset_id`/`entity_set_id`/`entity_id`) +exists in the object model, but nothing exercises it beyond `"samples"`. +Filters understand `label == value` fast-path and in-memory metadata equality, +and nothing else. There is no schema an agent or panel can discover +(`architecture.md` names this: typed Fields, priority Medium). + +So: **correctly architected control plane and extension system; deliberately +minimal data plane.** The order was right — a rich data model behind a bad +control plane is what most tools have, and it's much harder to fix in that +direction. But every parity feature below hangs off the data plane. + +Two structural debts worth naming before they calcify: + +1. **No frontend test harness.** The Phase 3 hook rewire shipped + build-verified only. Before panels get more logic, add a minimal vitest + + React Testing Library setup for the panel SDK hooks and one built-in panel. +2. **In-memory materialization paths.** Non-label filters and several + resolvers load `ds.samples` wholesale. Fine at 10k samples, dead at 1M. + Query pushdown into LanceDB (it can do filtered vector search + SQL-ish + predicates) is the scaling move, and it gets harder the more code grows on + top of the in-memory idiom. + +## 2. What "parity" actually means against these three + +The three tools are not one category. Chasing literal feature parity with all +of them is a multi-year, multi-team platform effort and the wrong goal for +where the company is (zero customers, eval→pilot motion, solo founder). +The useful framing: **which of their capabilities does HyperView's actual +buyer (hierarchy-aware retrieval evals/pilots, dataset inspection) hit a wall +without?** + +| Capability | FiftyOne | Encord | Rerun | HyperView today | +|---|---|---|---|---| +| Agent-native control plane, machine-readable outcomes | partial (operators) | API/SDK, not agent-first | no | **yes — differentiator** | +| Embedding spaces, similarity/text search, layouts | Brain (strong) | Active (embedding views) | weak | **yes**, incl. hyperbolic — differentiator | +| Static shareable demos, no backend | no | no | .rrd recordings + web viewer | **yes** — differentiator | +| Typed field schema, discoverable by UI | yes | ontologies | ECS archetypes | **no — root gap** | +| Rich label types (boxes, masks, keypoints, polylines) | yes | yes | as renderables | no (string label only) | +| Query/filter DSL over fields | ViewExpressions (strong) | filters + metrics | dataframe API | `label == value` | +| Model evaluation (mAP, confusion, PR, comparisons) | yes | yes | no | no | +| Data/label quality metrics (uniqueness, near-dupes, label errors) | Brain | Active (core) | no | no (but embeddings already in place) | +| Video / frame-level data | yes | yes (core) | yes (core, temporal) | no | +| 3D / point clouds / spatial transforms | yes (FO3D) | limited | yes (core) | no | +| Multi-view / grouped samples (e.g. product SKU with N views) | grouped datasets | yes | entity tree | no | +| Annotation workflows, review/QA, ontologies, teams | integrates out | **core product** | no | no | +| Time-series / streaming ingestion SDK | limited | no | **core product** | no | +| Scale beyond memory | MongoDB-backed views | cloud | out-of-core recordings | partial (LanceDB underused) | + +Read the column and the strategy writes itself: HyperView should reach +**functional parity with FiftyOne's curation/inspection core**, adopt +**Encord Active's quality-metrics ideas** (not Encord's annotation factory), +and borrow **Rerun's distribution ideas** (recordings/blueprints ≈ static +bundles/runtime layouts, where HyperView is already ahead). Full Encord +annotation workflows, DICOM, and Rerun's robotics streaming are **non-goals** +— integrate or ignore. + +## 3. Gap analysis, ordered by how much everything else depends on it + +### Gap 0 — Typed fields / generic dataset records (the root) + +Already named in `architecture.md` and the gap assessment as the one real +architectural gap. Everything in section 2's "no" column needs it: label +types are field types; query DSL needs a schema to validate against; quality +metrics need typed numeric fields to write scores into; multi-view needs +entity sets; video needs a `frames` entity set with a time field. + +Shape (per architecture.md's own vocabulary): `Field{path, kind, dtype, +schema}` registered per entity set, discoverable via `/api/dataset`, with +media/annotation/metadata/numeric/representation/coordinate kinds. The +`--image-key/--label-key` CLI flags become the first two entries of a real +`FieldMapping` instead of hardcoded parameters. + +### Gap 1 — Query algebra over collections + +`filter` collections should accept a small, composable predicate tree +(`and`/`or`/`not`, `eq/ne/in/lt/gt/contains/exists`, over typed field paths) +instead of a single field/op/value. This is deliberately *not* FiftyOne's +full ViewExpression language — a JSON predicate tree is easier for agents to +emit reliably, which turns the biggest FiftyOne feature into an agent-native +strength. Push predicates down to LanceDB where possible; that also retires +the in-memory scan debt (§1.2). + +### Gap 2 — Label types beyond `label: str` + +Detections (boxes), segmentation masks, keypoints, classifications-with- +confidence as field kinds; overlay rendering in the samples grid and a +region/crop view (RefCOCOg-style region search is already a sales demo — +today it's done with pre-cropped images because there is no box field). + +### Gap 3 — Evaluation + quality metrics ("Brain/Active-lite") + +Two halves, both jobs that write typed fields (which is why Gap 0 comes +first): + +- **Retrieval evaluation first, detection later.** HyperView's buyers are + retrieval buyers. `evaluate.retrieval` (qrels → R@K, mAP, MRR, leakage@K, + parent-P@K per collection/index) is directly the customer-eval workflow the + commercial assessment says must be standardized. FiftyOne parity here is + COCO-style `evaluate_detections`; that can wait for Gap 2. +- **Embedding-derived quality metrics**: uniqueness/near-duplicates (vector + index already exists), label-outlier scores (label vs. embedding + neighborhood disagreement — with hyperbolic geometry this is a + *differentiated* label-error detector for hierarchies), coverage per + taxonomy branch. This is Encord Active's core value, and HyperView already + owns the hard part (the embeddings). + +### Gap 4 — Grouped / multi-view samples and second entity sets + +First real use of `entity_set_id != "samples"`: product → N views, image → +regions. Directly serves catalog customers (multi-view SKU retrieval is in +the commercial assessment's eval wishlist). + +### Gap 5 — Video/frames and 3D + +Video = a `frames` entity set with a time axis plus frame-level fields — +architecturally just Gap 0 + Gap 4 applied again, but a large UI investment +(scrubbing, per-frame overlays). 3D/point-cloud rendering is a new renderer +class. Neither serves the current pipeline; sequence them behind demand from +an actual deal (an industrial-inspection or AV lead would pull video +forward). + +### Gap 6 — Ecosystem: annotation round-trip and import/export + +Parity move is FiftyOne's, not Encord's: export a collection to CVAT/Label +Studio/Encord, re-import labels into fields. Plus COCO/YOLO/HF dataset +import-export. This makes HyperView composable with annotation factories +instead of competing with them. + +## 4. Sequenced roadmap + +Ground rules carried over from the refactor: commands-first, LanceDB stays, +every phase lands green, skill docs updated. Sizes are relative +(S ≈ days, M ≈ 1–2 weeks, L ≈ 3+ weeks solo). + +| Phase | Contents | Size | Unblocks | +|---|---|---|---| +| **P1. Fields & records** | Field registry per entity set, typed field kinds, `FieldMapping` on ingestion, `/api/dataset` schema discovery, frontend sidebar driven by schema | M | everything | +| **P2. Query algebra** | JSON predicate tree in `filter` collections, LanceDB pushdown, `collection.filter.set` accepts it, sidebar filter UI emits it | M | FiftyOne-core parity | +| **P3. Retrieval eval + quality jobs** | `evaluate.retrieval` job + report panel; uniqueness/near-dupes/label-outlier jobs writing typed fields | M | customer evals, Active-lite | +| **P4. Label types + overlays** | detections/masks/keypoints field kinds, grid overlays, region view | L | Encord/FO annotation-adjacent parity | +| **P5. Groups & entity sets** | grouped samples, regions as entity set, group-aware grid | M | catalog/multi-view deals | +| **P6. Annotation round-trip + formats** | CVAT/Label Studio/Encord export-import, COCO/YOLO/HF formats | M | ecosystem composability | +| **P7. Video (demand-gated)** | frames entity set, time axis, scrubber UI | L | inspection/AV verticals | +| **P8. 3D (demand-gated)** | point-cloud renderer panel | L | robotics/AV verticals | + +Parallel hygiene (small, do alongside P1–P2): frontend test harness; +LanceDB-backed pagination for every remaining `ds.samples` scan; built-in +Samples grid rendering by `collection_id` (already a named follow-up). + +After P1–P3, HyperView is at *functional* parity with the FiftyOne workflows +its buyers actually use (curate → filter → embed → search → evaluate → +export), has Encord Active's most valuable ideas in differentiated form, and +keeps three moats none of them have: agent-native control, hyperbolic/ +hierarchy-aware retrieval, and zero-infra static demo bundles. That is +"parity where it matters"; literal parity (P4–P8) should be pulled by +revenue, not pushed by roadmap. + +## 5. Risks and honest caveats + +- **Sequencing risk:** P1 is the one phase where a wrong abstraction is + expensive (every later phase builds on Field). It deserves a design doc and + review before code, the way the July refactor plan did. +- **Solo-founder throughput:** P1–P3 is ~6–8 focused weeks in the best case. + Per the current strategy, sales blocks come first; this roadmap assumes + engineering happens in the remaining capacity, and P3 is the only phase + that directly manufactures sales artifacts (customer eval reports). If + pipeline pressure rises, do P3's retrieval-eval job *first* against the + current thin data model (qrels can be plain metadata) and accept rework. +- **Scale claims:** until LanceDB pushdown lands, don't demo above ~50k + samples live. Static bundles sidestep this for demos. +- **Competitor motion:** FiftyOne ships plugins fast and Voxel51 is + well-funded; the defensible ground is agent-nativeness and hierarchy-aware + retrieval, not feature count. Any quarter spent cloning their features + instead of deepening the wedge is a quarter they can't lose. diff --git a/docs/refactor-plan-2026-07-gap-assessment.md b/docs/refactor-plan-2026-07-gap-assessment.md new file mode 100644 index 0000000..86b703d --- /dev/null +++ b/docs/refactor-plan-2026-07-gap-assessment.md @@ -0,0 +1,80 @@ +# Gap assessment: refactor-plan-2026-07 vs the fuller AGENTS.md direction + +Assessed 2026-07-04, after Phase 1/2/4 landed. Cross-references AGENTS.md's +stated direction (agent-native workbench, provider registration, generic +dataset records/field mapping, embeddings, layouts, panels, jobs, +machine-readable outcomes) against the current codebase, to see what +`docs/refactor-plan-2026-07.md` still needs to cover beyond Phase 3. + +## 1. Provider registration — already adequate + +`ProviderRegistry` / `ProviderRegistration` (`src/hyperview/runtime.py:170-220`), +public `register_provider`/`unregister_provider` API (`src/hyperview/api.py:77-102`, +re-exported from `src/hyperview/__init__.py`). Embedding engine resolves +providers through it (`src/hyperview/embeddings/engine.py:130`, +`src/hyperview/embeddings/pipelines.py:34-106`). Real registry, not ad hoc. +**No gap.** + +## 2. Generic dataset records + explicit field mapping — partial gap + +Field mapping exists but is dataset-type-specific and hardcoded, not a generic +runtime concept: `--image-key`/`--label-key` CLI flags +(`src/hyperview/cli.py:188-210`) map directly to a fixed `image_key: str = "img"` +parameter on the HF ingestion path (`src/hyperview/core/dataset.py:252-438`). +There is no generic `FieldMapping`/`Field` schema that panels or extensions can +discover (per `docs/architecture.md`'s `Field` vocabulary). This matches +`docs/architecture.md`'s own "Current vs Target Model" table (Fields: "mostly +implicit" -> "typed fields/components discoverable by panels", priority +Medium). **Real gap, but architecture.md already scopes it Medium, not a Phase 3 +blocker.** Recommend as its own follow-up phase, not folded into Phase 3/4. + +## 3. Jobs — already adequate + +First-class `JobState` + `register_job` in `src/hyperview/runtime.py:2085-2145`, +run on background threads, tracked by id. Exposed via `/api/jobs` routes in +`server/app.py` (`list_jobs`, `get_job`). **No gap.** + +## 4. Machine-readable outcomes — already adequate + +`CommandResult` (`src/hyperview/control/models.py:51`) is the uniform shape +Phase 1 introduced; all control commands return it, asserted in +`tests/test_cli_control.py`. **No gap.** + +## 5. Extension surfaces — matches Phase 2 scope + +Panel extension via `RuntimeModulePanel`/`PanelHost` (Phase 2) plus Phase 4's +static export copying extension panel JS modules into the bundle. No evidence +of a second extension axis being asked for beyond what `register_provider` +already allows programmatically. **No gap beyond what AGENTS.md already +describes** — provider registration, native/runtime panels, and explicit +extension surfaces all have a concrete mechanism today. + +## 6. Layouts and embeddings as runtime-managed state — already adequate + +`space_key`/embedding spaces are runtime/storage-backed (`storage/backend.py`, +`lancedb_backend.py`, `memory_backend.py`), not frontend-local. Layouts resolve +server-side (`runtime.py:_resolve_retrieval_context`). Confirms "runtime/ +workspace state is the source of truth" is honored. **No gap.** + +## 7. Collections materialization (Phase 3, landed) + +`GET /api/collections/{id}` and `GET /api/collections/{id}/items` exist for +neighbors/search/filter/all collection kinds. `useSamples(collectionId)` in +`frontend/src/panel-sdk/index.tsx` now pages through that endpoint for those +kinds (51a712f), in live servers and static bundles alike; non-materializable +kinds (selection/lasso/tool_result/extension) keep the legacy client-side +filter. The `space_key` -> representation/index split landed at the contract +level (c90bfca): `/api/dataset` exposes derived `representations[]` and +`indexes[]`, and `index_id` (`space:`) is accepted at every +retrieval boundary. Remaining follow-ups: point the built-in Samples grid's +host view model at a `collection_id`, and eventually key storage by +representation/index instead of `space_key`. + +## Conclusion + +The only concrete architectural gap beyond what `refactor-plan-2026-07.md` +already scopes is generic field mapping / typed Field discovery — Medium +priority per architecture.md, own follow-up phase. Phase 3's frontend half +and the contract-level representation/index split have landed; the storage +rename and the built-in grid's collection_id rendering are named follow-ups. +No changes needed to phase ordering. diff --git a/docs/refactor-plan-2026-07.md b/docs/refactor-plan-2026-07.md new file mode 100644 index 0000000..e57ad2a --- /dev/null +++ b/docs/refactor-plan-2026-07.md @@ -0,0 +1,138 @@ +# Panel & Control Refactor Plan — July 2026 + +Status: active. Branch: `refactor/panel-control-2026-07` (WIP snapshot committed as baseline). + +## Why + +The uncommitted work of June/July 2026 (control command API, panel definitions, +Samples retrieval state, panel SDK expansion, text retrieval) was driven by the +hyperview-spaces demos. It moved the architecture in the right direction — +CLI/API-first, runtime as source of truth — but left migration debt behind. +This plan consolidates that work so the panel/control system matches +`docs/architecture.md` and `AGENTS.md` instead of accumulating demo-shaped +special cases. + +## Current pain points + +1. **Duplicated panel contracts.** Built-in panel definitions exist twice: + `src/hyperview/panel_definitions.py` (Python) and + `frontend/src/panels/definitions.tsx` (TS). They can silently drift. +2. **Three panel render paths.** Runtime scatter panels are special-cased apart + from `RuntimeBuiltInPanel.tsx` and `RuntimeModulePanel.tsx`; panel type vs + concrete Dockview tab id is muddled. +3. **Panel SDK is a leaky mega-surface.** `frontend/src/panel-sdk/index.tsx` + re-exports store internals, Dockview context, API helpers, tools, selection, + and layouts. It is becoming the de-facto frontend integration layer. +4. **Legacy state mirrors.** `ui.similarity_query` is still mirrored although + the source of truth is `ui.panels.samples.state.retrieval`. +5. **Mixed command taxonomy.** `ui.panel.*`, `samples.retrieval.*`, + `panel.samples.*`, `panel.labels.*` coexist without clear ownership. +6. **Collections are not yet the source of truth.** `CollectionState` exists, + but panels re-run searches and infer display behavior from Samples state + instead of rendering by `collection_id`. +7. **Inconsistent round trips.** Some command results carry a workspace + snapshot that the frontend ignores, then re-fetches `/api/runtime`. + +## Target architecture (from docs/architecture.md, made concrete) + +- **One canonical `PanelDefinition`** lives in Python. The frontend fetches + definitions from the runtime snapshot and only maps `panel_type` → + renderer component. No panel contract data is hardcoded in TS. +- **One panel instance model** for built-ins and extensions alike: + `{id, panel_type, sources (collection_id / layout_id / entity_ref), props, + state, layout}`. "Built-in" only selects a shipped renderer. +- **Command taxonomy** with three clean namespaces: + - `workspace.*` — layout, panels add/remove/move/focus, views + - `panel..*` — panel-owned state transitions (e.g. + `panel.samples.retrieval.set-anchor`) + - `collection.*` — create/update/delete filter/search/neighbor collections + Every command returns the same `CommandResult` shape: `{revision, + changed: [resource refs], result?}`. The frontend applies the returned + snapshot/diff instead of re-fetching. +- **Collections first-class.** Neighbor/filter/search results are materialized + into collections with stable ids; Samples/ImageGrid become collection + renderers, not owners of paging + filters + search modes. +- **Thin panel SDK.** The SDK exposes: command client (typed, discovered from + `/api/control/commands`), `usePanelState`, `useSelection`, `useCollection`, + `useSamples(collection_id)`, and host adapters for focus/resize. Nothing + else. Store and Dockview never leak. + +## Phases + +### Phase 1 — De-duplicate and de-legacy (mechanical, low risk) + +1. Export panel definitions from the runtime snapshot; delete the duplicated + contract data from `frontend/src/panels/definitions.tsx`, keeping only the + `panel_type → React component` renderer map. +2. Remove the `ui.similarity_query` mirror; migrate all readers to + `ui.panels.samples.state.retrieval` (grep frontend + Python + tests). +3. Normalize command names into the three namespaces above with aliases for + one release (`ui.panel.*` → `workspace.panel.*` etc.); emit deprecation + warnings for aliases. +4. Make every command handler return `CommandResult` with `revision` + + workspace snapshot, and make the frontend command client apply it (drop + the extra `/api/runtime` fetch). +5. Tests: update `test_control_commands.py`, `test_control_command_api.py`, + `test_cli_control.py`, `test_runtime_control_api.py`; all must pass; the + frontend must build (`npm run build`). + +### Phase 2 — One panel render path + +1. Fold the runtime-scatter special case into `RuntimeBuiltInPanel`. +2. Merge `RuntimeBuiltInPanel` and `RuntimeModulePanel` behind a single + `PanelHost` component: resolve renderer (built-in map or extension module + loader), wrap in `PanelInstanceProvider`, done. +3. Slim `panel-sdk/index.tsx` to the surface listed above; extension demos in + `examples/` and hyperview-spaces panels are the acceptance test. + +### Phase 3 — Collections as source of truth + +1. Materialize collection membership (result rows with ranks/scores) in the + backend, paged via `GET /api/collections/{id}/items`. **Done** (1fc55fe). +2. Samples panel renders whatever `collection_id` it is pointed at; retrieval + commands produce/update collections instead of panel-local result state. + **Done for the SDK surface** (51a712f): `useSamples(collectionId)` pages + through the endpoint for `all`/`filter`/`neighbors`/`search` kinds (live + and static bundles). The built-in Samples grid still renders through the + host view model; pointing it at a `collection_id` is follow-up work. +3. Split `space_key` into representation/index per architecture doc. + **First landing done** (contract level): `/api/dataset` exposes derived + `representations[]`/`indexes[]`, and `index_id` (`space:`) is + accepted at every retrieval boundary with `space_key` as the alias. + Storage/table rename deferred until representations get their own + lifecycle. + +### Phase 4 — Static export (`hyperview export`) + +Motivated by hyperview-spaces (see +`hyperview-spaces/docs/deployment-architecture.md`): a workspace snapshot that +serves demos with **no Python at request time**. + +1. `hyperview export --out bundle/` writes: + - the static frontend (already `output: "export"` in Next), + - the runtime snapshot JSON (panels, definitions, layouts, collections), + - materialized collection items + sample records as static JSON shards, + - media/thumbnails, + - precomputed layout coordinates. +2. A `fetch` adapter in the frontend API client resolves `/api/*` reads from + the static bundle when `window.__HYPERVIEW_STATIC__` is set. Bundles are + **read-only by product decision** (FiftyOne/Rerun examples-gallery + style): panel state changes stay client-side and ephemeral; mutating + commands are disabled behind a visible "read-only demo — run locally" + affordance. Extension panel JS modules ship inside the bundle and load + through the existing `RuntimeModulePanel` path. +3. Optional query path for text search over precomputed embeddings is a + deployment concern (see spaces doc) — not part of core. + +Phase 4 only needs Phases 1–2; it can start once the command client applies +snapshots (Phase 1.4). + +## Ground rules for implementation + +- Runtime/workspace state stays the source of truth; browser localStorage is + user preference only. +- No new REST routes for panel behavior — commands only. +- Do not change the LanceDB storage backend. +- Every phase lands with green `uv run pytest` and a green + `cd frontend && npm run build`, committed on + `refactor/panel-control-2026-07`. diff --git a/examples/multimodal_demo.py b/examples/multimodal_demo.py new file mode 100644 index 0000000..250899f --- /dev/null +++ b/examples/multimodal_demo.py @@ -0,0 +1,30 @@ +"""Demo: multimodal image+text retrieval with CLIP embeddings.""" + +from __future__ import annotations + +import hyperview as hv +from hyperview.core.sample import Sample + +DATASET_NAME = "coco_captions_demo" +WORKSPACE_NAME = "coco_captions_demo" + +dataset = hv.Dataset(DATASET_NAME) + +added, skipped = dataset.add_from_huggingface( + "HuggingFaceM4/COCO", + split="train", + max_samples=200, + image_key="image", + text_key="sentences", + label_key=None, +) +print(f"Ingested {added} samples ({skipped} skipped)") + +space_key = dataset.compute_embeddings(model="openai/clip-vit-base-patch32") +layout_key = dataset.compute_visualization(method="umap", layout_dimension=2) +print(f"Embeddings: {space_key}") +print(f"Layout: {layout_key}") + +session = hv.launch(dataset, workspace=WORKSPACE_NAME) +results = session.ui.query_by_text("a dog playing in the park", k=10) +print(f"Top text-query matches: {[sample.id for sample, _distance in results]}") diff --git a/frontend/src/app/layout.tsx b/frontend/src/app/layout.tsx index 6af29df..e6513e0 100644 --- a/frontend/src/app/layout.tsx +++ b/frontend/src/app/layout.tsx @@ -1,13 +1,6 @@ import type { Metadata } from "next"; -import { Inter } from "next/font/google"; import "./globals.css"; -const inter = Inter({ - subsets: ["latin"], - display: "swap", - weight: ["400", "500"], -}); - export const metadata: Metadata = { title: "HyperView", description: "Dataset visualization with hyperbolic embeddings", @@ -20,7 +13,7 @@ export default function RootLayout({ }>) { return ( - {children} + {children} ); } diff --git a/frontend/src/app/useHomeData.ts b/frontend/src/app/useHomeData.ts index 8ef4537..44d8857 100644 --- a/frontend/src/app/useHomeData.ts +++ b/frontend/src/app/useHomeData.ts @@ -79,11 +79,12 @@ function useSamplesDataFlow( const loadInitialData = useCallback(async (signal?: AbortSignal) => { setIsLoading(true); setError(null); + const initialLabelFilter = labelFilterRef.current ?? undefined; try { const [nextDatasetInfo, samplesRes] = await Promise.all([ fetchDataset(signal), - fetchSamples(0, SAMPLES_PER_PAGE, undefined, signal), + fetchSamples(0, SAMPLES_PER_PAGE, initialLabelFilter, signal), ]); if (signal?.aborted) return; diff --git a/frontend/src/app/useRuntimeSync.ts b/frontend/src/app/useRuntimeSync.ts index 79ae7fa..dfdafd4 100644 --- a/frontend/src/app/useRuntimeSync.ts +++ b/frontend/src/app/useRuntimeSync.ts @@ -2,7 +2,7 @@ import { useEffect, useRef, useState } from "react"; -import { fetchRuntimeState, getRuntimeEventsUrl } from "@/lib/api"; +import { fetchRuntimeState, getRuntimeEventsUrl, isStaticBundle } from "@/lib/api"; import { useStore } from "@/store/useStore"; import type { RuntimeSnapshot } from "@/types"; @@ -54,6 +54,12 @@ export function useRuntimeSync( void bootstrap(); + if (isStaticBundle()) { + return () => { + cancelled = true; + }; + } + const events = new EventSource(getRuntimeEventsUrl()); events.onmessage = (event) => { try { diff --git a/frontend/src/components/DockviewWorkspace.tsx b/frontend/src/components/DockviewWorkspace.tsx index 90cf8d1..3790b2b 100644 --- a/frontend/src/components/DockviewWorkspace.tsx +++ b/frontend/src/components/DockviewWorkspace.tsx @@ -33,6 +33,7 @@ import { addBuiltInCenterPanel, CENTER_PANEL_COMPONENTS, getBuiltInCenterPanelDefinition, + getBuiltInCenterPanelDefinitionForPanelType, getBuiltInCenterPanelIdForLayout, getScatterTabComponent, CENTER_PANEL_TAB_COMPONENTS, @@ -51,7 +52,7 @@ import { Button } from "./ui/button"; import { DockviewContext, useDockviewContext } from "./DockviewContext"; import { ExplorerPanel } from "./ExplorerPanel"; import { HyperViewLogo } from "./icons"; -import { RuntimeModulePanel } from "./RuntimeModulePanel"; +import { PanelHost } from "./PanelHost"; const LAYOUT_STORAGE_KEY = "hyperview:dockview-layout:v10"; const DEFAULT_CONTAINER_WIDTH = 1200; @@ -334,16 +335,39 @@ function isClosableDockPanel(panelId: string) { } function getExpectedRuntimePanelComponent(panel: RuntimePanel) { - if (panel.kind === "scatter") return "scatter"; - if (panel.kind === "builtin" && panel.builtin_panel === "samples") { - return getBuiltInCenterPanelDefinition(PANEL.GRID)?.component ?? null; - } - return "runtimeModulePanel"; + return "panelHost"; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function getSamplesRankFromPanelState(panel: RuntimePanel) { + const retrieval = panel.state?.retrieval; + if (!isRecord(retrieval)) return null; + const anchorSampleId = retrieval.anchor_sample_id; + const queryText = retrieval.query_text; + const hasAnchor = typeof anchorSampleId === "string" && anchorSampleId.length > 0; + const hasText = typeof queryText === "string" && queryText.length > 0; + if (!hasAnchor && !hasText) return null; + const layoutKey = retrieval.layout_key; + const spaceKey = retrieval.space_key; + const k = retrieval.k; + const source = retrieval.source; + return { + anchorSampleId: hasAnchor ? anchorSampleId : undefined, + queryText: hasText ? queryText : undefined, + layoutKey: typeof layoutKey === "string" ? layoutKey : undefined, + spaceKey: typeof spaceKey === "string" ? spaceKey : undefined, + k: typeof k === "number" ? k : undefined, + source: typeof source === "string" ? source : undefined, + }; } function getRuntimeSamplesPanelParams(panel: RuntimePanel) { - const mode = panel.props?.mode; - const rank = panel.props?.rank; + const stateMode = panel.state?.mode; + const mode = stateMode === "retrieval" ? "ranked" : panel.props?.mode; + const rank = getSamplesRankFromPanelState(panel) ?? panel.props?.rank; return { panelId: panel.id, runtimePlacementKey: getRuntimePanelPlacementKey(panel), @@ -355,6 +379,35 @@ function getRuntimeSamplesPanelParams(panel: RuntimePanel) { }; } +function getRuntimePanelHostParams(panel: RuntimePanel) { + const baseParams = { + ...(panel.props ?? {}), + panelId: panel.id, + builtinPanelType: panel.builtin_panel ?? panel.panel_type, + runtimePlacementKey: getRuntimePanelPlacementKey(panel), + }; + + if (panel.kind === "scatter") { + const layoutDimension = panel.layout_dimension === 3 ? 3 : 2; + return { + ...baseParams, + layoutKey: panel.layout_key ?? undefined, + geometry: (panel.geometry ?? undefined) as Geometry | undefined, + layoutDimension, + pinnedLayout: true, + }; + } + + if (panel.builtin_panel === "samples" || panel.panel_type === "samples") { + return { + ...baseParams, + ...getRuntimeSamplesPanelParams(panel), + }; + } + + return baseParams; +} + function DockviewPanelActions(props: IDockviewHeaderActionsProps) { const activeWorkspaceId = useStore((state) => state.activeWorkspaceId); const customPanels = useStore((state) => state.customPanels); @@ -538,7 +591,7 @@ const Watermark = React.memo(function Watermark(_props: IWatermarkPanelProps) { const COMPONENTS = { ...CENTER_PANEL_COMPONENTS, explorer: ExplorerDockPanel, - runtimeModulePanel: RuntimeModulePanel, + panelHost: PanelHost, }; const TAB_COMPONENTS = CENTER_PANEL_TAB_COMPONENTS; @@ -798,92 +851,49 @@ export function DockviewWorkspace() { const existingPlacementKey = (existingPanel.api.getParameters() as { runtimePlacementKey?: string; }).runtimePlacementKey; - if (!expectedComponent || currentComponent !== expectedComponent || existingPlacementKey !== placementKey) { + const placementChanged = + existingPlacementKey !== undefined && existingPlacementKey !== placementKey; + if (!expectedComponent || currentComponent !== expectedComponent || placementChanged) { runtimeSyncClosedPanels.current.add(panel.id); existingPanel.api.close(); existingPanel = undefined; } else { existingPanel.api.setTitle(panel.title); - if (panel.kind === "scatter") { - const layoutDimension = panel.layout_dimension === 3 ? 3 : 2; - existingPanel.api.updateParameters({ - layoutKey: panel.layout_key ?? undefined, - geometry: (panel.geometry ?? undefined) as Geometry | undefined, - layoutDimension, - pinnedLayout: true, - runtimePlacementKey: placementKey, - }); - } else if (panel.kind === "builtin" && panel.builtin_panel === "samples") { - existingPanel.api.updateParameters(getRuntimeSamplesPanelParams(panel)); - } else { - existingPanel.api.updateParameters({ - panelId: panel.id, - runtimePlacementKey: placementKey, - }); - } + existingPanel.api.updateParameters(getRuntimePanelHostParams(panel)); continue; } } - if (panel.kind === "scatter") { - const layoutDimension = panel.layout_dimension === 3 ? 3 : 2; - const layout = getRuntimePanelAddLayout(panel); - api.addPanel({ - id: runtimePanelId, - component: "scatter", - title: panel.title, - tabComponent: getScatterTabComponent({ - geometry: panel.geometry, - layoutDimension, - }), - params: { - layoutKey: panel.layout_key ?? undefined, - geometry: (panel.geometry ?? undefined) as Geometry | undefined, - layoutDimension, - pinnedLayout: true, - runtimePlacementKey: getRuntimePanelPlacementKey(panel), - }, - position: getRuntimePanelPosition(api, panel.position, panel), - ...layout, - }); - continue; - } - - if (panel.kind === "builtin" && panel.builtin_panel === "samples") { - const samplesParams = getRuntimeSamplesPanelParams(panel); - const samplesDefinition = getBuiltInCenterPanelDefinition(PANEL.GRID); - if (!samplesDefinition) continue; - const layout = getRuntimePanelAddLayout(panel); - - const options = samplesDefinition.buildAddPanelOptions({ - api, - datasetInfo, - position: getRuntimePanelPosition(api, panel.position, panel), - }); - - api.addPanel({ - ...options, - id: runtimePanelId, - title: panel.title, - params: { - ...(options.params ?? {}), - ...samplesParams, - }, - ...layout, - }); - continue; - } - + const builtInPanelType = panel.builtin_panel ?? panel.panel_type; + const definition = + panel.kind === "module" + ? null + : getBuiltInCenterPanelDefinitionForPanelType(builtInPanelType); + const builtInOptions = definition?.buildAddPanelOptions({ + api, + datasetInfo, + position: getRuntimePanelPosition(api, panel.position, panel), + }); const layout = getRuntimePanelAddLayout(panel); + const layoutDimension = panel.layout_dimension === 3 ? 3 : 2; api.addPanel({ id: runtimePanelId, - component: "runtimeModulePanel", + component: "panelHost", title: panel.title, + tabComponent: + panel.kind === "scatter" + ? getScatterTabComponent({ + geometry: panel.geometry, + layoutDimension, + }) + : builtInOptions?.tabComponent, params: { - panelId: panel.id, - runtimePlacementKey: getRuntimePanelPlacementKey(panel), + ...(builtInOptions?.params ?? {}), + ...getRuntimePanelHostParams(panel), }, - position: getRuntimePanelPosition(api, panel.position, panel), + position: + builtInOptions?.position ?? + getRuntimePanelPosition(api, panel.position, panel), initialWidth: layout.initialWidth ?? (panel.position === "right" ? getDefaultRightPanelWidth(getContainerWidth(api)) : undefined), diff --git a/frontend/src/components/ExplorerPanel.tsx b/frontend/src/components/ExplorerPanel.tsx index b18c36a..d134314 100644 --- a/frontend/src/components/ExplorerPanel.tsx +++ b/frontend/src/components/ExplorerPanel.tsx @@ -8,6 +8,7 @@ import { Search, Tag } from "lucide-react"; import { cn } from "@/lib/utils"; import { FALLBACK_LABEL_COLOR, MISSING_LABEL_COLOR, normalizeLabel } from "@/lib/labelColors"; import { useLabelLegend } from "./useLabelLegend"; +import { setLabelFilterCollection } from "@/lib/api"; interface ExplorerPanelProps { className?: string; @@ -20,6 +21,8 @@ export function ExplorerPanel({ className }: ExplorerPanelProps) { activeLayoutKey, labelFilter, setLabelFilter, + activeWorkspaceId, + applyRuntimeSnapshot, } = useStore(); const [labelSearch, setLabelSearch] = React.useState(""); const [isSearchOpen, setIsSearchOpen] = React.useState(false); @@ -55,6 +58,24 @@ export function ExplorerPanel({ className }: ExplorerPanelProps) { } }; + const applyLabelFilter = React.useCallback( + (nextLabel: string | null) => { + setLabelFilter(nextLabel); + if (!activeWorkspaceId) return; + + void setLabelFilterCollection({ + workspaceId: activeWorkspaceId, + value: nextLabel, + clear: nextLabel === null, + }) + .then(applyRuntimeSnapshot) + .catch((err) => { + console.error("Failed to persist label filter:", err); + }); + }, + [activeWorkspaceId, applyRuntimeSnapshot, setLabelFilter] + ); + return ( }> @@ -105,7 +126,7 @@ export function ExplorerPanel({ className }: ExplorerPanelProps) { + + + ); +} + export const SamplesImageGridPanel = React.memo(function SamplesImageGridPanel( props: IDockviewPanelProps ) { - const { collection, derivedSpace } = usePanelSamplesView(); - const { clearLassoSelection } = usePanelCommands(); - const { sampleGridSize, setSampleGridSize } = usePanelUiState(); + const { collection, derivedSpace } = useHyperViewSamplesView(); + const { clearLassoSelection } = useSamplesPanelCommands(); + const sampleGridSize = useStore((state) => state.sampleGridSize); + const setSampleGridSize = useStore((state) => state.setSampleGridSize); const panelMode = props.params?.mode ?? "auto"; const showRankedPanel = panelMode === "ranked"; const rankedPanel = useRankedSamplesPanel(props.params?.rank, showRankedPanel); @@ -224,7 +353,15 @@ export const SamplesImageGridPanel = React.memo(function SamplesImageGridPanel( label: "Results", value: rankedPanel.rankedSamples.length.toLocaleString(), }, - ...(rankedPanel.anchorSampleId + ...(rankedPanel.queryText + ? [ + { + id: "query", + label: "Query", + value: rankedPanel.queryText, + } satisfies PanelToolbarItem, + ] + : rankedPanel.anchorSampleId ? [ { id: "anchor", @@ -263,6 +400,7 @@ export const SamplesImageGridPanel = React.memo(function SamplesImageGridPanel( collection.meta.source, collection.total, rankedPanel.anchorSampleId, + rankedPanel.queryText, rankedPanel.rankedSamples.length, showRankedPanel, ] @@ -308,26 +446,41 @@ export const SamplesImageGridPanel = React.memo(function SamplesImageGridPanel( {showRankedPanel ? ( - !rankedPanel.anchorSampleId ? ( + !rankedPanel.anchorSampleId && !rankedPanel.queryText ? ( - ) : rankedPanel.anchorSample === null && rankedPanel.loading ? ( + ) : rankedPanel.queryText && rankedPanel.loading && rankedPanel.rankedSamples.length === 0 ? ( + + ) : !rankedPanel.queryText && rankedPanel.anchorSample === null && rankedPanel.loading ? ( - ) : rankedPanel.anchorSample === null ? ( + ) : !rankedPanel.queryText && rankedPanel.anchorSample === null ? ( + ) : rankedPanel.queryText ? ( + ) : ( ) : null} + {!showRankedPanel ? : null} + {showRankedPanel ? null : collection.error ? ( = Record, > { id: string; + panelType: string; component: string; title: string; label: string; diff --git a/frontend/src/panels/registry.tsx b/frontend/src/panels/registry.tsx index 166ea7e..9003f10 100644 --- a/frontend/src/panels/registry.tsx +++ b/frontend/src/panels/registry.tsx @@ -9,7 +9,11 @@ import type { import { Circle, Disc, Globe2 } from "lucide-react"; import { ScatterPanel } from "@/components/ScatterPanel"; -import { defineBuiltInCenterPanel, type BuiltInCenterPanelDefinition, type DockviewPanelPosition } from "@/panels/definitions"; +import { + defineBuiltInCenterPanel, + type BuiltInCenterPanelDefinition, + type DockviewPanelPosition, +} from "@/panels/definitions"; import { samplesImageGridBuiltInPanel } from "@/panels/builtins/samplesImageGridPanel"; import { findLayoutByGeometry, findLayoutByKey, getLayoutDimension } from "@/lib/layouts"; import type { DatasetInfo, Geometry } from "@/types"; @@ -77,6 +81,7 @@ function createScatterPanelDefinition(args: { return defineBuiltInCenterPanel({ id, + panelType: "scatter", component: "scatter", title, label, @@ -158,6 +163,7 @@ const scatterSpherical3DBuiltInPanel = createScatterPanelDefinition({ const fallbackScatterBuiltInPanel = defineBuiltInCenterPanel({ id: PANEL.SCATTER_DEFAULT, + panelType: "scatter", component: "scatter", title: "Embeddings", label: "Embeddings", @@ -207,14 +213,20 @@ const builtInCenterPanelById = new Map( BUILT_IN_CENTER_PANELS.map((panel) => [panel.id, panel]) ); +const builtInCenterPanelByPanelType = new Map( + BUILT_IN_CENTER_PANELS.map((panel) => [panel.panelType, panel]) +); + export const CENTER_PANEL_DEFS = BUILT_IN_CENTER_PANELS.filter( (panel) => panel.visibleInViewMenu !== false ).map((panel) => ({ id: panel.id, + panelType: panel.panelType, label: panel.label, icon: panel.icon, })) as ReadonlyArray<{ id: string; + panelType: string; label: string; icon: BuiltInCenterPanelDefinition["icon"]; }>; @@ -232,6 +244,11 @@ export function getBuiltInCenterPanelDefinition(panelId: string) { return builtInCenterPanelById.get(panelId) ?? null; } +export function getBuiltInCenterPanelDefinitionForPanelType(panelType: string | null | undefined) { + if (!panelType) return null; + return builtInCenterPanelByPanelType.get(panelType) ?? null; +} + export function getBuiltInCenterPanelIdForLayout(args: { datasetInfo: DatasetInfo | null; layoutKey: string | null; diff --git a/frontend/src/store/useStore.ts b/frontend/src/store/useStore.ts index b654233..dc7f149 100644 --- a/frontend/src/store/useStore.ts +++ b/frontend/src/store/useStore.ts @@ -2,7 +2,10 @@ import { create } from "zustand"; import type { DatasetInfo, EmbeddingsData, + RuntimeCollection, RuntimePanel, + RuntimePanelDefinition, + RuntimePanelStateEntry, RuntimeSnapshot, Sample, SimilarityQuery, @@ -37,6 +40,8 @@ function createClearedDatasetScopedState() { totalSamples: 0, samplesLoaded: 0, embeddingsByLayoutKey: {} as Record, + runtimeCollections: [] as RuntimeCollection[], + panelStates: {} as Record, activeLayoutKey: null as string | null, activeSimilarityQuery: null as SimilarityQuery | null, selectionLayoutKey: null as string | null, @@ -55,6 +60,41 @@ function areSetsEqual(left: Set, right: Set) { return true; } +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function coerceSimilarityQuery(value: unknown): SimilarityQuery | null { + if (!isRecord(value)) return null; + const anchorSampleId = + typeof value.anchor_sample_id === "string" && value.anchor_sample_id.length > 0 + ? value.anchor_sample_id + : null; + const queryText = + typeof value.query_text === "string" && value.query_text.length > 0 + ? value.query_text + : null; + if (!anchorSampleId && !queryText) return null; + const k = typeof value.k === "number" ? value.k : Number(value.k ?? 18); + return { + anchor_sample_id: anchorSampleId, + query_text: queryText, + layout_key: typeof value.layout_key === "string" ? value.layout_key : null, + space_key: typeof value.space_key === "string" ? value.space_key : null, + k: Number.isFinite(k) ? Math.max(1, Math.min(k, 100)) : 18, + source: typeof value.source === "string" ? value.source : null, + }; +} + +function labelFilterFromSamplesPanelState(state: Record): string | null { + if (state.mode !== "collection" || !isRecord(state.collection)) return null; + const collection = state.collection; + if (collection.kind !== "filter" || !isRecord(collection.query)) return null; + const { field, op, value } = collection.query; + if (field !== "label" || op !== "eq") return null; + return normalizeLabel(typeof value === "string" ? value : null); +} + type SelectionSource = "scatter" | "grid" | "panel"; export interface OrbitView3DPayload { @@ -86,6 +126,9 @@ interface AppState { workspaces: WorkspaceSummary[]; runtimeDatasetName: string | null; customPanels: RuntimePanel[]; + panelDefinitions: RuntimePanelDefinition[]; + runtimeCollections: RuntimeCollection[]; + panelStates: Record; hasExplicitView: boolean; activePanelId: string | null; viewRevision: number; @@ -111,7 +154,6 @@ interface AppState { activeLayoutKey: string | null; setActiveLayoutKey: (layoutKey: string | null) => void; activeSimilarityQuery: SimilarityQuery | null; - setActiveSimilarityQuery: (query: SimilarityQuery | null) => void; // Label filter (sidebar-driven) labelFilter: string | null; @@ -179,6 +221,9 @@ export const useStore = create((set) => ({ workspaces: [], runtimeDatasetName: null, customPanels: [], + panelDefinitions: [], + runtimeCollections: [], + panelStates: {}, hasExplicitView: false, activePanelId: null, viewRevision: 0, @@ -202,10 +247,12 @@ export const useStore = create((set) => ({ state.activeWorkspaceId !== nextWorkspaceId || state.runtimeDatasetName !== nextDatasetName; - const activeSimilarityQuery = snapshot.workspace.ui.similarity_query ?? null; + const samplesPanelState = snapshot.workspace.ui.panels?.samples?.state ?? {}; + const activeSimilarityQuery = coerceSimilarityQuery(samplesPanelState.retrieval); const selectedIds = activeSimilarityQuery ? [] : snapshot.workspace.ui.selected_ids; + const labelFilter = labelFilterFromSamplesPanelState(samplesPanelState); return { ...(runtimeScopeChanged ? createClearedDatasetScopedState() : {}), @@ -213,6 +260,9 @@ export const useStore = create((set) => ({ workspaces: snapshot.workspaces, runtimeDatasetName: nextDatasetName, customPanels: snapshot.workspace.ui.custom_panels, + panelDefinitions: snapshot.panel_definitions ?? [], + runtimeCollections: snapshot.workspace.collections ?? [], + panelStates: snapshot.workspace.ui.panels ?? {}, hasExplicitView: snapshot.workspace.ui.has_explicit_view, activePanelId: snapshot.workspace.ui.active_panel_id, viewRevision: snapshot.workspace.ui.view_revision ?? 0, @@ -222,6 +272,7 @@ export const useStore = create((set) => ({ selectionSource: selectedIds.length > 0 ? "scatter" : null, selectionLayoutKey: null, activeSimilarityQuery, + labelFilter, ...createClearedLassoState(), ...createClearedNeighborsState(), }; @@ -262,7 +313,6 @@ export const useStore = create((set) => ({ activeLayoutKey: null, setActiveLayoutKey: (layoutKey) => set({ activeLayoutKey: layoutKey }), activeSimilarityQuery: null, - setActiveSimilarityQuery: (query) => set({ activeSimilarityQuery: query }), // Label filter labelFilter: null, @@ -309,7 +359,8 @@ export const useStore = create((set) => ({ selectionSource: nextSelectionSource, selectionLayoutKey: nextSelectionLayoutKey, activeSimilarityQuery: - state.activeSimilarityQuery && ids.has(state.activeSimilarityQuery.anchor_sample_id) + state.activeSimilarityQuery?.anchor_sample_id && + ids.has(state.activeSimilarityQuery.anchor_sample_id) ? state.activeSimilarityQuery : null, ...createClearedLassoState(), @@ -330,7 +381,8 @@ export const useStore = create((set) => ({ selectionSource: newSet.size > 0 ? "grid" : null, selectionLayoutKey: null, activeSimilarityQuery: - state.activeSimilarityQuery && newSet.has(state.activeSimilarityQuery.anchor_sample_id) + state.activeSimilarityQuery?.anchor_sample_id && + newSet.has(state.activeSimilarityQuery.anchor_sample_id) ? state.activeSimilarityQuery : null, ...createClearedLassoState(), @@ -347,7 +399,8 @@ export const useStore = create((set) => ({ selectionSource: newSet.size > 0 ? "grid" : null, selectionLayoutKey: null, activeSimilarityQuery: - state.activeSimilarityQuery && newSet.has(state.activeSimilarityQuery.anchor_sample_id) + state.activeSimilarityQuery?.anchor_sample_id && + newSet.has(state.activeSimilarityQuery.anchor_sample_id) ? state.activeSimilarityQuery : null, ...createClearedLassoState(), diff --git a/frontend/src/types/index.ts b/frontend/src/types/index.ts index 553063e..bca98bc 100644 --- a/frontend/src/types/index.ts +++ b/frontend/src/types/index.ts @@ -3,6 +3,8 @@ export interface Sample { filepath: string; filename: string; label: string | null; + text?: string | null; + modality?: string | null; thumbnail: string | null; media_url?: string | null; thumbnail_url?: string | null; @@ -32,11 +34,33 @@ export interface LayoutInfo { params: Record | null; } +export interface RepresentationInfo { + id: string; + entity_set_id: string; + field_path: string; + kind: string; + shape: number[]; + model_id: string; + provider: string; + modality: string; + geometry: Geometry | string; + count: number; +} + +export interface IndexInfo { + id: string; + representation_id: string; + query_modes: string[]; + scorer: string; +} + export interface DatasetInfo { name: string; num_samples: number; labels: string[]; spaces: SpaceInfo[]; + representations?: RepresentationInfo[]; + indexes?: IndexInfo[]; layouts: LayoutInfo[]; } @@ -60,7 +84,8 @@ export interface SimilarSample extends Sample { } export interface SimilaritySearchResponse { - query_id: string; + query_id?: string | null; + query_text?: string | null; query_sample: Sample | null; space_key: string | null; metric: string; @@ -69,26 +94,85 @@ export interface SimilaritySearchResponse { } export interface SimilarityQuery { - anchor_sample_id: string; + anchor_sample_id?: string | null; + query_text?: string | null; layout_key: string | null; space_key: string | null; k: number; source: string | null; } +export type CollectionKind = + | "all" + | "filter" + | "selection" + | "neighbors" + | "lasso" + | "search" + | "tool_result" + | "extension"; + +export interface RuntimeCollection { + id: string; + dataset_id: string; + entity_set_id: string; + kind: CollectionKind; + query: Record; + scores: Record | null; + created_at: number; +} + +export interface RuntimePanelStateEntry { + state: Record; + state_revision: number; +} + export interface RuntimePanelData { module_src: string | null; } +export interface RuntimePanelDefinition { + panel_type: string; + label: string; + title: string; + source: string; + extension: string | null; + default_props: Record; + default_state: Record; + props_schema: Record | null; + state_schema: Record | null; + commands: string[]; + queries: string[]; + lifecycle: Record; + default_layout: Record; + allow_multiple: boolean; + icon: string | null; + category: string | null; +} + export type RuntimePanelKind = "module" | "scatter" | "builtin"; export type RuntimePanelPosition = "center" | "right" | "bottom"; export type RuntimePanelDirection = "right" | "left" | "above" | "below" | "within"; +export interface RuntimePanelLayout { + position: RuntimePanelPosition; + reference_panel_id: string | null; + direction: RuntimePanelDirection | null; + width: number | null; + height: number | null; + min_width: number | null; + min_height: number | null; + max_width: number | null; + max_height: number | null; +} + export interface RuntimePanel { id: string; kind: RuntimePanelKind; + panel_type: string; + source: string; title: string; position: RuntimePanelPosition; builtin_panel: "samples" | string | null; @@ -108,6 +192,9 @@ export interface RuntimePanel { max_height: number | null; visible: boolean; props: Record; + state: Record; + state_revision: number; + layout: RuntimePanelLayout; data: RuntimePanelData; } @@ -119,10 +206,11 @@ export interface WorkspaceSummary { export interface RuntimeWorkspaceState { id: string; dataset_name: string | null; + collections: RuntimeCollection[]; ui: { active_layout_key: string | null; selected_ids: string[]; - similarity_query: SimilarityQuery | null; + panels: Record; layout_views: Record< string, { @@ -148,6 +236,7 @@ export interface RuntimeSnapshot { runtime_id: string; version: number; active_workspace_id: string | null; + panel_definitions: RuntimePanelDefinition[]; workspaces: WorkspaceSummary[]; workspace: RuntimeWorkspaceState; } diff --git a/src/hyperview/__init__.py b/src/hyperview/__init__.py index 6b00ff6..780338f 100644 --- a/src/hyperview/__init__.py +++ b/src/hyperview/__init__.py @@ -1,12 +1,17 @@ """HyperView - Open-source dataset curation with hyperbolic embeddings visualization.""" from . import _version as _version +from ._compat import disable_blocked_datasets_torch_shared_memory + +disable_blocked_datasets_torch_shared_memory() + from . import api as _api from . import ui as ui Dataset = _api.Dataset Session = _api.Session launch = _api.launch +export_workspace = _api.export_workspace register_provider = _api.register_provider unregister_provider = _api.unregister_provider __version__ = _version.__version__ @@ -15,6 +20,7 @@ "Dataset", "Session", "launch", + "export_workspace", "register_provider", "unregister_provider", "ui", diff --git a/src/hyperview/_compat.py b/src/hyperview/_compat.py new file mode 100644 index 0000000..eb6c220 --- /dev/null +++ b/src/hyperview/_compat.py @@ -0,0 +1,27 @@ +"""Compatibility helpers for optional third-party runtime behavior.""" + +from __future__ import annotations + +import sys + + +def disable_blocked_datasets_torch_shared_memory() -> None: + """Keep HF streaming datasets usable when torch shared memory is sandboxed.""" + + if "datasets" not in sys.modules: + return + try: + from datasets import config as datasets_config + except Exception: + return + if not getattr(datasets_config, "TORCH_AVAILABLE", False): + return + try: + import torch + + torch.tensor(0).share_memory_() + except RuntimeError as exc: + if "torch_shm_manager" in str(exc) or "share" in str(exc).lower(): + datasets_config.TORCH_AVAILABLE = False + except Exception: + return diff --git a/src/hyperview/api.py b/src/hyperview/api.py index 5409b89..e6e2dea 100644 --- a/src/hyperview/api.py +++ b/src/hyperview/api.py @@ -18,11 +18,20 @@ import uvicorn import hyperview.ui as ui_module +from hyperview.control import CommandEnvelope, ControlService, create_default_command_registry from hyperview.core.dataset import Dataset from hyperview.runtime import HyperViewRuntime, ProviderRegistry from hyperview.server.app import create_app, set_runtime +from hyperview.static_export import export_runtime_workspace, export_workspace -__all__ = ["Dataset", "launch", "Session", "register_provider", "unregister_provider"] +__all__ = [ + "Dataset", + "launch", + "Session", + "export_workspace", + "register_provider", + "unregister_provider", +] @dataclass(frozen=True) @@ -127,6 +136,9 @@ def __init__( self._server: uvicorn.Server | None = None self._startup_error: BaseException | None = None self.session_id = uuid4().hex + self._control_registry = create_default_command_registry() + self._control_service: ControlService | None = None + self.control = SessionControlController(self) self.ui = SessionUiController(self) @property @@ -294,6 +306,61 @@ def open_browser(self): """Open the visualizer in a browser window.""" webbrowser.open(self.url) + def export( + self, + out: str | os.PathLike[str], + *, + workspace_id: str = "default", + ) -> dict[str, Any]: + """Export a read-only static bundle for a workspace in this session.""" + + return export_runtime_workspace(self.runtime, workspace_id, out).to_dict() + + +class SessionControlController: + """Generic command runner for a HyperView session.""" + + def __init__(self, session: Session): + self._session = session + + def _service(self) -> ControlService: + if not self._session._controls_runtime: + raise RuntimeError( + "This session is attached to an existing HyperView server. " + "session.control can only mutate sessions started by this process; " + "use the control-plane CLI/API or launch with reuse_server=False." + ) + if self._session._control_service is None: + self._session._control_service = ControlService( + self._session.runtime, + self._session._control_registry, + ) + return self._session._control_service + + def run( + self, + command: str, + *, + target: dict[str, Any] | None = None, + args: dict[str, Any] | None = None, + ) -> dict[str, Any]: + """Run a backend-owned control command and return the result envelope.""" + + result = self._service().run( + CommandEnvelope( + command=command, + target=target or {}, + args=args or {}, + ) + ) + payload = result.to_dict() + if not result.ok: + error = result.error + if error is None: + raise RuntimeError("Command failed") + raise RuntimeError(f"{error.code}: {error.message}") + return payload + class SessionUiController: """Public UI control surface for a HyperView session.""" @@ -332,21 +399,23 @@ def add_scatter( ) -> None: """Add a scatter panel pinned to an explicit layout.""" - runtime = self._runtime() - runtime.add_runtime_panel( - workspace_id, - panel_id=panel_id, - title=title, - kind="scatter", - layout_key=layout_key, - position=position, - reference_panel_id=reference_panel_id, - direction=direction, - geometry=geometry, - layout_dimension=layout_dimension, - props=dict(props or {}), - **(layout.to_runtime_kwargs() if layout is not None else {}), - require_resolved_layout=False, + self._session.control.run( + "workspace.panel.add", + target={"workspace_id": workspace_id}, + args={ + "panel_id": panel_id, + "title": title, + "kind": "scatter", + "layout_key": layout_key, + "position": position, + "reference_panel_id": reference_panel_id, + "direction": direction, + "geometry": geometry, + "layout_dimension": layout_dimension, + "props": dict(props or {}), + **(layout.to_runtime_kwargs() if layout is not None else {}), + "require_resolved_layout": False, + }, ) def update_panel( @@ -381,15 +450,22 @@ def update_panel( placement_kwargs["reference_panel_id"] = reference_panel_id if position is not None or direction is not None: placement_kwargs["direction"] = direction - self._runtime().update_custom_panel( - workspace_id, - panel_id, - title=title, - position=position, - active=active, - props=dict(props) if props is not None else None, + update_args: dict[str, object | None] = { **placement_kwargs, **layout_kwargs, + } + if title is not None: + update_args["title"] = title + if position is not None: + update_args["position"] = position + if active is not None: + update_args["active"] = active + if props is not None: + update_args["props"] = dict(props) + self._session.control.run( + "workspace.panel.update", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + args=update_args, ) def resize_panel( @@ -418,10 +494,10 @@ def resize_panel( }.items() if value is not None } - self._runtime().update_custom_panel( - workspace_id, - panel_id, - **layout_kwargs, + self._session.control.run( + "workspace.panel.resize", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + args=layout_kwargs, ) def move_panel( @@ -435,31 +511,81 @@ def move_panel( ) -> None: """Move a panel in the durable workspace view.""" - self._runtime().update_custom_panel( - workspace_id, - panel_id, - position=position, - reference_panel_id=reference_panel_id, - direction=direction, + self._session.control.run( + "workspace.panel.move", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + args={ + "position": position, + "reference_panel_id": reference_panel_id, + "direction": direction, + }, ) def focus_panel(self, panel_id: str, *, workspace_id: str = "default") -> None: """Set the active panel for the workspace view.""" - self._runtime().update_custom_panel(workspace_id, panel_id, active=True) + self._session.control.run( + "workspace.panel.focus", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + ) def close_panel(self, panel_id: str, *, workspace_id: str = "default") -> None: """Hide a panel without deleting it from the workspace view.""" - self._runtime().update_custom_panel(workspace_id, panel_id, visible=False) + self._session.control.run( + "workspace.panel.close", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + ) def show_panel(self, panel_id: str, *, workspace_id: str = "default") -> None: """Show a panel that was hidden in the workspace view.""" - self._runtime().update_custom_panel(workspace_id, panel_id, visible=True) + self._session.control.run( + "workspace.panel.show", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + ) + + def get_panel_state( + self, + panel_id: str, + *, + workspace_id: str = "default", + ) -> dict[str, object]: + """Return durable runtime-managed state for a panel.""" + + payload = self._session.control.run( + "workspace.panel.state.get", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + ) + return dict(payload.get("result") or {}) + + def patch_panel_state( + self, + panel_id: str, + state: dict[str, object], + *, + workspace_id: str = "default", + replace_state: bool = False, + expected_revision: int | None = None, + ) -> dict[str, object]: + """Patch durable runtime-managed panel state.""" + + payload = self._session.control.run( + "workspace.panel.state.patch", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + args={ + "state": dict(state), + "replace_state": replace_state, + "expected_revision": expected_revision, + }, + ) + return dict(payload.get("result") or {}) def remove_panel(self, panel_id: str, *, workspace_id: str = "default") -> None: - self._runtime().remove_custom_panel(workspace_id, panel_id) + self._session.control.run( + "workspace.panel.remove", + target={"workspace_id": workspace_id, "panel_id": panel_id}, + ) def set_active_layout( self, @@ -491,23 +617,93 @@ def show_similar( k: int = 18, source: str = "python", ) -> None: - """Set the workspace's active similarity query.""" + """Show nearest-neighbor results in the Samples panel.""" - runtime = self._runtime() - query = runtime.resolve_similarity_query( - workspace_id, + self.set_samples_retrieval( sample_id, + workspace_id=workspace_id, layout_key=layout_key, space_key=space_key, k=k, source=source, ) - runtime.set_similarity_query(workspace_id, query) - def clear_similarity(self, *, workspace_id: str = "default") -> None: - """Clear the workspace's active similarity query.""" + def set_samples_retrieval( + self, + sample_id: str, + *, + workspace_id: str = "default", + layout_key: str | None = None, + space_key: str | None = None, + k: int = 18, + source: str = "python", + ) -> None: + """Set Samples panel retrieval state.""" + + self._session.control.run( + "panel.samples.retrieval.set-anchor", + target={"workspace_id": workspace_id}, + args={ + "sample_id": sample_id, + "layout_key": layout_key, + "space_key": space_key, + "k": k, + "source": source, + }, + ) + + def set_samples_retrieval_k( + self, + k: int, + *, + workspace_id: str = "default", + ) -> None: + """Update the active Samples retrieval result count.""" + + self._session.control.run( + "panel.samples.retrieval.set-k", + target={"workspace_id": workspace_id}, + args={"k": k}, + ) + + def clear_samples_retrieval(self, *, workspace_id: str = "default") -> None: + """Clear Samples panel retrieval state.""" + + self._session.control.run( + "panel.samples.retrieval.clear", + target={"workspace_id": workspace_id}, + ) - self._runtime().set_similarity_query(workspace_id, None) + def query_by_text( + self, + query_text: str, + *, + workspace_id: str = "default", + layout_key: str | None = None, + space_key: str | None = None, + k: int = 18, + source: str = "python", + ) -> list[tuple[Any, float]]: + """Run a text query against the workspace dataset and show results in the Samples panel.""" + + self._session.control.run( + "panel.samples.retrieval.set-text-query", + target={"workspace_id": workspace_id}, + args={ + "query_text": query_text, + "layout_key": layout_key, + "space_key": space_key, + "k": k, + "source": source, + }, + ) + dataset = self._runtime().get_dataset(workspace_id=workspace_id) + return dataset.find_similar_by_text( + query_text, + k=k, + space_key=space_key, + layout_key=layout_key, + ) def add_extension( self, @@ -529,6 +725,14 @@ def add_extension( add_panels=add_panels, ) + def list_panel_definitions(self) -> list[dict[str, Any]]: + """Return built-in and installed extension panel definitions.""" + + return [ + definition.to_dict() + for definition in self._runtime().list_panel_definitions() + ] + def launch( dataset: Dataset, diff --git a/src/hyperview/cli.py b/src/hyperview/cli.py index b4a7903..5f6bed5 100644 --- a/src/hyperview/cli.py +++ b/src/hyperview/cli.py @@ -16,6 +16,7 @@ from hyperview.core.selection import OrbitViewState3D from hyperview.figures import FigureRenderOptions, render_layout_figure from hyperview.runtime import HyperViewRuntime, ProviderRegistry, WorkspaceRegistry +from hyperview.static_export import export_workspace from hyperview.storage.schema import parse_layout_dimension @@ -68,6 +69,18 @@ def _parse_provider_args(values: list[str] | None) -> dict[str, Any]: return parsed +def _parse_key_value_pairs(values: list[str] | None, *, label: str) -> dict[str, Any]: + parsed: dict[str, Any] = {} + for value in values or []: + if "=" not in value: + raise ValueError(f"{label} values must use the form key=value, got '{value}'") + key, raw_value = value.split("=", 1) + if not key: + raise ValueError(f"{label} values must include a non-empty key") + parsed[key] = _parse_scalar(raw_value) + return parsed + + def _parse_json_object(value: str | None, *, label: str) -> dict[str, Any] | None: if value is None: return None @@ -138,6 +151,37 @@ def _panel_layout_payload(args: argparse.Namespace) -> dict[str, int]: return payload +def _run_control_command( + base_url: str, + command: str, + *, + target: dict[str, Any], + args: dict[str, Any] | None = None, + raise_on_error: bool = True, +) -> dict[str, Any]: + payload = _http_send_json( + f"{base_url}/api/control/commands/run", + { + "command": command, + "target": target, + "args": args or {}, + }, + ) + if raise_on_error and payload.get("ok") is False: + error = payload.get("error") or {} + code = error.get("code", "unknown_error") + message = error.get("message", "Command failed") + raise RuntimeError(f"{code}: {message}") + return cast(dict[str, Any], payload) + + +def _workspace_payload_from_command(payload: dict[str, Any]) -> dict[str, Any]: + workspace = payload.get("workspace") + if not isinstance(workspace, dict): + raise RuntimeError("Command did not return a workspace") + return {"workspace": workspace} + + def _add_dataset_source_flags(parser: argparse.ArgumentParser) -> None: parser.add_argument("--hf-dataset") parser.add_argument("--split", default=None) @@ -145,6 +189,7 @@ def _add_dataset_source_flags(parser: argparse.ArgumentParser) -> None: parser.add_argument("--image-key", default=None) parser.add_argument("--label-key", default=None) parser.add_argument("--label-names-key", default=None) + parser.add_argument("--text-key", default=None) parser.add_argument("--images-dir") parser.add_argument("--label-from-folder", action="store_true") parser.add_argument("--samples", type=int, default=None) @@ -209,6 +254,11 @@ def _build_control_parser() -> argparse.ArgumentParser: _add_server_flags(status_parser) _add_json_flag(status_parser) + export_parser = subparsers.add_parser("export") + export_parser.add_argument("workspace_id") + export_parser.add_argument("--out", required=True) + _add_json_flag(export_parser) + dataset_parser = subparsers.add_parser("dataset") dataset_subparsers = dataset_parser.add_subparsers(dest="dataset_command", required=True) dataset_create = dataset_subparsers.add_parser("create") @@ -307,6 +357,20 @@ def _build_control_parser() -> argparse.ArgumentParser: jobs_inspect.add_argument("job_id") _add_json_flag(jobs_inspect) + commands_parser = subparsers.add_parser("commands") + commands_subparsers = commands_parser.add_subparsers( + dest="commands_command", required=True + ) + commands_list = commands_subparsers.add_parser("list") + _add_server_flags(commands_list) + _add_json_flag(commands_list) + commands_run = commands_subparsers.add_parser("run") + _add_server_flags(commands_run) + commands_run.add_argument("command_id") + commands_run.add_argument("--target", action="append", default=[]) + commands_run.add_argument("--arg", action="append", dest="args", default=[]) + _add_json_flag(commands_run) + figure_parser = subparsers.add_parser("figure") figure_subparsers = figure_parser.add_subparsers(dest="figure_command", required=True) figure_export = figure_subparsers.add_parser("export") @@ -331,6 +395,35 @@ def _build_control_parser() -> argparse.ArgumentParser: figure_export.add_argument("--ignore-selection", action="store_true") _add_json_flag(figure_export) + panel_parser = subparsers.add_parser("panel") + panel_subparsers = panel_parser.add_subparsers(dest="panel_command", required=True) + panel_samples = panel_subparsers.add_parser("samples") + panel_samples_subparsers = panel_samples.add_subparsers( + dest="panel_samples_command", required=True + ) + panel_samples_neighbors = panel_samples_subparsers.add_parser("show-neighbors") + _add_server_flags(panel_samples_neighbors) + panel_samples_neighbors.add_argument("--workspace", required=True) + panel_samples_neighbors.add_argument("--sample-id", required=True) + panel_samples_neighbors.add_argument("--layout-key") + panel_samples_neighbors.add_argument("--space-key") + panel_samples_neighbors.add_argument("--k", type=int, default=18) + _add_json_flag(panel_samples_neighbors) + + panel_labels = panel_subparsers.add_parser("labels") + panel_labels_subparsers = panel_labels.add_subparsers( + dest="panel_labels_command", required=True + ) + panel_labels_filter = panel_labels_subparsers.add_parser("filter") + _add_server_flags(panel_labels_filter) + panel_labels_filter.add_argument("--workspace", required=True) + panel_labels_filter.add_argument("--field", default="label") + value_group = panel_labels_filter.add_mutually_exclusive_group() + value_group.add_argument("--value") + value_group.add_argument("--missing-label", action="store_true") + value_group.add_argument("--clear", action="store_true") + _add_json_flag(panel_labels_filter) + ui_parser = subparsers.add_parser("ui") ui_subparsers = ui_parser.add_subparsers(dest="ui_command", required=True) ui_workspace = ui_subparsers.add_parser("workspace") @@ -364,22 +457,39 @@ def _build_control_parser() -> argparse.ArgumentParser: ui_selection_clear.add_argument("--workspace", required=True) _add_json_flag(ui_selection_clear) - ui_similarity = ui_subparsers.add_parser("similarity") - ui_similarity_subparsers = ui_similarity.add_subparsers( - dest="ui_similarity_command", required=True + ui_samples = ui_subparsers.add_parser("samples") + ui_samples_subparsers = ui_samples.add_subparsers( + dest="ui_samples_command", required=True + ) + ui_samples_retrieval = ui_samples_subparsers.add_parser("retrieval") + ui_samples_retrieval_subparsers = ui_samples_retrieval.add_subparsers( + dest="ui_samples_retrieval_command", required=True ) - ui_similarity_set = ui_similarity_subparsers.add_parser("set") - _add_server_flags(ui_similarity_set) - ui_similarity_set.add_argument("--workspace", required=True) - ui_similarity_set.add_argument("--sample-id", required=True) - ui_similarity_set.add_argument("--layout-key") - ui_similarity_set.add_argument("--space-key") - ui_similarity_set.add_argument("--k", type=int, default=18) - _add_json_flag(ui_similarity_set) - ui_similarity_clear = ui_similarity_subparsers.add_parser("clear") - _add_server_flags(ui_similarity_clear) - ui_similarity_clear.add_argument("--workspace", required=True) - _add_json_flag(ui_similarity_clear) + ui_samples_retrieval_set_anchor = ui_samples_retrieval_subparsers.add_parser("set-anchor") + _add_server_flags(ui_samples_retrieval_set_anchor) + ui_samples_retrieval_set_anchor.add_argument("--workspace", required=True) + ui_samples_retrieval_set_anchor.add_argument("--sample-id", required=True) + ui_samples_retrieval_set_anchor.add_argument("--layout-key") + ui_samples_retrieval_set_anchor.add_argument("--space-key") + ui_samples_retrieval_set_anchor.add_argument("--k", type=int, default=18) + _add_json_flag(ui_samples_retrieval_set_anchor) + ui_samples_retrieval_set_k = ui_samples_retrieval_subparsers.add_parser("set-k") + _add_server_flags(ui_samples_retrieval_set_k) + ui_samples_retrieval_set_k.add_argument("--workspace", required=True) + ui_samples_retrieval_set_k.add_argument("--k", type=int, required=True) + _add_json_flag(ui_samples_retrieval_set_k) + ui_samples_retrieval_set_text = ui_samples_retrieval_subparsers.add_parser("set-text") + _add_server_flags(ui_samples_retrieval_set_text) + ui_samples_retrieval_set_text.add_argument("--workspace", required=True) + ui_samples_retrieval_set_text.add_argument("--query", required=True) + ui_samples_retrieval_set_text.add_argument("--layout-key") + ui_samples_retrieval_set_text.add_argument("--space-key") + ui_samples_retrieval_set_text.add_argument("--k", type=int, default=18) + _add_json_flag(ui_samples_retrieval_set_text) + ui_samples_retrieval_clear = ui_samples_retrieval_subparsers.add_parser("clear") + _add_server_flags(ui_samples_retrieval_clear) + ui_samples_retrieval_clear.add_argument("--workspace", required=True) + _add_json_flag(ui_samples_retrieval_clear) ui_panel = ui_subparsers.add_parser("panel") ui_panel_subparsers = ui_panel.add_subparsers(dest="ui_panel_command", required=True) @@ -391,11 +501,11 @@ def _build_control_parser() -> argparse.ArgumentParser: ui_panel_add.add_argument( "--kind", choices=["auto", "extension", "scatter", "builtin"], default="auto" ) - ui_panel_add.add_argument("--builtin-panel", choices=["samples"]) + ui_panel_add.add_argument("--builtin-panel") ui_panel_add.add_argument("--extension") ui_panel_add.add_argument("--extension-panel") ui_panel_add.add_argument("--layout-key") - ui_panel_add.add_argument("--position", choices=["center", "right", "bottom"], default="right") + ui_panel_add.add_argument("--position", choices=["center", "right", "bottom"]) ui_panel_add.add_argument("--reference-panel-id") ui_panel_add.add_argument("--direction", choices=["right", "left", "above", "below", "within"]) _add_panel_layout_flags(ui_panel_add) @@ -406,6 +516,10 @@ def _build_control_parser() -> argparse.ArgumentParser: ) _add_json_flag(ui_panel_add) + ui_panel_definitions = ui_panel_subparsers.add_parser("definitions") + _add_server_flags(ui_panel_definitions) + _add_json_flag(ui_panel_definitions) + ui_panel_update = ui_panel_subparsers.add_parser("update") _add_server_flags(ui_panel_update) ui_panel_update.add_argument("--workspace", required=True) @@ -458,6 +572,24 @@ def _build_control_parser() -> argparse.ArgumentParser: ui_panel_show.add_argument("--panel-id", required=True) _add_json_flag(ui_panel_show) + ui_panel_state = ui_panel_subparsers.add_parser("state") + ui_panel_state_subparsers = ui_panel_state.add_subparsers( + dest="ui_panel_state_command", required=True + ) + ui_panel_state_get = ui_panel_state_subparsers.add_parser("get") + _add_server_flags(ui_panel_state_get) + ui_panel_state_get.add_argument("--workspace", required=True) + ui_panel_state_get.add_argument("--panel-id", required=True) + _add_json_flag(ui_panel_state_get) + ui_panel_state_patch = ui_panel_state_subparsers.add_parser("patch") + _add_server_flags(ui_panel_state_patch) + ui_panel_state_patch.add_argument("--workspace", required=True) + ui_panel_state_patch.add_argument("--panel-id", required=True) + ui_panel_state_patch.add_argument("--state-json", required=True) + ui_panel_state_patch.add_argument("--replace", action="store_true") + ui_panel_state_patch.add_argument("--expected-revision", type=int) + _add_json_flag(ui_panel_state_patch) + ui_panel_remove = ui_panel_subparsers.add_parser("remove") _add_server_flags(ui_panel_remove) ui_panel_remove.add_argument("--workspace", required=True) @@ -567,6 +699,18 @@ def _run_status_command(args: argparse.Namespace) -> None: _print_output(payload, as_json=args.json) +def _run_export_command(args: argparse.Namespace) -> None: + result = export_workspace(args.workspace_id, args.out) + payload = {"export": result.to_dict()} + if args.json: + _print_output(payload, as_json=True) + return + print( + f"Wrote static HyperView bundle to {result.output_dir} " + f"({result.num_samples} samples, {result.num_layouts} layouts)" + ) + + def _run_dataset_command(args: argparse.Namespace) -> None: if args.dataset_command == "list": runtime = HyperViewRuntime() @@ -588,6 +732,7 @@ def _run_dataset_command(args: argparse.Namespace) -> None: image_key=args.image_key, label_key=args.label_key, label_names_key=args.label_names_key, + text_key=args.text_key, max_samples=args.samples, shuffle=args.shuffle, seed=args.seed, @@ -740,6 +885,26 @@ def _run_jobs_command(args: argparse.Namespace) -> None: raise RuntimeError(f"Unsupported jobs command: {args.jobs_command}") +def _run_commands_command(args: argparse.Namespace) -> None: + base_url = _server_base_url(args.host, args.port) + if args.commands_command == "list": + _print_output(_http_get_json(f"{base_url}/api/control/commands"), as_json=args.json) + return + if args.commands_command == "run": + target = _parse_key_value_pairs(args.target, label="--target") + command_args = _parse_key_value_pairs(args.args, label="--arg") + payload = _run_control_command( + base_url, + args.command_id, + target=target, + args=command_args, + raise_on_error=False, + ) + _print_output(payload, as_json=args.json) + return + raise RuntimeError(f"Unsupported commands command: {args.commands_command}") + + def _resolve_figure_layout_key( dataset: Dataset, active_layout_key: str | None, @@ -825,6 +990,52 @@ def _run_figure_command(args: argparse.Namespace) -> None: ) +def _run_panel_command(args: argparse.Namespace) -> None: + base_url = _server_base_url(args.host, args.port) + target = {"workspace_id": args.workspace} + + if args.panel_command == "samples" and args.panel_samples_command == "show-neighbors": + payload = _run_control_command( + base_url, + "collection.neighbors.create", + target=target, + args={ + "sample_id": args.sample_id, + "layout_key": args.layout_key, + "space_key": args.space_key, + "k": args.k, + "source": "cli", + }, + ) + _print_output(payload, as_json=args.json) + return + + if args.panel_command == "labels" and args.panel_labels_command == "filter": + command_args: dict[str, Any] = { + "field": args.field, + "source": "cli", + } + if args.clear: + command_args["clear"] = True + elif args.missing_label: + command_args["value"] = None + elif args.value is not None: + command_args["value"] = args.value + else: + raise RuntimeError("Labels filter requires --value, --missing-label, or --clear") + + payload = _run_control_command( + base_url, + "collection.filter.set", + target=target, + args=command_args, + ) + _print_output(payload, as_json=args.json) + return + + raise RuntimeError(f"Unsupported panel command: {args.panel_command}") + + def _run_ui_command(args: argparse.Namespace) -> None: base_url = _server_base_url(args.host, args.port) if args.ui_command == "workspace" and args.ui_workspace_command == "set": @@ -856,27 +1067,65 @@ def _run_ui_command(args: argparse.Namespace) -> None: ) _print_output(payload, as_json=args.json) return - if args.ui_command == "similarity" and args.ui_similarity_command == "set": - payload = _http_send_json( - f"{base_url}/api/control/ui/similarity", - { - "workspace_id": args.workspace, - "sample_id": args.sample_id, - "layout_key": args.layout_key, - "space_key": args.space_key, - "k": args.k, - "source": "cli", - }, - ) - _print_output(payload, as_json=args.json) - return - if args.ui_command == "similarity" and args.ui_similarity_command == "clear": - payload = _http_send_json( - f"{base_url}/api/control/ui/similarity", - {"workspace_id": args.workspace}, - method="DELETE", - ) - _print_output(payload, as_json=args.json) + if args.ui_command == "samples" and args.ui_samples_command == "retrieval": + target = {"workspace_id": args.workspace} + if args.ui_samples_retrieval_command == "set-anchor": + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "panel.samples.retrieval.set-anchor", + target=target, + args={ + "sample_id": args.sample_id, + "layout_key": args.layout_key, + "space_key": args.space_key, + "k": args.k, + "source": "cli", + }, + ) + ) + _print_output(payload, as_json=args.json) + return + if args.ui_samples_retrieval_command == "set-k": + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "panel.samples.retrieval.set-k", + target=target, + args={"k": args.k}, + ) + ) + _print_output(payload, as_json=args.json) + return + if args.ui_samples_retrieval_command == "set-text": + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "panel.samples.retrieval.set-text-query", + target=target, + args={ + "query_text": args.query, + "layout_key": args.layout_key, + "space_key": args.space_key, + "k": args.k, + "source": "cli", + }, + ) + ) + _print_output(payload, as_json=args.json) + return + if args.ui_samples_retrieval_command == "clear": + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "panel.samples.retrieval.clear", + target=target, + ) + ) + _print_output(payload, as_json=args.json) + return + if args.ui_command == "panel" and args.ui_panel_command == "definitions": + _print_output(_http_get_json(f"{base_url}/api/panel-definitions"), as_json=args.json) return if args.ui_command == "panel" and args.ui_panel_command == "add": panel_kind = args.kind @@ -885,8 +1134,11 @@ def _run_ui_command(args: argparse.Namespace) -> None: panel_kind = "builtin" else: panel_kind = "scatter" if args.layout_key else "extension" - if panel_kind == "builtin" and args.builtin_panel != "samples": - raise RuntimeError("Built-in panels require --builtin-panel samples") + if panel_kind == "builtin" and not args.builtin_panel: + raise RuntimeError( + "Built-in panels require --builtin-panel. " + "Run `hyperview ui panel definitions` to list available panel types." + ) if panel_kind == "extension" and (not args.extension or not args.extension_panel): raise RuntimeError( "Extension panels require --extension and --extension-panel. " @@ -896,24 +1148,27 @@ def _run_ui_command(args: argparse.Namespace) -> None: raise RuntimeError("Scatter panels require --title") props = _parse_json_object(args.props_json, label="--props-json") layout_payload = _panel_layout_payload(args) - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - "title": args.title, - "kind": panel_kind, - "builtin_panel": args.builtin_panel, - "extension": args.extension, - "extension_panel": args.extension_panel, - "layout_key": args.layout_key, - "position": args.position, - "reference_panel_id": args.reference_panel_id, - "direction": args.direction, - **layout_payload, - "visible": not args.hidden, - "props": props, - }, + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.add", + target={"workspace_id": args.workspace}, + args={ + "panel_id": args.panel_id, + "title": args.title, + "kind": panel_kind, + "builtin_panel": args.builtin_panel, + "extension": args.extension, + "extension_panel": args.extension_panel, + "layout_key": args.layout_key, + "position": args.position, + "reference_panel_id": args.reference_panel_id, + "direction": args.direction, + **layout_payload, + "visible": not args.hidden, + "props": props, + }, + ) ) _print_output(payload, as_json=args.json) return @@ -931,10 +1186,7 @@ def _run_ui_command(args: argparse.Namespace) -> None: ): raise RuntimeError("Panel update requires at least one field to update") props = _parse_json_object(args.props_json, label="--props-json") - update_payload: dict[str, object | None] = { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - } + update_payload: dict[str, object | None] = {} if args.title is not None: update_payload["title"] = args.title if args.position is not None: @@ -950,10 +1202,13 @@ def _run_ui_command(args: argparse.Namespace) -> None: update_payload["visible"] = visible if props is not None: update_payload["props"] = props - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - update_payload, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.update", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + args=update_payload, + ) ) _print_output(payload, as_json=args.json) return @@ -961,73 +1216,93 @@ def _run_ui_command(args: argparse.Namespace) -> None: layout_payload = _panel_layout_payload(args) if not layout_payload: raise RuntimeError("Panel resize requires at least one size or constraint flag") - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - **layout_payload, - }, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.resize", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + args=layout_payload, + ) ) _print_output(payload, as_json=args.json) return if args.ui_command == "panel" and args.ui_panel_command == "move": - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - "position": args.position, - "reference_panel_id": args.reference_panel_id, - "direction": args.direction, - }, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.move", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + args={ + "position": args.position, + "reference_panel_id": args.reference_panel_id, + "direction": args.direction, + }, + ) ) _print_output(payload, as_json=args.json) return if args.ui_command == "panel" and args.ui_panel_command == "focus": - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - "active": True, - "visible": True, - }, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.focus", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + ) ) _print_output(payload, as_json=args.json) return if args.ui_command == "panel" and args.ui_panel_command == "close": - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - "visible": False, - }, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.close", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + ) ) _print_output(payload, as_json=args.json) return if args.ui_command == "panel" and args.ui_panel_command == "show": - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - { - "workspace_id": args.workspace, - "panel_id": args.panel_id, - "visible": True, - }, - method="PATCH", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.show", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + ) ) _print_output(payload, as_json=args.json) return + if args.ui_command == "panel" and args.ui_panel_command == "state": + if args.ui_panel_state_command == "get": + payload = _run_control_command( + base_url, + "workspace.panel.state.get", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + raise_on_error=False, + ) + _print_output(payload, as_json=args.json) + return + if args.ui_panel_state_command == "patch": + state = _parse_json_object(args.state_json, label="--state-json") + payload = _run_control_command( + base_url, + "workspace.panel.state.patch", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + args={ + "state": state, + "replace_state": args.replace, + "expected_revision": args.expected_revision, + }, + raise_on_error=False, + ) + _print_output(payload, as_json=args.json) + return if args.ui_command == "panel" and args.ui_panel_command == "remove": - payload = _http_send_json( - f"{base_url}/api/control/ui/panels", - {"workspace_id": args.workspace, "panel_id": args.panel_id}, - method="DELETE", + payload = _workspace_payload_from_command( + _run_control_command( + base_url, + "workspace.panel.remove", + target={"workspace_id": args.workspace, "panel_id": args.panel_id}, + ) ) _print_output(payload, as_json=args.json) return @@ -1139,6 +1414,9 @@ def main(argv: list[str] | None = None): if args.command == "status": _run_status_command(args) return + if args.command == "export": + _run_export_command(args) + return if args.command == "dataset": _run_dataset_command(args) return @@ -1157,9 +1435,15 @@ def main(argv: list[str] | None = None): if args.command == "jobs": _run_jobs_command(args) return + if args.command == "commands": + _run_commands_command(args) + return if args.command == "figure": _run_figure_command(args) return + if args.command == "panel": + _run_panel_command(args) + return if args.command == "ui": _run_ui_command(args) return diff --git a/src/hyperview/control/__init__.py b/src/hyperview/control/__init__.py new file mode 100644 index 0000000..74ea365 --- /dev/null +++ b/src/hyperview/control/__init__.py @@ -0,0 +1,24 @@ +"""Typed control command dispatch for HyperView.""" + +from hyperview.control.models import ( + CommandEnvelope, + CommandError, + CommandErrorPayload, + CommandMetadata, + CommandResult, +) +from hyperview.control.registry import CommandRegistry, CommandSpec +from hyperview.control.service import ControlService +from hyperview.control.ui_panel import create_default_command_registry + +__all__ = [ + "CommandEnvelope", + "CommandError", + "CommandErrorPayload", + "CommandMetadata", + "CommandRegistry", + "CommandResult", + "CommandSpec", + "ControlService", + "create_default_command_registry", +] diff --git a/src/hyperview/control/aliases.py b/src/hyperview/control/aliases.py new file mode 100644 index 0000000..ec93254 --- /dev/null +++ b/src/hyperview/control/aliases.py @@ -0,0 +1,41 @@ +"""Control command names and deprecated aliases.""" + +from __future__ import annotations + +import logging + +LOGGER = logging.getLogger(__name__) + +DEPRECATED_COMMAND_ALIASES: dict[str, str] = { + "ui.panel.add": "workspace.panel.add", + "ui.panel.update": "workspace.panel.update", + "ui.panel.remove": "workspace.panel.remove", + "ui.panel.resize": "workspace.panel.resize", + "ui.panel.move": "workspace.panel.move", + "ui.panel.focus": "workspace.panel.focus", + "ui.panel.close": "workspace.panel.close", + "ui.panel.show": "workspace.panel.show", + "ui.panel.update-props": "workspace.panel.update-props", + "ui.panel.state.get": "workspace.panel.state.get", + "ui.panel.state.patch": "workspace.panel.state.patch", + "samples.retrieval.set-anchor": "panel.samples.retrieval.set-anchor", + "samples.retrieval.set-text-query": "panel.samples.retrieval.set-text-query", + "samples.retrieval.set-k": "panel.samples.retrieval.set-k", + "samples.retrieval.clear": "panel.samples.retrieval.clear", + "panel.samples.show-neighbors": "collection.neighbors.create", + "panel.labels.filter": "collection.filter.set", +} + + +def resolve_command_alias(command_id: str) -> str: + """Return the canonical command id, warning when a deprecated alias is used.""" + + canonical_id = DEPRECATED_COMMAND_ALIASES.get(command_id) + if canonical_id is None: + return command_id + LOGGER.warning( + "Deprecated HyperView command '%s' used; use '%s' instead.", + command_id, + canonical_id, + ) + return canonical_id diff --git a/src/hyperview/control/models.py b/src/hyperview/control/models.py new file mode 100644 index 0000000..fb75c50 --- /dev/null +++ b/src/hyperview/control/models.py @@ -0,0 +1,74 @@ +"""Models shared by HyperView control command surfaces.""" + +from __future__ import annotations + +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field + +CommandOwner = Literal["backend", "frontend"] +CommandErrorCode = Literal[ + "unknown_command", + "validation_error", + "not_found", + "conflict", + "requires_ui_client", + "internal_error", +] + + +class CommandEnvelope(BaseModel): + """Generic command request envelope.""" + + model_config = ConfigDict(extra="forbid") + + command: str + target: dict[str, Any] = Field(default_factory=dict) + args: dict[str, Any] = Field(default_factory=dict) + + +class CommandMetadata(BaseModel): + """Serializable command discovery metadata.""" + + model_config = ConfigDict(extra="forbid") + + id: str + owner: CommandOwner + summary: str + target_schema: dict[str, Any] = Field(default_factory=dict) + args_schema: dict[str, Any] = Field(default_factory=dict) + + +class CommandErrorPayload(BaseModel): + """Machine-readable command error.""" + + model_config = ConfigDict(extra="forbid") + + code: CommandErrorCode + message: str + + +class CommandResult(BaseModel): + """Generic command result envelope.""" + + model_config = ConfigDict(extra="forbid") + + ok: bool + command: str + result: dict[str, Any] = Field(default_factory=dict) + workspace: dict[str, Any] | None = None + snapshot: dict[str, Any] | None = None + revision: int | None = None + error: CommandErrorPayload | None = None + + def to_dict(self) -> dict[str, Any]: + return self.model_dump(exclude_none=True) + + +class CommandError(Exception): + """Expected command failure with a stable public code.""" + + def __init__(self, code: CommandErrorCode, message: str) -> None: + super().__init__(message) + self.code = code + self.message = message diff --git a/src/hyperview/control/registry.py b/src/hyperview/control/registry.py new file mode 100644 index 0000000..2d0c763 --- /dev/null +++ b/src/hyperview/control/registry.py @@ -0,0 +1,85 @@ +"""Command registry and typed dispatch helpers.""" + +from __future__ import annotations + +from collections.abc import Callable +from dataclasses import dataclass +from typing import TYPE_CHECKING + +from pydantic import BaseModel, ValidationError + +from hyperview.control.aliases import resolve_command_alias +from hyperview.control.models import CommandError, CommandMetadata, CommandOwner + +if TYPE_CHECKING: + from hyperview.runtime import HyperViewRuntime, WorkspaceState + + +class EmptyArgs(BaseModel): + """Argument model for commands without arguments.""" + + model_config = {"extra": "forbid"} + + +@dataclass(frozen=True) +class CommandExecution: + """Internal successful command execution result.""" + + workspace: WorkspaceState | None = None + result: dict[str, object] | None = None + revision: int | None = None + + +CommandHandler = Callable[ + ["HyperViewRuntime", BaseModel, BaseModel], + CommandExecution, +] + + +@dataclass(frozen=True) +class CommandSpec: + """Registered command definition.""" + + id: str + owner: CommandOwner + summary: str + target_model: type[BaseModel] + args_model: type[BaseModel] + handler: CommandHandler + + def metadata(self) -> CommandMetadata: + return CommandMetadata( + id=self.id, + owner=self.owner, + summary=self.summary, + target_schema=self.target_model.model_json_schema(), + args_schema=self.args_model.model_json_schema(), + ) + + +class CommandRegistry: + """In-memory command registry.""" + + def __init__(self) -> None: + self._commands: dict[str, CommandSpec] = {} + + def register(self, spec: CommandSpec) -> None: + if spec.id in self._commands: + raise ValueError(f"Command already registered: {spec.id}") + self._commands[spec.id] = spec + + def get(self, command_id: str) -> CommandSpec: + command_id = resolve_command_alias(command_id) + try: + return self._commands[command_id] + except KeyError as exc: + raise CommandError("unknown_command", f"Unknown command: {command_id}") from exc + + def list_metadata(self) -> list[CommandMetadata]: + return [self._commands[key].metadata() for key in sorted(self._commands)] + + def validate_target_and_args(self, spec: CommandSpec, target: object, args: object) -> tuple[BaseModel, BaseModel]: + try: + return spec.target_model.model_validate(target), spec.args_model.model_validate(args) + except ValidationError as exc: + raise CommandError("validation_error", str(exc)) from exc diff --git a/src/hyperview/control/service.py b/src/hyperview/control/service.py new file mode 100644 index 0000000..43d4eef --- /dev/null +++ b/src/hyperview/control/service.py @@ -0,0 +1,84 @@ +"""Command execution service.""" + +from __future__ import annotations + +from pydantic import ValidationError + +from hyperview.control.models import ( + CommandEnvelope, + CommandError, + CommandErrorCode, + CommandErrorPayload, + CommandResult, +) +from hyperview.control.registry import CommandRegistry +from hyperview.runtime import HyperViewRuntime + + +class ControlService: + """Execute typed control commands against a runtime.""" + + def __init__(self, runtime: HyperViewRuntime, registry: CommandRegistry) -> None: + self.runtime = runtime + self.registry = registry + + def list_commands(self) -> list[dict[str, object]]: + return [metadata.model_dump() for metadata in self.registry.list_metadata()] + + def run(self, envelope: CommandEnvelope | dict[str, object]) -> CommandResult: + try: + request = ( + envelope + if isinstance(envelope, CommandEnvelope) + else CommandEnvelope.model_validate(envelope) + ) + spec = self.registry.get(request.command) + target, args = self.registry.validate_target_and_args( + spec, + request.target, + request.args, + ) + execution = spec.handler(self.runtime, target, args) + workspace = execution.workspace.to_dict() if execution.workspace is not None else None + snapshot = ( + self.runtime.snapshot(execution.workspace.id) + if execution.workspace is not None + else None + ) + revision = execution.revision + if revision is None and execution.workspace is not None: + revision = execution.workspace.ui.view_revision + return CommandResult( + ok=True, + command=spec.id, + result=dict(execution.result or {}), + workspace=workspace, + snapshot=snapshot, + revision=revision, + ) + except ValidationError as exc: + command = envelope.command if isinstance(envelope, CommandEnvelope) else "" + return self._error_result(command, "validation_error", str(exc)) + except CommandError as exc: + command = envelope.command if isinstance(envelope, CommandEnvelope) else "" + if not command and isinstance(envelope, dict): + command_value = envelope.get("command") + command = command_value if isinstance(command_value, str) else "" + return self._error_result(command, exc.code, exc.message) + except KeyError as exc: + command = envelope.command if isinstance(envelope, CommandEnvelope) else str(envelope.get("command", "")) + message = str(exc.args[0]) if exc.args else str(exc) + return self._error_result(command, "not_found", message) + except LookupError as exc: + command = envelope.command if isinstance(envelope, CommandEnvelope) else str(envelope.get("command", "")) + return self._error_result(command, "not_found", str(exc)) + except ValueError as exc: + command = envelope.command if isinstance(envelope, CommandEnvelope) else str(envelope.get("command", "")) + return self._error_result(command, "validation_error", str(exc)) + + def _error_result(self, command: str, code: CommandErrorCode, message: str) -> CommandResult: + return CommandResult( + ok=False, + command=command, + error=CommandErrorPayload(code=code, message=message), + ) diff --git a/src/hyperview/control/ui_panel.py b/src/hyperview/control/ui_panel.py new file mode 100644 index 0000000..853c300 --- /dev/null +++ b/src/hyperview/control/ui_panel.py @@ -0,0 +1,731 @@ +"""Backend-owned panel control commands.""" + +from __future__ import annotations + +from typing import Annotated, Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, model_validator + +from hyperview.control.models import CommandError +from hyperview.control.registry import CommandExecution, CommandRegistry, CommandSpec, EmptyArgs +from hyperview.runtime import HyperViewRuntime, SimilarityQueryState + +PositivePanelDimension = Annotated[int, Field(gt=0)] + + +class PanelTarget(BaseModel): + model_config = ConfigDict(extra="forbid") + + workspace_id: str + panel_id: str + + +class PanelResizeArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + width: PositivePanelDimension | None = None + height: PositivePanelDimension | None = None + min_width: PositivePanelDimension | None = None + min_height: PositivePanelDimension | None = None + max_width: PositivePanelDimension | None = None + max_height: PositivePanelDimension | None = None + + @model_validator(mode="after") + def require_at_least_one_field(self) -> PanelResizeArgs: + if not self.model_fields_set: + raise ValueError("Panel resize requires at least one size or constraint field") + return self + + +class PanelMoveArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + position: Literal["center", "right", "bottom"] + reference_panel_id: str | None = None + direction: Literal["right", "left", "above", "below", "within"] | None = None + + +class PanelAddArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + panel_id: str + kind: Literal["extension", "scatter", "builtin"] + title: str | None = None + builtin_panel: str | None = None + extension: str | None = None + extension_panel: str | None = None + layout_key: str | None = None + position: Literal["center", "right", "bottom"] | None = None + reference_panel_id: str | None = None + direction: Literal["right", "left", "above", "below", "within"] | None = None + width: PositivePanelDimension | None = None + height: PositivePanelDimension | None = None + min_width: PositivePanelDimension | None = None + min_height: PositivePanelDimension | None = None + max_width: PositivePanelDimension | None = None + max_height: PositivePanelDimension | None = None + visible: bool = True + props: dict[str, Any] | None = None + geometry: str | None = None + layout_dimension: int | None = None + require_resolved_layout: bool = True + + +class PanelUpdateArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + title: str | None = None + position: Literal["center", "right", "bottom"] | None = None + reference_panel_id: str | None = None + direction: Literal["right", "left", "above", "below", "within"] | None = None + width: PositivePanelDimension | None = None + height: PositivePanelDimension | None = None + min_width: PositivePanelDimension | None = None + min_height: PositivePanelDimension | None = None + max_width: PositivePanelDimension | None = None + max_height: PositivePanelDimension | None = None + visible: bool | None = None + active: bool | None = None + props: dict[str, Any] | None = None + + @model_validator(mode="after") + def require_at_least_one_field(self) -> PanelUpdateArgs: + if not self.model_fields_set: + raise ValueError("Panel update requires at least one field") + return self + + +class PanelPropsArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + props: dict[str, Any] + + +class PanelStatePatchArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + state: dict[str, Any] + replace_state: bool = False + expected_revision: int | None = None + client_id: str | None = None + + +class WorkspaceTarget(BaseModel): + model_config = ConfigDict(extra="forbid") + + workspace_id: str + + +class SamplesRetrievalSetArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + sample_id: str + layout_key: str | None = None + space_key: str | None = None + k: int = 18 + source: str | None = None + + @model_validator(mode="after") + def validate_limit(self) -> SamplesRetrievalSetArgs: + if self.k < 1: + raise ValueError("k must be a positive integer") + return self + + +class SamplesRetrievalSetKArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + k: int + + @model_validator(mode="after") + def validate_limit(self) -> SamplesRetrievalSetKArgs: + if self.k < 1: + raise ValueError("k must be a positive integer") + return self + + +class SamplesRetrievalSetTextArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + query_text: str + layout_key: str | None = None + space_key: str | None = None + k: int = 18 + source: str | None = None + + @model_validator(mode="after") + def validate_query(self) -> SamplesRetrievalSetTextArgs: + if not self.query_text.strip(): + raise ValueError("query_text must be a non-empty string") + if self.k < 1: + raise ValueError("k must be a positive integer") + return self + + +class LabelsFilterArgs(BaseModel): + model_config = ConfigDict(extra="forbid") + + field: str = "label" + value: Any = None + clear: bool = False + source: str | None = None + + @model_validator(mode="after") + def require_value_unless_clear(self) -> LabelsFilterArgs: + if not self.clear and "value" not in self.model_fields_set: + raise ValueError("Labels filter requires value unless clear=true") + return self + + +def _panel_target(target: BaseModel) -> PanelTarget: + if not isinstance(target, PanelTarget): + raise CommandError("validation_error", "Invalid panel target") + return target + + +def _resize_args(args: BaseModel) -> PanelResizeArgs: + if not isinstance(args, PanelResizeArgs): + raise CommandError("validation_error", "Invalid panel resize args") + return args + + +def _move_args(args: BaseModel) -> PanelMoveArgs: + if not isinstance(args, PanelMoveArgs): + raise CommandError("validation_error", "Invalid panel move args") + return args + + +def _panel_add_args(args: BaseModel) -> PanelAddArgs: + if not isinstance(args, PanelAddArgs): + raise CommandError("validation_error", "Invalid panel add args") + return args + + +def _panel_update_args(args: BaseModel) -> PanelUpdateArgs: + if not isinstance(args, PanelUpdateArgs): + raise CommandError("validation_error", "Invalid panel update args") + return args + + +def _panel_props_args(args: BaseModel) -> PanelPropsArgs: + if not isinstance(args, PanelPropsArgs): + raise CommandError("validation_error", "Invalid panel props args") + return args + + +def _panel_state_patch_args(args: BaseModel) -> PanelStatePatchArgs: + if not isinstance(args, PanelStatePatchArgs): + raise CommandError("validation_error", "Invalid panel state patch args") + return args + + +def _workspace_target(target: BaseModel) -> WorkspaceTarget: + if not isinstance(target, WorkspaceTarget): + raise CommandError("validation_error", "Invalid workspace target") + return target + + +def _samples_retrieval_set_args(args: BaseModel) -> SamplesRetrievalSetArgs: + if not isinstance(args, SamplesRetrievalSetArgs): + raise CommandError("validation_error", "Invalid samples retrieval set args") + return args + + +def _samples_retrieval_set_k_args(args: BaseModel) -> SamplesRetrievalSetKArgs: + if not isinstance(args, SamplesRetrievalSetKArgs): + raise CommandError("validation_error", "Invalid samples retrieval k args") + return args + + +def _samples_retrieval_set_text_args(args: BaseModel) -> SamplesRetrievalSetTextArgs: + if not isinstance(args, SamplesRetrievalSetTextArgs): + raise CommandError("validation_error", "Invalid samples text retrieval args") + return args + + +def _labels_filter_args(args: BaseModel) -> LabelsFilterArgs: + if not isinstance(args, LabelsFilterArgs): + raise CommandError("validation_error", "Invalid labels filter args") + return args + + +def _workspace_execution(workspace) -> CommandExecution: + return CommandExecution(workspace=workspace, revision=workspace.ui.view_revision) + + +def _fields_set_payload(model: BaseModel) -> dict[str, Any]: + return {field: getattr(model, field) for field in model.model_fields_set} + + +def _samples_panel_collection_result(workspace) -> dict[str, object]: + state_entry = workspace.ui.panels.get("samples") + state = state_entry.state if state_entry is not None else {} + collection = state.get("collection") + collection_id = state.get("collection_id") + return { + "panel_id": "samples", + "collection_id": collection_id if isinstance(collection_id, str) else None, + "collection": collection if isinstance(collection, dict) else None, + } + + +def _add_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + add_args = _panel_add_args(args) + workspace = runtime.add_runtime_panel( + workspace_target.workspace_id, + panel_id=add_args.panel_id, + title=add_args.title, + kind=add_args.kind, + builtin_panel=add_args.builtin_panel, + extension=add_args.extension, + extension_panel=add_args.extension_panel, + layout_key=add_args.layout_key, + position=add_args.position, + reference_panel_id=add_args.reference_panel_id, + direction=add_args.direction, + width=add_args.width, + height=add_args.height, + min_width=add_args.min_width, + min_height=add_args.min_height, + max_width=add_args.max_width, + max_height=add_args.max_height, + visible=add_args.visible, + props=add_args.props, + geometry=add_args.geometry, + layout_dimension=add_args.layout_dimension, + require_resolved_layout=add_args.require_resolved_layout, + ) + return _workspace_execution(workspace) + + +def _update_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + update_args = _panel_update_args(args) + patch = _fields_set_payload(update_args) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + **patch, + ) + ) + + +def _remove_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + return _workspace_execution( + runtime.remove_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + ) + ) + + +def _resize_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + resize_args = _resize_args(args) + patch = _fields_set_payload(resize_args) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + **patch, + ) + ) + + +def _move_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + move_args = _move_args(args) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + position=move_args.position, + reference_panel_id=move_args.reference_panel_id, + direction=move_args.direction, + ) + ) + + +def _focus_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + active=True, + visible=True, + ) + ) + + +def _close_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + visible=False, + ) + ) + + +def _show_panel( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + visible=True, + ) + ) + + +def _update_panel_props( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + props_args = _panel_props_args(args) + return _workspace_execution( + runtime.update_custom_panel( + panel_target.workspace_id, + panel_target.panel_id, + props=props_args.props, + ) + ) + + +def _get_panel_state( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + workspace = runtime.get_workspace(panel_target.workspace_id) + state_payload = runtime.get_panel_state( + panel_target.workspace_id, + panel_target.panel_id, + ) + return CommandExecution( + workspace=workspace, + result=state_payload, + revision=int(state_payload.get("state_revision") or 0), + ) + + +def _patch_panel_state( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + panel_target = _panel_target(target) + patch_args = _panel_state_patch_args(args) + try: + workspace = runtime.patch_panel_state( + panel_target.workspace_id, + panel_target.panel_id, + patch_args.state, + replace_state=patch_args.replace_state, + expected_revision=patch_args.expected_revision, + source_client_id=patch_args.client_id, + ) + except ValueError as exc: + if "revision conflict" in str(exc): + raise CommandError("conflict", str(exc)) from exc + raise + state_payload = runtime.get_panel_state(panel_target.workspace_id, panel_target.panel_id) + return CommandExecution( + workspace=workspace, + result=state_payload, + revision=int(state_payload.get("state_revision") or 0), + ) + + +def _set_samples_retrieval_anchor( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + retrieval_args = _samples_retrieval_set_args(args) + query = runtime.resolve_similarity_query( + workspace_target.workspace_id, + retrieval_args.sample_id, + layout_key=retrieval_args.layout_key, + space_key=retrieval_args.space_key, + k=retrieval_args.k, + source=retrieval_args.source, + ) + workspace = runtime.set_samples_retrieval(workspace_target.workspace_id, query) + return CommandExecution( + workspace=workspace, + result=_samples_panel_collection_result(workspace), + revision=workspace.ui.view_revision, + ) + + +def _clear_samples_retrieval( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + workspace = runtime.clear_samples_retrieval(workspace_target.workspace_id) + return CommandExecution( + workspace=workspace, + result=_samples_panel_collection_result(workspace), + revision=workspace.ui.view_revision, + ) + + +def _set_samples_retrieval_k( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + k_args = _samples_retrieval_set_k_args(args) + current_query = runtime.get_samples_retrieval_query(workspace_target.workspace_id) + if current_query is None: + raise CommandError("validation_error", "Samples retrieval has no active anchor") + + next_query = SimilarityQueryState( + anchor_sample_id=current_query.anchor_sample_id, + query_text=current_query.query_text, + layout_key=current_query.layout_key, + space_key=current_query.space_key, + k=k_args.k, + source=current_query.source, + ) + workspace = runtime.set_samples_retrieval(workspace_target.workspace_id, next_query) + return CommandExecution( + workspace=workspace, + result=_samples_panel_collection_result(workspace), + revision=workspace.ui.view_revision, + ) + + +def _set_samples_text_retrieval( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + retrieval_args = _samples_retrieval_set_text_args(args) + query = runtime.resolve_text_retrieval_query( + workspace_target.workspace_id, + retrieval_args.query_text, + layout_key=retrieval_args.layout_key, + space_key=retrieval_args.space_key, + k=retrieval_args.k, + source=retrieval_args.source, + ) + workspace = runtime.set_samples_retrieval(workspace_target.workspace_id, query) + return CommandExecution( + workspace=workspace, + result=_samples_panel_collection_result(workspace), + revision=workspace.ui.view_revision, + ) + + +def _filter_labels( + runtime: HyperViewRuntime, + target: BaseModel, + args: BaseModel, +) -> CommandExecution: + workspace_target = _workspace_target(target) + filter_args = _labels_filter_args(args) + if filter_args.clear: + workspace = runtime.clear_samples_filter(workspace_target.workspace_id) + else: + workspace = runtime.set_samples_filter( + workspace_target.workspace_id, + field=filter_args.field, + value=filter_args.value, + source=filter_args.source, + ) + return CommandExecution( + workspace=workspace, + result=_samples_panel_collection_result(workspace), + revision=workspace.ui.view_revision, + ) + + +def create_default_command_registry() -> CommandRegistry: + registry = CommandRegistry() + for spec in ( + CommandSpec( + id="workspace.panel.add", + owner="backend", + summary="Add or replace a runtime-managed panel.", + target_model=WorkspaceTarget, + args_model=PanelAddArgs, + handler=_add_panel, + ), + CommandSpec( + id="workspace.panel.update", + owner="backend", + summary="Update durable runtime panel fields.", + target_model=PanelTarget, + args_model=PanelUpdateArgs, + handler=_update_panel, + ), + CommandSpec( + id="workspace.panel.remove", + owner="backend", + summary="Remove a runtime-managed panel from the workspace view.", + target_model=PanelTarget, + args_model=EmptyArgs, + handler=_remove_panel, + ), + CommandSpec( + id="workspace.panel.resize", + owner="backend", + summary="Resize a runtime-managed panel.", + target_model=PanelTarget, + args_model=PanelResizeArgs, + handler=_resize_panel, + ), + CommandSpec( + id="workspace.panel.move", + owner="backend", + summary="Move a runtime-managed panel.", + target_model=PanelTarget, + args_model=PanelMoveArgs, + handler=_move_panel, + ), + CommandSpec( + id="workspace.panel.focus", + owner="backend", + summary="Focus a runtime-managed panel.", + target_model=PanelTarget, + args_model=EmptyArgs, + handler=_focus_panel, + ), + CommandSpec( + id="workspace.panel.close", + owner="backend", + summary="Hide a runtime-managed panel without removing it.", + target_model=PanelTarget, + args_model=EmptyArgs, + handler=_close_panel, + ), + CommandSpec( + id="workspace.panel.show", + owner="backend", + summary="Show a hidden runtime-managed panel.", + target_model=PanelTarget, + args_model=EmptyArgs, + handler=_show_panel, + ), + CommandSpec( + id="workspace.panel.update-props", + owner="backend", + summary="Replace documented props for a runtime-managed panel.", + target_model=PanelTarget, + args_model=PanelPropsArgs, + handler=_update_panel_props, + ), + CommandSpec( + id="workspace.panel.state.get", + owner="backend", + summary="Read durable runtime-managed state for a panel.", + target_model=PanelTarget, + args_model=EmptyArgs, + handler=_get_panel_state, + ), + CommandSpec( + id="workspace.panel.state.patch", + owner="backend", + summary="Patch durable runtime-managed state for a panel.", + target_model=PanelTarget, + args_model=PanelStatePatchArgs, + handler=_patch_panel_state, + ), + CommandSpec( + id="panel.samples.retrieval.set-anchor", + owner="backend", + summary="Set Samples panel retrieval anchor state.", + target_model=WorkspaceTarget, + args_model=SamplesRetrievalSetArgs, + handler=_set_samples_retrieval_anchor, + ), + CommandSpec( + id="panel.samples.retrieval.clear", + owner="backend", + summary="Clear Samples panel retrieval state.", + target_model=WorkspaceTarget, + args_model=EmptyArgs, + handler=_clear_samples_retrieval, + ), + CommandSpec( + id="panel.samples.retrieval.set-k", + owner="backend", + summary="Set Samples panel retrieval result count.", + target_model=WorkspaceTarget, + args_model=SamplesRetrievalSetKArgs, + handler=_set_samples_retrieval_k, + ), + CommandSpec( + id="panel.samples.retrieval.set-text-query", + owner="backend", + summary="Set Samples panel text retrieval query state.", + target_model=WorkspaceTarget, + args_model=SamplesRetrievalSetTextArgs, + handler=_set_samples_text_retrieval, + ), + CommandSpec( + id="collection.neighbors.create", + owner="backend", + summary="Create a nearest-neighbor collection for the Samples panel.", + target_model=WorkspaceTarget, + args_model=SamplesRetrievalSetArgs, + handler=_set_samples_retrieval_anchor, + ), + CommandSpec( + id="collection.filter.set", + owner="backend", + summary="Create or clear a label filter collection for the Samples panel.", + target_model=WorkspaceTarget, + args_model=LabelsFilterArgs, + handler=_filter_labels, + ), + ): + registry.register(spec) + return registry diff --git a/src/hyperview/core/dataset.py b/src/hyperview/core/dataset.py index a38f847..c36c44e 100644 --- a/src/hyperview/core/dataset.py +++ b/src/hyperview/core/dataset.py @@ -15,6 +15,7 @@ import numpy as np from PIL import Image +from hyperview._compat import disable_blocked_datasets_torch_shared_memory from hyperview.core.sample import Sample from hyperview.storage.backend import StorageBackend from hyperview.storage.schema import ( @@ -146,6 +147,7 @@ def __init__( persist: bool = True, storage: StorageBackend | None = None, ): + disable_blocked_datasets_torch_shared_memory() self.name = name or f"dataset_{uuid.uuid4().hex[:8]}" if storage is not None: @@ -250,6 +252,7 @@ def add_from_huggingface( image_key: str = "img", label_key: str | None = "fine_label", label_names_key: str | None = None, + text_key: str | None = None, max_samples: int | None = None, shuffle: bool = False, seed: int = 42, @@ -325,6 +328,8 @@ def attach_huggingface_source_index( columns = [image_key] if label_key: columns.append(label_key) + if text_key: + columns.append(text_key) stream = stream.select_columns(list(dict.fromkeys(columns))) stream = stream.map(attach_huggingface_source_index, with_indices=True) if shuffle: @@ -445,6 +450,16 @@ def heartbeat_reporter() -> None: else: label = str(label_idx) + text: str | None = None + if text_key and text_key in item: + raw_text = item[text_key] + if isinstance(raw_text, list): + text = " ".join(str(part).strip() for part in raw_text if str(part).strip()) or None + elif raw_text is not None: + text = str(raw_text).strip() or None + + modality = "multimodal" if text else "image" + safe_name = dataset_name.replace("/", "_") sample_id = f"{safe_name}_{config_name}_{fingerprint}_{split}_{source_index}" @@ -482,6 +497,8 @@ def heartbeat_reporter() -> None: id=sample_id, filepath=str(image_path), label=label, + text=text, + modality=modality, metadata=metadata, ) @@ -584,6 +601,7 @@ def compute_embeddings( model_id=model, checkpoint=checkpoint, provider_kwargs=provider_kwargs, + modality="multimodal" if provider == "embed-anything" else "image", ) space_key, _num_computed, _num_skipped = compute_embeddings( @@ -715,6 +733,65 @@ def find_similar_by_vector( ) return self._storage.find_similar_by_vector(vector, k, resolved_space_key) + def _embedding_spec_for_space(self, space_key: str) -> Any: + from hyperview.embeddings.engine import EmbeddingSpec + + space = next( + (item for item in self.list_spaces() if item.space_key == space_key), + None, + ) + if space is None: + raise ValueError(f"Space not found: {space_key}") + + config = dict(space.config or {}) + provider = str(config.get("provider") or space.provider or "embed-anything") + model_id = str(space.model_id) + checkpoint = config.get("checkpoint") + provider_kwargs = { + key: value + for key, value in config.items() + if key not in {"provider", "model_id", "checkpoint", "dim", "geometry", "modality", "params", "params_source", "spatial_dim"} + } + modality = config.get("modality") or ("multimodal" if provider == "embed-anything" else "image") + return EmbeddingSpec( + provider=provider, + model_id=model_id, + checkpoint=checkpoint if isinstance(checkpoint, str) else None, + provider_kwargs=provider_kwargs, + modality=modality, # type: ignore[arg-type] + ) + + def find_similar_by_text( + self, + text: str, + k: int = 10, + space_key: str | None = None, + *, + layout_key: str | None = None, + ) -> list[tuple[Sample, float]]: + """Find k most similar samples to a natural-language text query.""" + query = str(text or "").strip() + if not query: + raise ValueError("text query must be a non-empty string") + + resolved_space_key = self._resolve_similarity_space_key( + space_key=space_key, + layout_key=layout_key, + ) + if resolved_space_key is None: + raise ValueError("No embedding spaces available") + + from hyperview.embeddings.engine import get_engine + + spec = self._embedding_spec_for_space(resolved_space_key) + vector = get_engine().embed_texts([query], spec)[0] + return self._storage.find_similar_by_text( + query, + k, + resolved_space_key, + query_vector=vector, + ) + def set_coords( self, geometry: str, @@ -799,13 +876,19 @@ def get_samples_paginated( offset: int = 0, limit: int = 100, label: str | None = None, + missing_label: bool = False, ) -> tuple[list[Sample], int]: """Get paginated samples. This avoids loading all samples into memory and is used by the server API for efficient pagination. """ - return self._storage.get_samples_paginated(offset=offset, limit=limit, label=label) + return self._storage.get_samples_paginated( + offset=offset, + limit=limit, + label=label, + missing_label=missing_label, + ) def get_samples_by_ids(self, sample_ids: list[str]) -> list[Sample]: """Retrieve multiple samples by ID. @@ -851,6 +934,7 @@ def get_lasso_candidates_aabb( y_min: float, y_max: float, label_filter: str | None = None, + missing_label_filter: bool = False, ) -> tuple[list[str], np.ndarray]: """Return candidate (id, xy) rows within an AABB for a layout.""" return self._storage.get_lasso_candidates_aabb( @@ -860,6 +944,7 @@ def get_lasso_candidates_aabb( y_min=y_min, y_max=y_max, label_filter=label_filter, + missing_label_filter=missing_label_filter, ) def save(self, filepath: str, include_thumbnails: bool = True) -> None: diff --git a/src/hyperview/core/sample.py b/src/hyperview/core/sample.py index 01ede8e..f196d2b 100644 --- a/src/hyperview/core/sample.py +++ b/src/hyperview/core/sample.py @@ -19,10 +19,12 @@ class Sample(BaseModel): id: str = Field(..., description="Unique identifier for the sample") filepath: str = Field(..., description="Path to the image file") label: str | None = Field(default=None, description="Label for the sample") + text: str | None = Field(default=None, description="Text content or caption") metadata: dict[str, Any] = Field(default_factory=dict, description="Additional metadata") thumbnail_base64: str | None = Field(default=None, description="Cached thumbnail as base64") width: int | None = Field(default=None, description="Image width in pixels") height: int | None = Field(default=None, description="Image height in pixels") + modality: str = Field(default="image", description="Data modality: image, text, or multimodal") model_config = {"arbitrary_types_allowed": True} @@ -78,6 +80,8 @@ def to_api_dict( "filepath": self.filepath, "filename": self.filename, "label": self.label, + "text": self.text, + "modality": self.modality, "metadata": self.metadata, "width": self.width, "height": self.height, diff --git a/src/hyperview/core/selection.py b/src/hyperview/core/selection.py index ef25c14..50fd1bf 100644 --- a/src/hyperview/core/selection.py +++ b/src/hyperview/core/selection.py @@ -267,11 +267,23 @@ def select_ids_for_3d_lasso( viewport_width: int, viewport_height: int, label_filter: str | None, + missing_label_filter: bool = False, ) -> list[str]: width = max(1, int(viewport_width)) height = max(1, int(viewport_height)) - if label_filter is not None: + if missing_label_filter: + label_mask = np.fromiter( + (not label for label in labels), + dtype=bool, + count=len(labels), + ) + if not np.any(label_mask): + return [] + kept_indices = np.flatnonzero(label_mask) + ids = [ids[int(i)] for i in kept_indices] + coords = coords[label_mask] + elif label_filter is not None: label_mask = np.fromiter( (label == label_filter for label in labels), dtype=bool, diff --git a/src/hyperview/embeddings/providers/lancedb_providers.py b/src/hyperview/embeddings/providers/lancedb_providers.py index 749cdd2..b3aa855 100644 --- a/src/hyperview/embeddings/providers/lancedb_providers.py +++ b/src/hyperview/embeddings/providers/lancedb_providers.py @@ -85,6 +85,12 @@ def compute_source_embeddings( def compute_query_embeddings( self, query: Any, *args: Any, **kwargs: Any ) -> list[np.ndarray | None]: + if isinstance(query, str): + model = self._get_computer()._get_model() + result = model.embed_query(query) + if not result: + raise RuntimeError(f"EmbedAnything returned no embedding for text query: {query!r}") + return [np.asarray(result[0].embedding, dtype=np.float32)] return self.compute_source_embeddings([query], *args, **kwargs) diff --git a/src/hyperview/extensions.py b/src/hyperview/extensions.py index 692dbe2..d6fe82f 100644 --- a/src/hyperview/extensions.py +++ b/src/hyperview/extensions.py @@ -36,11 +36,23 @@ except ImportError: # pragma: no cover - exercised on Python 3.10 import tomli as tomllib # type: ignore[no-redef] +from hyperview.panel_definitions import PanelDefinition from hyperview.tools import ToolRecord, drain_pending_tools EXTENSION_MANIFEST_NAME = "extension.toml" DEFAULT_LOCAL_EXTENSIONS_DIR = ".hyperview/extensions" VALID_PANEL_POSITIONS = {"center", "right", "bottom"} +VALID_PANEL_DIRECTIONS = {"right", "left", "above", "below", "within"} +PANEL_LAYOUT_FIELDS = ( + "reference_panel_id", + "direction", + "width", + "height", + "min_width", + "min_height", + "max_width", + "max_height", +) @dataclass @@ -49,6 +61,43 @@ class PanelSpecEntry: title: str position: str = "right" file: str = "panel.jsx" + panel_type: str | None = None + label: str | None = None + default_props: dict[str, object] = field(default_factory=dict) + default_state: dict[str, object] = field(default_factory=dict) + props_schema: dict[str, object] | None = None + state_schema: dict[str, object] | None = None + commands: list[str] = field(default_factory=list) + queries: list[str] = field(default_factory=list) + lifecycle: dict[str, object] = field(default_factory=dict) + default_layout: dict[str, object] = field(default_factory=dict) + allow_multiple: bool = True + icon: str | None = None + category: str | None = None + + def resolved_panel_type(self, extension_name: str) -> str: + panel_type = (self.panel_type or "").strip() + return panel_type or f"{extension_name}.{self.id}" + + def to_definition(self, extension_name: str) -> PanelDefinition: + return PanelDefinition( + panel_type=self.resolved_panel_type(extension_name), + label=self.label or self.title, + title=self.title, + source="extension", + extension=extension_name, + default_props=dict(self.default_props), + default_state=dict(self.default_state), + props_schema=dict(self.props_schema) if self.props_schema is not None else None, + state_schema=dict(self.state_schema) if self.state_schema is not None else None, + commands=list(self.commands), + queries=list(self.queries), + lifecycle=dict(self.lifecycle), + default_layout=dict(self.default_layout), + allow_multiple=self.allow_multiple, + icon=self.icon, + category=self.category, + ) @dataclass @@ -101,6 +150,19 @@ def load(cls, folder: Path) -> ExtensionManifest: title=str(entry.get("title") or panel_id), position=position, file=str(entry.get("file") or "panel.jsx"), + panel_type=_optional_str(entry.get("panel_type")), + label=_optional_str(entry.get("label")), + default_props=_dict_or_empty(entry.get("default_props")), + default_state=_dict_or_empty(entry.get("default_state")), + props_schema=_optional_dict(entry.get("props_schema")), + state_schema=_optional_dict(entry.get("state_schema")), + commands=_string_list(entry.get("commands")), + queries=_string_list(entry.get("queries")), + lifecycle=_dict_or_empty(entry.get("lifecycle")), + default_layout=_panel_default_layout(entry, position), + allow_multiple=bool(entry.get("allow_multiple", True)), + icon=_optional_str(entry.get("icon")), + category=_optional_str(entry.get("category")), ) ) @@ -113,6 +175,51 @@ def load(cls, folder: Path) -> ExtensionManifest: ) +def _optional_str(value: object) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _dict_or_empty(value: object) -> dict[str, object]: + return dict(value) if isinstance(value, dict) else {} + + +def _optional_dict(value: object) -> dict[str, object] | None: + return dict(value) if isinstance(value, dict) else None + + +def _string_list(value: object) -> list[str]: + if not isinstance(value, list): + return [] + return [str(item) for item in value if str(item).strip()] + + +def _panel_default_layout(entry: dict[str, object], position: str) -> dict[str, object]: + layout = _dict_or_empty(entry.get("default_layout")) + layout.setdefault("position", position) + for field_name in PANEL_LAYOUT_FIELDS: + value = entry.get(field_name) + if value is not None: + layout[field_name] = value + layout_position = str(layout.get("position") or position) + if layout_position not in VALID_PANEL_POSITIONS: + raise ValueError( + f"unsupported panel default_layout position '{layout_position}'" + ) + layout["position"] = layout_position + layout_direction = layout.get("direction") + if layout_direction is not None: + layout_direction = str(layout_direction) + if layout_direction not in VALID_PANEL_DIRECTIONS: + raise ValueError( + f"unsupported panel default_layout direction '{layout_direction}'" + ) + layout["direction"] = layout_direction + return layout + + @dataclass class LoadedExtension: """Result of loading an extension's Python tool modules.""" diff --git a/src/hyperview/panel_definitions.py b/src/hyperview/panel_definitions.py new file mode 100644 index 0000000..c41f5f8 --- /dev/null +++ b/src/hyperview/panel_definitions.py @@ -0,0 +1,114 @@ +"""Serializable panel definition metadata. + +Panel definitions describe what a panel type is and what default runtime +metadata it declares. They intentionally exclude frontend implementation +bindings such as React components or Dockview adapter details. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from typing import Any + + +def _json_object_copy(value: dict[str, Any]) -> dict[str, Any]: + return json.loads(json.dumps(value)) + + +@dataclass(frozen=True) +class PanelDefinition: + panel_type: str + label: str + source: str + title: str | None = None + extension: str | None = None + default_props: dict[str, Any] = field(default_factory=dict) + default_state: dict[str, Any] = field(default_factory=dict) + props_schema: dict[str, Any] | None = None + state_schema: dict[str, Any] | None = None + commands: list[str] = field(default_factory=list) + queries: list[str] = field(default_factory=list) + lifecycle: dict[str, Any] = field(default_factory=dict) + default_layout: dict[str, Any] = field(default_factory=dict) + allow_multiple: bool = True + icon: str | None = None + category: str | None = None + + def __post_init__(self) -> None: + if not self.panel_type.strip(): + raise ValueError("panel_type must be a non-empty string") + if not self.label.strip(): + raise ValueError("panel definition label must be a non-empty string") + if not self.source.strip(): + raise ValueError("panel definition source must be a non-empty string") + + def to_dict(self) -> dict[str, Any]: + return { + "panel_type": self.panel_type, + "label": self.label, + "title": self.title or self.label, + "source": self.source, + "extension": self.extension, + "default_props": _json_object_copy(self.default_props), + "default_state": _json_object_copy(self.default_state), + "props_schema": ( + _json_object_copy(self.props_schema) + if self.props_schema is not None + else None + ), + "state_schema": ( + _json_object_copy(self.state_schema) + if self.state_schema is not None + else None + ), + "commands": list(self.commands), + "queries": list(self.queries), + "lifecycle": _json_object_copy(self.lifecycle), + "default_layout": _json_object_copy(self.default_layout), + "allow_multiple": self.allow_multiple, + "icon": self.icon, + "category": self.category, + } + + +BUILTIN_PANEL_DEFINITIONS = ( + PanelDefinition( + panel_type="samples", + label="Samples", + title="Samples", + source="builtin", + default_layout={"position": "right"}, + commands=[ + "workspace.panel.state.get", + "workspace.panel.state.patch", + "panel.samples.retrieval.set-anchor", + "panel.samples.retrieval.set-text-query", + "panel.samples.retrieval.set-k", + "panel.samples.retrieval.clear", + ], + queries=["samples.query", "samples.similar"], + icon="grid", + category="dataset", + ), + PanelDefinition( + panel_type="scatter", + label="Scatter", + title="Embeddings", + source="builtin", + default_layout={"position": "center"}, + commands=["workspace.panel.state.get", "workspace.panel.state.patch"], + queries=["embeddings", "layouts"], + icon="scatter", + category="embedding", + ), +) + + +def merge_default_props( + definition: PanelDefinition | None, + props: dict[str, Any] | None, +) -> dict[str, Any]: + merged = _json_object_copy(definition.default_props) if definition is not None else {} + merged.update(_json_object_copy(dict(props or {}))) + return merged diff --git a/src/hyperview/runtime.py b/src/hyperview/runtime.py index ed183db..381fb11 100644 --- a/src/hyperview/runtime.py +++ b/src/hyperview/runtime.py @@ -22,8 +22,13 @@ resolve_panel_source, unload_extension_modules, ) +from hyperview.panel_definitions import ( + BUILTIN_PANEL_DEFINITIONS, + PanelDefinition, + merge_default_props, +) from hyperview.storage.config import StorageConfig -from hyperview.storage.schema import parse_layout_dimension +from hyperview.storage.schema import parse_layout_dimension, space_key_from_index_ref from hyperview.tools import RunContext, ToolRegistry @@ -31,14 +36,99 @@ def _now_ts() -> int: return int(time.time()) -def _positive_int_or_none(value: int | None) -> int | None: +def _positive_int_or_none(value: Any) -> int | None: if value is None: return None parsed = int(value) return parsed if parsed > 0 else None +def _str_or_none(value: Any) -> str | None: + if value is None: + return None + text = str(value).strip() + return text or None + + +def _panel_layout_fields( + default_layout: dict[str, Any] | None, + *, + position: str | None, + reference_panel_id: str | None, + direction: str | None, + width: int | None, + height: int | None, + min_width: int | None, + min_height: int | None, + max_width: int | None, + max_height: int | None, + fallback_position: str = "right", +) -> dict[str, Any]: + layout = dict(default_layout or {}) + resolved_position = str(position or layout.get("position") or fallback_position) + if resolved_position not in {"center", "right", "bottom"}: + raise ValueError("position must be one of center, right, bottom") + resolved_direction = direction or _str_or_none(layout.get("direction")) + if resolved_direction is not None and resolved_direction not in { + "right", + "left", + "above", + "below", + "within", + }: + raise ValueError("direction must be one of right, left, above, below, within") + + def dimension(arg_value: int | None, layout_key: str) -> int | None: + return _positive_int_or_none( + arg_value if arg_value is not None else layout.get(layout_key) + ) + + return { + "position": resolved_position, + "reference_panel_id": reference_panel_id + if reference_panel_id is not None + else _str_or_none(layout.get("reference_panel_id")), + "direction": resolved_direction, + "width": dimension(width, "width"), + "height": dimension(height, "height"), + "min_width": dimension(min_width, "min_width"), + "min_height": dimension(min_height, "min_height"), + "max_width": dimension(max_width, "max_width"), + "max_height": dimension(max_height, "max_height"), + } + + _UNSET = object() +SAMPLES_PANEL_STATE_ID = "samples" +SAMPLES_PANEL_STATE_ALIASES = {SAMPLES_PANEL_STATE_ID, "grid"} +RESERVED_PANEL_STATE_IDS = {SAMPLES_PANEL_STATE_ID} + + +def _stable_collection_id(kind: str, query: dict[str, Any]) -> str: + payload = json.dumps(query, sort_keys=True, separators=(",", ":"), default=str) + digest = hashlib.sha1(payload.encode("utf-8")).hexdigest()[:12] + return f"{kind}:{digest}" + + +def _json_object_copy(value: dict[str, Any]) -> dict[str, Any]: + """Return a JSON-safe copy and reject non-serializable state early.""" + + return json.loads(json.dumps(value)) + + +def _json_merge_patch(current: dict[str, Any], patch: dict[str, Any]) -> dict[str, Any]: + """Apply RFC 7396-style merge patch semantics to panel state.""" + + result = _json_object_copy(current) + for key, value in patch.items(): + if value is None: + result.pop(key, None) + continue + if isinstance(value, dict) and isinstance(result.get(key), dict): + result[key] = _json_merge_patch(result[key], value) + continue + result[key] = json.loads(json.dumps(value)) + return result def get_runtime_config_dir() -> Path: @@ -210,7 +300,9 @@ class CustomPanelSpec: title: str module_file: str | None = None kind: Literal["module", "scatter", "builtin"] = "module" - builtin_panel: Literal["samples"] | None = None + panel_type: str | None = None + source: str | None = None + builtin_panel: str | None = None extension: str | None = None extension_panel: str | None = None position: Literal["center", "right", "bottom"] = "right" @@ -236,9 +328,7 @@ def from_dict(cls, data: dict[str, Any]) -> CustomPanelSpec: builtin_panel = data.get("builtin_panel") if builtin_panel is not None: - builtin_panel = str(builtin_panel) - if builtin_panel != "samples": - builtin_panel = None + builtin_panel = str(builtin_panel).strip() or None position = str(data.get("position") or "right") if position not in {"center", "right", "bottom"}: @@ -272,6 +362,8 @@ def positive_int(key: str) -> int | None: title=str(data["title"]), module_file=data.get("module_file"), kind=kind, # type: ignore[arg-type] + panel_type=data.get("panel_type"), + source=data.get("source"), builtin_panel=builtin_panel, # type: ignore[arg-type] extension=data.get("extension"), extension_panel=data.get("extension_panel"), @@ -292,7 +384,44 @@ def positive_int(key: str) -> int | None: ) def to_dict(self) -> dict[str, Any]: - return asdict(self) + payload = asdict(self) + payload["panel_type"] = self.resolved_panel_type() + payload["source"] = self.resolved_source() + payload["layout"] = self.layout_dict() + return payload + + def resolved_panel_type(self) -> str: + if self.panel_type: + return self.panel_type + if self.kind == "builtin": + return self.builtin_panel or "builtin" + if self.kind == "scatter": + return "scatter" + if self.extension and self.extension_panel: + return f"{self.extension}.{self.extension_panel}" + return "module" + + def resolved_source(self) -> str: + if self.source: + return self.source + if self.kind in {"builtin", "scatter"}: + return "builtin" + if self.extension: + return "extension" + return "module" + + def layout_dict(self) -> dict[str, Any]: + return { + "position": self.position, + "reference_panel_id": self.reference_panel_id, + "direction": self.direction, + "width": self.width, + "height": self.height, + "min_width": self.min_width, + "min_height": self.min_height, + "max_width": self.max_width, + "max_height": self.max_height, + } def resolved_module_file(self) -> Path | None: if not self.module_file: @@ -353,7 +482,8 @@ def to_dict(self) -> dict[str, Any]: @dataclass class SimilarityQueryState: - anchor_sample_id: str + anchor_sample_id: str | None = None + query_text: str | None = None layout_key: str | None = None space_key: str | None = None k: int = 18 @@ -361,8 +491,9 @@ class SimilarityQueryState: @classmethod def from_dict(cls, data: dict[str, Any]) -> SimilarityQueryState | None: - anchor_sample_id = str(data.get("anchor_sample_id") or "").strip() - if not anchor_sample_id: + anchor_sample_id = str(data.get("anchor_sample_id") or "").strip() or None + query_text = str(data.get("query_text") or "").strip() or None + if not anchor_sample_id and not query_text: return None try: k = int(data.get("k") or 18) @@ -370,28 +501,215 @@ def from_dict(cls, data: dict[str, Any]) -> SimilarityQueryState | None: k = 18 return cls( anchor_sample_id=anchor_sample_id, + query_text=query_text, layout_key=data.get("layout_key"), - space_key=data.get("space_key"), + space_key=data.get("space_key") + or space_key_from_index_ref(data.get("index_id")), k=max(1, min(k, 100)), source=data.get("source"), ) def to_dict(self) -> dict[str, Any]: - return { - "anchor_sample_id": self.anchor_sample_id, + payload: dict[str, Any] = { "layout_key": self.layout_key, "space_key": self.space_key, "k": self.k, "source": self.source, } + if self.anchor_sample_id: + payload["anchor_sample_id"] = self.anchor_sample_id + if self.query_text: + payload["query_text"] = self.query_text + return payload + + +@dataclass +class EntityRef: + dataset_id: str + entity_set_id: str + entity_id: str + + @classmethod + def from_dict(cls, data: dict[str, Any]) -> EntityRef: + return cls( + dataset_id=str(data.get("datasetId") or data.get("dataset_id") or ""), + entity_set_id=str(data.get("entitySetId") or data.get("entity_set_id") or "samples"), + entity_id=str(data.get("entityId") or data.get("entity_id") or ""), + ) + + def to_dict(self) -> dict[str, Any]: + return { + "datasetId": self.dataset_id, + "entitySetId": self.entity_set_id, + "entityId": self.entity_id, + } + + +@dataclass +class CollectionState: + id: str + dataset_id: str + entity_set_id: str = "samples" + kind: Literal[ + "all", + "filter", + "selection", + "neighbors", + "lasso", + "search", + "tool_result", + "extension", + ] = "all" + query: dict[str, Any] = field(default_factory=dict) + scores: dict[str, float] | None = None + created_at: int = field(default_factory=_now_ts) + + @classmethod + def from_dict(cls, data: dict[str, Any]) -> CollectionState: + kind = str(data.get("kind") or "all") + if kind not in { + "all", + "filter", + "selection", + "neighbors", + "lasso", + "search", + "tool_result", + "extension", + }: + kind = "extension" + + scores: dict[str, float] | None = None + raw_scores = data.get("scores") + if isinstance(raw_scores, dict): + scores = {} + for key, value in raw_scores.items(): + try: + scores[str(key)] = float(value) + except (TypeError, ValueError): + continue + + return cls( + id=str(data["id"]), + dataset_id=str(data.get("dataset_id") or data.get("datasetId") or ""), + entity_set_id=str(data.get("entity_set_id") or data.get("entitySetId") or "samples"), + kind=kind, # type: ignore[arg-type] + query=dict(data.get("query") or {}), + scores=scores, + created_at=int(data.get("created_at") or data.get("createdAt") or _now_ts()), + ) + + def to_dict(self) -> dict[str, Any]: + return { + "id": self.id, + "dataset_id": self.dataset_id, + "entity_set_id": self.entity_set_id, + "kind": self.kind, + "query": _json_object_copy(self.query), + "scores": dict(self.scores) if self.scores is not None else None, + "created_at": self.created_at, + } + + +@dataclass +class PanelStateEntry: + state: dict[str, Any] = field(default_factory=dict) + state_revision: int = 0 + + @classmethod + def from_dict(cls, data: dict[str, Any]) -> PanelStateEntry: + raw_state = data.get("state") if isinstance(data, dict) else None + state = raw_state if isinstance(raw_state, dict) else {} + try: + state_revision = int(data.get("state_revision") or 0) + except (TypeError, ValueError): + state_revision = 0 + return cls( + state=_json_object_copy(state), + state_revision=max(0, state_revision), + ) + + def to_dict(self) -> dict[str, Any]: + return { + "state": _json_object_copy(self.state), + "state_revision": self.state_revision, + } + + +def _samples_collection_state(collection: CollectionState) -> dict[str, Any]: + return { + "collection_id": collection.id, + "collection": collection.to_dict(), + } + + +def _samples_filter_state(collection: CollectionState) -> dict[str, Any]: + return { + "mode": "collection", + "retrieval": None, + **_samples_collection_state(collection), + } + + +def _samples_retrieval_state( + query: SimilarityQueryState, + collection: CollectionState | None = None, +) -> dict[str, Any]: + state: dict[str, Any] = { + "mode": "retrieval", + "retrieval": query.to_dict(), + } + if collection is not None: + state.update(_samples_collection_state(collection)) + return state + + +def _clear_samples_collection_state_patch() -> dict[str, Any]: + return { + "collection_id": None, + "collection": None, + } + + +def _samples_panel_collection_kind(state: dict[str, Any]) -> str | None: + collection = state.get("collection") + if not isinstance(collection, dict): + return None + kind = collection.get("kind") + return kind if isinstance(kind, str) else None + + +def _samples_panel_retrieval_query( + panels: dict[str, PanelStateEntry], +) -> SimilarityQueryState | None: + state_entry = panels.get(SAMPLES_PANEL_STATE_ID) + retrieval = state_entry.state.get("retrieval") if state_entry is not None else None + if not isinstance(retrieval, dict): + return None + return SimilarityQueryState.from_dict(retrieval) + + +def _custom_panel_instance_payload( + panel: CustomPanelSpec, + panel_states: dict[str, PanelStateEntry], + *, + data: dict[str, Any] | None = None, +) -> dict[str, Any]: + payload = { + **panel.to_dict(), + **panel_states.get(panel.id, PanelStateEntry()).to_dict(), + } + if data is not None: + payload["data"] = data + return payload @dataclass class WorkspaceUiState: active_layout_key: str | None = None selected_ids: list[str] = field(default_factory=list) - similarity_query: SimilarityQueryState | None = None custom_panels: list[CustomPanelSpec] = field(default_factory=list) + panels: dict[str, PanelStateEntry] = field(default_factory=dict) has_explicit_view: bool = False active_panel_id: str | None = None layout_views: dict[str, LayoutViewState] = field(default_factory=dict) @@ -412,16 +730,40 @@ def from_dict(cls, data: dict[str, Any]) -> WorkspaceUiState: if isinstance(layout_key, str) and isinstance(view_data, dict): layout_views[layout_key] = LayoutViewState.from_dict(view_data) - similarity_query = SimilarityQueryState.from_dict(data.get("similarity_query") or {}) + panels: dict[str, PanelStateEntry] = {} + raw_panels = data.get("panels") or {} + if isinstance(raw_panels, dict): + for panel_id, panel_data in raw_panels.items(): + if isinstance(panel_id, str) and isinstance(panel_data, dict): + panels[panel_id] = PanelStateEntry.from_dict(panel_data) + + for entry in list(data.get("custom_panels") or []): + if not isinstance(entry, dict) or "state" not in entry: + continue + panel_id = str(entry.get("id") or "").strip() + if not panel_id or panel_id in panels: + continue + panels[panel_id] = PanelStateEntry.from_dict(entry) + + # One-way migration reader for workspaces persisted before July 2026. + # Runtime state now lives in ui.panels["samples"].state["retrieval"]. + legacy_similarity_query = SimilarityQueryState.from_dict(data.get("similarity_query") or {}) selected_ids = list(data.get("selected_ids") or []) - if similarity_query is not None: + active_retrieval = _samples_panel_retrieval_query(panels) + if active_retrieval is None and legacy_similarity_query is not None: + selected_ids = [] + panels.setdefault( + SAMPLES_PANEL_STATE_ID, + PanelStateEntry(state=_samples_retrieval_state(legacy_similarity_query)), + ) + elif active_retrieval is not None: selected_ids = [] return cls( active_layout_key=data.get("active_layout_key"), selected_ids=selected_ids, - similarity_query=similarity_query, custom_panels=custom_panels, + panels=panels, has_explicit_view=bool(data.get("has_explicit_view", False)), active_panel_id=data.get("active_panel_id"), layout_views=layout_views, @@ -432,10 +774,13 @@ def to_dict(self) -> dict[str, Any]: return { "active_layout_key": self.active_layout_key, "selected_ids": list(self.selected_ids), - "similarity_query": ( - self.similarity_query.to_dict() if self.similarity_query is not None else None - ), - "custom_panels": [panel.to_dict() for panel in self.custom_panels], + "custom_panels": [ + _custom_panel_instance_payload(panel, self.panels) + for panel in self.custom_panels + ], + "panels": { + panel_id: state.to_dict() for panel_id, state in sorted(self.panels.items()) + }, "has_explicit_view": self.has_explicit_view, "active_panel_id": self.active_panel_id, "layout_views": { @@ -449,14 +794,26 @@ def to_dict(self) -> dict[str, Any]: class WorkspaceState: id: str dataset_name: str | None = None + collections: dict[str, CollectionState] = field(default_factory=dict) ui: WorkspaceUiState = field(default_factory=WorkspaceUiState) created_at: int = field(default_factory=_now_ts) @classmethod def from_dict(cls, data: dict[str, Any]) -> WorkspaceState: + collections: dict[str, CollectionState] = {} + raw_collections = data.get("collections") or [] + if isinstance(raw_collections, dict): + raw_collections = raw_collections.values() + for entry in list(raw_collections): + if not isinstance(entry, dict) or "id" not in entry: + continue + collection = CollectionState.from_dict(entry) + collections[collection.id] = collection + return cls( id=str(data["id"]), dataset_name=data.get("dataset_name"), + collections=collections, ui=WorkspaceUiState.from_dict(data.get("ui") or {}), created_at=int(data.get("created_at") or _now_ts()), ) @@ -465,6 +822,10 @@ def to_dict(self) -> dict[str, Any]: return { "id": self.id, "dataset_name": self.dataset_name, + "collections": [ + collection.to_dict() + for collection in sorted(self.collections.values(), key=lambda item: item.id) + ], "ui": self.ui.to_dict(), "created_at": self.created_at, } @@ -607,12 +968,7 @@ def to_dict(self) -> dict[str, Any]: "workspace_id": self.workspace_id, "panels": list(self.panel_ids), "panel_definitions": [ - { - "id": panel.id, - "title": panel.title, - "position": panel.position, - "file": panel.file, - } + panel.to_definition(self.manifest.name).to_dict() for panel in self.manifest.panels ], "tools": [record.to_dict() for record in self.loaded.tools], @@ -712,8 +1068,9 @@ def attach_dataset_instance( if previous_dataset_name != dataset.name: workspace.ui.active_layout_key = None workspace.ui.selected_ids = [] - workspace.ui.similarity_query = None + workspace.ui.panels = {} workspace.ui.layout_views = {} + workspace.collections = {} self.workspace_registry.update_workspace(workspace) if activate_workspace: self.workspace_registry.set_active_workspace(workspace_id) @@ -745,8 +1102,9 @@ def set_workspace_dataset( if previous_dataset_name != dataset_name: workspace.ui.active_layout_key = None workspace.ui.selected_ids = [] - workspace.ui.similarity_query = None + workspace.ui.panels = {} workspace.ui.layout_views = {} + workspace.collections = {} self.workspace_registry.update_workspace(workspace) self._bump_version() return workspace @@ -776,6 +1134,316 @@ def get_dataset( self._dataset_cache[resolved_dataset_name] = dataset return dataset + def _resolve_panel_state_id_locked(self, workspace: WorkspaceState, panel_id: str) -> str: + requested_panel_id = panel_id.strip() + if not requested_panel_id: + raise ValueError("panel_id must be a non-empty string") + + custom_panel_ids = {panel.id for panel in workspace.ui.custom_panels} + if requested_panel_id in custom_panel_ids: + return requested_panel_id + if requested_panel_id in SAMPLES_PANEL_STATE_ALIASES: + return SAMPLES_PANEL_STATE_ID + raise KeyError(f"Panel not found: {panel_id}") + + def _get_panel_state_entry_locked( + self, + workspace: WorkspaceState, + panel_id: str, + *, + create: bool = False, + ) -> tuple[str, PanelStateEntry]: + resolved_panel_id = self._resolve_panel_state_id_locked(workspace, panel_id) + entry = workspace.ui.panels.get(resolved_panel_id) + if entry is None: + if not create: + return resolved_panel_id, PanelStateEntry() + entry = PanelStateEntry() + workspace.ui.panels[resolved_panel_id] = entry + return resolved_panel_id, entry + + def _prune_panel_states_locked(self, workspace: WorkspaceState) -> None: + retained_panel_ids = {panel.id for panel in workspace.ui.custom_panels} + retained_panel_ids.update(RESERVED_PANEL_STATE_IDS) + workspace.ui.panels = { + panel_id: state + for panel_id, state in workspace.ui.panels.items() + if panel_id in retained_panel_ids + } + + def _seed_default_panel_state_locked( + self, + workspace: WorkspaceState, + panel: CustomPanelSpec, + ) -> None: + definition = self._definition_for_panel_spec_locked(panel) + if ( + definition is not None + and definition.default_state + and panel.id not in workspace.ui.panels + ): + workspace.ui.panels[panel.id] = PanelStateEntry( + state=_json_object_copy(definition.default_state) + ) + + def get_panel_state( + self, + workspace_id: str, + panel_id: str, + ) -> dict[str, Any]: + with self._lock: + workspace = self.get_workspace(workspace_id) + resolved_panel_id, entry = self._get_panel_state_entry_locked( + workspace, + panel_id, + create=False, + ) + return { + "panel_id": resolved_panel_id, + **entry.to_dict(), + } + + def patch_panel_state( + self, + workspace_id: str, + panel_id: str, + patch: dict[str, Any], + *, + replace_state: bool = False, + expected_revision: int | None = None, + source_client_id: str | None = None, + ) -> WorkspaceState: + if not isinstance(patch, dict): + raise ValueError("panel state patch must be a JSON object") + + with self._lock: + workspace = self.get_workspace(workspace_id) + resolved_panel_id, entry = self._get_panel_state_entry_locked( + workspace, + panel_id, + create=True, + ) + if expected_revision is not None and entry.state_revision != expected_revision: + raise ValueError( + "panel state revision conflict: " + f"expected {expected_revision}, got {entry.state_revision}" + ) + + next_state = ( + _json_object_copy(patch) + if replace_state + else _json_merge_patch(entry.state, patch) + ) + if next_state == entry.state: + return workspace + + workspace.ui.panels[resolved_panel_id] = PanelStateEntry( + state=next_state, + state_revision=entry.state_revision + 1, + ) + self.workspace_registry.update_workspace(workspace) + self._bump_version(source_client_id=source_client_id) + return workspace + + def _workspace_dataset_id_locked(self, workspace: WorkspaceState) -> str: + if not workspace.dataset_name: + raise ValueError(f"Workspace '{workspace.id}' has no active dataset") + return workspace.dataset_name + + def _store_collection_locked( + self, + workspace: WorkspaceState, + collection: CollectionState, + ) -> CollectionState: + workspace.collections[collection.id] = collection + return collection + + def _build_label_filter_collection_locked( + self, + workspace: WorkspaceState, + *, + field: str, + value: Any, + source: str | None = None, + ) -> CollectionState: + dataset_id = self._workspace_dataset_id_locked(workspace) + field = field.strip() + if not field: + raise ValueError("field must be a non-empty string") + query = { + "field": field, + "op": "eq", + "value": value, + } + if source: + query["source"] = source + collection = CollectionState( + id=_stable_collection_id("filter", query), + dataset_id=dataset_id, + entity_set_id="samples", + kind="filter", + query=query, + ) + return self._store_collection_locked(workspace, collection) + + def _build_neighbors_collection_locked( + self, + workspace: WorkspaceState, + query: SimilarityQueryState, + ) -> CollectionState: + dataset_id = self._workspace_dataset_id_locked(workspace) + if query.query_text: + collection_query: dict[str, Any] = { + "queryText": query.query_text, + "indexId": f"space:{query.space_key}" if query.space_key else None, + "layoutId": query.layout_key, + "spaceKey": query.space_key, + "k": query.k, + } + if query.source: + collection_query["source"] = query.source + collection = CollectionState( + id=_stable_collection_id("search", collection_query), + dataset_id=dataset_id, + entity_set_id="samples", + kind="search", + query=collection_query, + ) + return self._store_collection_locked(workspace, collection) + + if not query.anchor_sample_id: + raise ValueError("Similarity retrieval requires anchor_sample_id or query_text") + + anchor = EntityRef( + dataset_id=dataset_id, + entity_set_id="samples", + entity_id=query.anchor_sample_id, + ) + collection_query = { + "anchor": anchor.to_dict(), + "indexId": f"space:{query.space_key}" if query.space_key else None, + "layoutId": query.layout_key, + "spaceKey": query.space_key, + "k": query.k, + } + if query.source: + collection_query["source"] = query.source + collection = CollectionState( + id=_stable_collection_id("neighbors", collection_query), + dataset_id=dataset_id, + entity_set_id="samples", + kind="neighbors", + query=collection_query, + ) + return self._store_collection_locked(workspace, collection) + + def _set_samples_filter_locked( + self, + workspace: WorkspaceState, + collection: CollectionState | None, + ) -> bool: + resolved_panel_id, entry = self._get_panel_state_entry_locked( + workspace, + SAMPLES_PANEL_STATE_ID, + create=collection is not None, + ) + if collection is None: + if ( + entry.state.get("mode") != "collection" + or _samples_panel_collection_kind(entry.state) != "filter" + ): + return False + next_state = _json_merge_patch( + entry.state, + { + "mode": None, + **_clear_samples_collection_state_patch(), + }, + ) + else: + next_state = _json_merge_patch(entry.state, _samples_filter_state(collection)) + next_state["collection"] = collection.to_dict() + next_state["collection_id"] = collection.id + + if next_state == entry.state: + return False + + workspace.ui.panels[resolved_panel_id] = PanelStateEntry( + state=next_state, + state_revision=entry.state_revision + 1, + ) + return True + + def _set_samples_retrieval_locked( + self, + workspace: WorkspaceState, + query: SimilarityQueryState | None, + ) -> bool: + resolved_panel_id, entry = self._get_panel_state_entry_locked( + workspace, + SAMPLES_PANEL_STATE_ID, + create=query is not None, + ) + if query is None: + should_clear_collection = ( + entry.state.get("mode") == "retrieval" + or _samples_panel_collection_kind(entry.state) in {"neighbors", "search"} + ) + clear_patch: dict[str, Any] = {"retrieval": None} + if should_clear_collection: + clear_patch.update(_clear_samples_collection_state_patch()) + next_state = _json_merge_patch( + entry.state, + clear_patch, + ) + if next_state.get("mode") == "retrieval": + next_state.pop("mode", None) + else: + collection = self._build_neighbors_collection_locked(workspace, query) + next_state = _json_object_copy(entry.state) + next_state.update(_samples_retrieval_state(query, collection)) + next_state["collection"] = collection.to_dict() + next_state["collection_id"] = collection.id + + if next_state == entry.state: + return False + + workspace.ui.panels[resolved_panel_id] = PanelStateEntry( + state=next_state, + state_revision=entry.state_revision + 1, + ) + return True + + def set_samples_filter( + self, + workspace_id: str, + *, + field: str = "label", + value: Any, + source: str | None = None, + ) -> WorkspaceState: + with self._lock: + workspace = self.get_workspace(workspace_id) + collection = self._build_label_filter_collection_locked( + workspace, + field=field, + value=value, + source=source, + ) + workspace.ui.selected_ids = [] + self._set_samples_filter_locked(workspace, collection) + self.workspace_registry.update_workspace(workspace) + self._bump_version() + return workspace + + def clear_samples_filter(self, workspace_id: str) -> WorkspaceState: + with self._lock: + workspace = self.get_workspace(workspace_id) + self._set_samples_filter_locked(workspace, None) + self.workspace_registry.update_workspace(workspace) + self._bump_version() + return workspace + def set_active_layout(self, workspace_id: str, layout_key: str | None) -> WorkspaceState: with self._lock: workspace = self.get_workspace(workspace_id) @@ -788,29 +1456,43 @@ def set_selection(self, workspace_id: str, sample_ids: list[str]) -> WorkspaceSt with self._lock: workspace = self.get_workspace(workspace_id) workspace.ui.selected_ids = list(dict.fromkeys(sample_ids)) - if ( - workspace.ui.similarity_query is not None - and workspace.ui.similarity_query.anchor_sample_id not in workspace.ui.selected_ids - ): - workspace.ui.similarity_query = None + active_retrieval = _samples_panel_retrieval_query(workspace.ui.panels) + if active_retrieval is not None and active_retrieval.query_text is None: + if active_retrieval.anchor_sample_id not in workspace.ui.selected_ids: + self._set_samples_retrieval_locked(workspace, None) self.workspace_registry.update_workspace(workspace) self._bump_version() return workspace - def set_similarity_query( + def get_samples_retrieval_query( + self, + workspace_id: str, + ) -> SimilarityQueryState | None: + with self._lock: + workspace = self.get_workspace(workspace_id) + return _samples_panel_retrieval_query(workspace.ui.panels) + + def set_samples_retrieval( self, workspace_id: str, query: SimilarityQueryState | None, ) -> WorkspaceState: with self._lock: workspace = self.get_workspace(workspace_id) - workspace.ui.similarity_query = query + changed = False if query is not None: - workspace.ui.selected_ids = [] - self.workspace_registry.update_workspace(workspace) - self._bump_version() + if workspace.ui.selected_ids: + workspace.ui.selected_ids = [] + changed = True + changed = self._set_samples_retrieval_locked(workspace, query) or changed + if changed: + self.workspace_registry.update_workspace(workspace) + self._bump_version() return workspace + def clear_samples_retrieval(self, workspace_id: str) -> WorkspaceState: + return self.set_samples_retrieval(workspace_id, None) + def patch_ui_state( self, workspace_id: str, @@ -819,8 +1501,6 @@ def patch_ui_state( active_layout_key: str | None = None, set_selection: bool = False, selected_ids: list[str] | None = None, - set_similarity_query: bool = False, - similarity_query: SimilarityQueryState | None = None, source_client_id: str | None = None, ) -> WorkspaceState: """Apply multiple UI-state updates under one runtime version bump.""" @@ -841,21 +1521,15 @@ def patch_ui_state( if workspace.ui.selected_ids != next_selected_ids: workspace.ui.selected_ids = next_selected_ids changed = True + active_retrieval = _samples_panel_retrieval_query(workspace.ui.panels) if ( - workspace.ui.similarity_query is not None - and workspace.ui.similarity_query.anchor_sample_id + active_retrieval is not None + and active_retrieval.anchor_sample_id not in workspace.ui.selected_ids ): - workspace.ui.similarity_query = None + self._set_samples_retrieval_locked(workspace, None) changed = True - if set_similarity_query and workspace.ui.similarity_query != similarity_query: - workspace.ui.similarity_query = similarity_query - changed = True - if set_similarity_query and similarity_query is not None and workspace.ui.selected_ids: - workspace.ui.selected_ids = [] - changed = True - if changed: self.workspace_registry.update_workspace(workspace) self._bump_version(source_client_id=source_client_id) @@ -877,8 +1551,69 @@ def resolve_similarity_query( except KeyError as exc: raise KeyError(f"Sample not found: {sample_id}") from exc + resolved_layout_key, resolved_space_key = self._resolve_retrieval_context( + workspace_id=workspace_id, + layout_key=layout_key, + space_key=space_key, + ) + + try: + limit = int(k) + except (TypeError, ValueError): + limit = 18 + + return SimilarityQueryState( + anchor_sample_id=sample_id, + layout_key=resolved_layout_key, + space_key=resolved_space_key, + k=max(1, min(limit, 100)), + source=source, + ) + + def resolve_text_retrieval_query( + self, + workspace_id: str, + query_text: str, + *, + layout_key: str | None = None, + space_key: str | None = None, + k: int = 18, + source: str | None = None, + ) -> SimilarityQueryState: + text = str(query_text or "").strip() + if not text: + raise ValueError("query_text must be a non-empty string") + + resolved_layout_key, resolved_space_key = self._resolve_retrieval_context( + workspace_id=workspace_id, + layout_key=layout_key, + space_key=space_key, + ) + + try: + limit = int(k) + except (TypeError, ValueError): + limit = 18 + + return SimilarityQueryState( + query_text=text, + layout_key=resolved_layout_key, + space_key=resolved_space_key, + k=max(1, min(limit, 100)), + source=source, + ) + + def _resolve_retrieval_context( + self, + *, + workspace_id: str, + layout_key: str | None, + space_key: str | None, + ) -> tuple[str | None, str | None]: resolved_space_key = space_key + resolved_layout_key = layout_key if layout_key is not None: + dataset = self.get_dataset(workspace_id=workspace_id) layout = next( (item for item in dataset.list_layouts() if item.layout_key == layout_key), None, @@ -890,25 +1625,34 @@ def resolve_similarity_query( resolved_space_key = layout.space_key if resolved_space_key is not None: + dataset = self.get_dataset(workspace_id=workspace_id) space = next( (item for item in dataset.list_spaces() if item.space_key == resolved_space_key), None, ) if space is None: raise LookupError(f"Space not found: {resolved_space_key}") + else: + workspace = self.get_workspace(workspace_id) + dataset = self.get_dataset(workspace_id=workspace_id) + if workspace.ui.active_layout_key: + active_layout = next( + ( + item + for item in dataset.list_layouts() + if item.layout_key == workspace.ui.active_layout_key + ), + None, + ) + if active_layout is not None: + resolved_layout_key = active_layout.layout_key + resolved_space_key = active_layout.space_key + if resolved_space_key is None: + spaces = dataset.list_spaces() + if spaces: + resolved_space_key = spaces[0].space_key - try: - limit = int(k) - except (TypeError, ValueError): - limit = 18 - - return SimilarityQueryState( - anchor_sample_id=sample_id, - layout_key=layout_key, - space_key=resolved_space_key, - k=max(1, min(limit, 100)), - source=source, - ) + return resolved_layout_key, resolved_space_key def set_layout_view( self, @@ -934,11 +1678,11 @@ def build_custom_panel( panel_id: str, kind: Literal["extension", "scatter", "module", "builtin"], title: str | None = None, - builtin_panel: Literal["samples"] | None = None, + builtin_panel: str | None = None, extension: str | None = None, extension_panel: str | None = None, layout_key: str | None = None, - position: str = "right", + position: str | None = None, reference_panel_id: str | None = None, direction: str | None = None, width: int | None = None, @@ -955,17 +1699,6 @@ def build_custom_panel( ) -> CustomPanelSpec: """Resolve a transport-level panel request into a runtime panel spec.""" - if position not in {"center", "right", "bottom"}: - raise ValueError("position must be one of center, right, bottom") - if direction is not None and direction not in { - "right", - "left", - "above", - "below", - "within", - }: - raise ValueError("direction must be one of right, left, above, below, within") - if kind == "module": raise ValueError( "Direct module panels are no longer a public API. " @@ -973,24 +1706,32 @@ def build_custom_panel( ) if kind == "builtin": - if builtin_panel != "samples": - raise ValueError("builtin_panel must be 'samples'") + builtin_panel_type = str(builtin_panel or "").strip() + definition = self.get_panel_definition(builtin_panel_type, source="builtin") + if definition is None: + raise ValueError(f"Unknown built-in panel type: {builtin_panel_type}") + layout = _panel_layout_fields( + definition.default_layout, + position=position, + reference_panel_id=reference_panel_id, + direction=direction, + width=width, + height=height, + min_width=min_width, + min_height=min_height, + max_width=max_width, + max_height=max_height, + ) return CustomPanelSpec( id=panel_id, - title=title or "Samples", + title=title or definition.title or definition.label, kind="builtin", - builtin_panel="samples", - position=position, # type: ignore[arg-type] - reference_panel_id=reference_panel_id, - direction=direction, # type: ignore[arg-type] - width=_positive_int_or_none(width), - height=_positive_int_or_none(height), - min_width=_positive_int_or_none(min_width), - min_height=_positive_int_or_none(min_height), - max_width=_positive_int_or_none(max_width), - max_height=_positive_int_or_none(max_height), + panel_type=definition.panel_type, + source=definition.source, + builtin_panel=definition.panel_type, + **layout, visible=visible, - props=dict(props or {}), + props=merge_default_props(definition, props), ) if kind == "extension": @@ -1011,24 +1752,31 @@ def build_custom_panel( installation.manifest.folder, manifest_panel.file, ) + definition = manifest_panel.to_definition(installation.manifest.name) + layout = _panel_layout_fields( + definition.default_layout, + position=position, + reference_panel_id=reference_panel_id, + direction=direction, + width=width, + height=height, + min_width=min_width, + min_height=min_height, + max_width=max_width, + max_height=max_height, + ) return CustomPanelSpec( id=panel_id, - title=title or manifest_panel.title, + title=title or definition.title or definition.label, kind="module", + panel_type=definition.panel_type, + source=definition.source, extension=extension, extension_panel=extension_panel, module_file=str(module_file), - position=position, # type: ignore[arg-type] - reference_panel_id=reference_panel_id, - direction=direction, # type: ignore[arg-type] - width=_positive_int_or_none(width), - height=_positive_int_or_none(height), - min_width=_positive_int_or_none(min_width), - min_height=_positive_int_or_none(min_height), - max_width=_positive_int_or_none(max_width), - max_height=_positive_int_or_none(max_height), + **layout, visible=visible, - props=dict(props or {}), + props=merge_default_props(definition, props), ) if kind == "scatter": @@ -1063,24 +1811,33 @@ def build_custom_panel( ) if not title: raise ValueError("title is required for scatter panels") + layout = _panel_layout_fields( + None, + position=position, + reference_panel_id=reference_panel_id, + direction=direction, + width=width, + height=height, + min_width=min_width, + min_height=min_height, + max_width=max_width, + max_height=max_height, + ) return CustomPanelSpec( id=panel_id, title=title, kind="scatter", - position=position, # type: ignore[arg-type] + panel_type="scatter", + source="builtin", layout_key=layout_key, geometry=geometry, layout_dimension=layout_dimension, - reference_panel_id=reference_panel_id, - direction=direction, # type: ignore[arg-type] - width=_positive_int_or_none(width), - height=_positive_int_or_none(height), - min_width=_positive_int_or_none(min_width), - min_height=_positive_int_or_none(min_height), - max_width=_positive_int_or_none(max_width), - max_height=_positive_int_or_none(max_height), + **layout, visible=visible, - props=dict(props or {}), + props=merge_default_props( + self.get_panel_definition("scatter", source="builtin"), + props, + ), ) raise ValueError(f"Unsupported panel kind: {kind}") @@ -1092,11 +1849,11 @@ def add_runtime_panel( panel_id: str, kind: Literal["extension", "scatter", "module", "builtin"], title: str | None = None, - builtin_panel: Literal["samples"] | None = None, + builtin_panel: str | None = None, extension: str | None = None, extension_panel: str | None = None, layout_key: str | None = None, - position: str = "right", + position: str | None = None, reference_panel_id: str | None = None, direction: str | None = None, width: int | None = None, @@ -1145,6 +1902,7 @@ def add_custom_panel(self, workspace_id: str, panel: CustomPanelSpec) -> Workspa ] panels.append(panel) workspace.ui.custom_panels = panels + self._seed_default_panel_state_locked(workspace, panel) workspace.ui.view_revision += 1 self.workspace_registry.update_workspace(workspace) self._bump_version() @@ -1287,6 +2045,9 @@ def replace_custom_panels( workspace.ui.custom_panels = next_panels workspace.ui.has_explicit_view = next_has_explicit_view workspace.ui.active_panel_id = next_active_panel_id + self._prune_panel_states_locked(workspace) + for panel in next_panels: + self._seed_default_panel_state_locked(workspace, panel) if bump_view_revision: workspace.ui.view_revision += 1 self.workspace_registry.update_workspace(workspace) @@ -1299,6 +2060,7 @@ def remove_custom_panel(self, workspace_id: str, panel_id: str) -> WorkspaceStat workspace.ui.custom_panels = [ panel for panel in workspace.ui.custom_panels if panel.id != panel_id ] + workspace.ui.panels.pop(panel_id, None) if workspace.ui.active_panel_id == panel_id: workspace.ui.active_panel_id = None workspace.ui.view_revision += 1 @@ -1510,6 +2272,82 @@ def get_extension(self, name: str) -> ExtensionInstallation | None: with self._lock: return self._extensions.get(name) + def list_panel_definitions(self) -> list[PanelDefinition]: + with self._lock: + definitions = list(BUILTIN_PANEL_DEFINITIONS) + for installation in self._extensions.values(): + definitions.extend( + panel.to_definition(installation.manifest.name) + for panel in installation.manifest.panels + ) + return sorted( + definitions, + key=lambda definition: (definition.source, definition.panel_type), + ) + + def get_panel_definition( + self, + panel_type: str, + *, + source: str | None = None, + extension: str | None = None, + ) -> PanelDefinition | None: + with self._lock: + return self._get_panel_definition_locked( + panel_type, + source=source, + extension=extension, + ) + + def _get_panel_definition_locked( + self, + panel_type: str, + *, + source: str | None = None, + extension: str | None = None, + ) -> PanelDefinition | None: + definitions = list(BUILTIN_PANEL_DEFINITIONS) + for installation in self._extensions.values(): + definitions.extend( + panel.to_definition(installation.manifest.name) + for panel in installation.manifest.panels + ) + for definition in definitions: + if definition.panel_type != panel_type: + continue + if source is not None and definition.source != source: + continue + if extension is not None and definition.extension != extension: + continue + return definition + return None + + def _definition_for_panel_spec_locked( + self, + panel: CustomPanelSpec, + ) -> PanelDefinition | None: + if panel.kind in {"builtin", "scatter"}: + return self._get_panel_definition_locked( + panel.resolved_panel_type(), + source="builtin", + ) + if panel.extension and panel.extension_panel: + installation = self._extensions.get(panel.extension) + if installation is None: + return None + manifest_panel = next( + ( + item + for item in installation.manifest.panels + if item.id == panel.extension_panel + ), + None, + ) + if manifest_panel is None: + return None + return manifest_panel.to_definition(installation.manifest.name) + return None + def install_extension( self, workspace_id: str, @@ -1524,14 +2362,30 @@ def install_extension( prepared_panels: list[CustomPanelSpec] = [] for panel_entry in manifest.panels: panel_file = resolve_panel_source(manifest.folder, panel_entry.file) + definition = panel_entry.to_definition(manifest.name) + layout = _panel_layout_fields( + definition.default_layout, + position=None, + reference_panel_id=None, + direction=None, + width=None, + height=None, + min_width=None, + min_height=None, + max_width=None, + max_height=None, + ) prepared_panels.append( CustomPanelSpec( id=panel_entry.id, - title=panel_entry.title, + title=definition.title or definition.label, + panel_type=definition.panel_type, + source=definition.source, extension=manifest.name, extension_panel=panel_entry.id, module_file=str(panel_file), - position=panel_entry.position, # type: ignore[arg-type] + **layout, + props=merge_default_props(definition, None), ) ) @@ -1569,6 +2423,11 @@ def install_extension( add_panels=add_panels, ) self._extensions[manifest.name] = installation + if add_panels: + workspace = self.get_workspace(workspace_id) + for panel in prepared_panels: + self._seed_default_panel_state_locked(workspace, panel) + self.workspace_registry.update_workspace(workspace) self._bump_version() return installation except Exception: @@ -1579,6 +2438,8 @@ def install_extension( for panel in workspace.ui.custom_panels if panel.id not in installed_panel_ids ] + for panel_id in installed_panel_ids: + workspace.ui.panels.pop(panel_id, None) workspace.ui.view_revision += 1 self.workspace_registry.update_workspace(workspace) @@ -1618,6 +2479,8 @@ def _uninstall_extension_locked(self, name: str) -> ExtensionInstallation | None for panel in workspace.ui.custom_panels if panel.id not in installation.panel_ids ] + for panel_id in installation.panel_ids: + workspace.ui.panels.pop(panel_id, None) workspace.ui.view_revision += 1 self.workspace_registry.update_workspace(workspace) @@ -1641,6 +2504,7 @@ def _add_custom_panel_locked(self, workspace_id: str, panel: CustomPanelSpec) -> panels = [existing for existing in workspace.ui.custom_panels if existing.id != panel.id] panels.append(panel) workspace.ui.custom_panels = panels + self._seed_default_panel_state_locked(workspace, panel) workspace.ui.view_revision += 1 self.workspace_registry.update_workspace(workspace) @@ -1694,6 +2558,9 @@ def snapshot(self, workspace_id: str | None = None) -> dict[str, Any]: "version": self.version, "active_workspace_id": self.workspace_registry.active_workspace_id, "extensions": [installation.to_dict() for installation in self.list_extensions()], + "panel_definitions": [ + definition.to_dict() for definition in self.list_panel_definitions() + ], "tools": [record.to_dict() for record in self.tools.list()], "workspaces": [ { @@ -1705,23 +2572,30 @@ def snapshot(self, workspace_id: str | None = None) -> dict[str, Any]: "workspace": { "id": workspace.id, "dataset_name": workspace.dataset_name, + "collections": [ + collection.to_dict() + for collection in sorted( + workspace.collections.values(), + key=lambda item: item.id, + ) + ], "ui": { "active_layout_key": workspace.ui.active_layout_key, "selected_ids": list(workspace.ui.selected_ids), - "similarity_query": ( - workspace.ui.similarity_query.to_dict() - if workspace.ui.similarity_query is not None - else None - ), "layout_views": { layout_key: view.to_dict() for layout_key, view in sorted(workspace.ui.layout_views.items()) }, + "panels": { + panel_id: state.to_dict() + for panel_id, state in sorted(workspace.ui.panels.items()) + }, "custom_panels": [ - { - **panel.to_dict(), - "data": self.get_panel_payload(workspace.id, panel), - } + _custom_panel_instance_payload( + panel, + workspace.ui.panels, + data=self.get_panel_payload(workspace.id, panel), + ) for panel in workspace.ui.custom_panels ], "has_explicit_view": workspace.ui.has_explicit_view, diff --git a/src/hyperview/server/app.py b/src/hyperview/server/app.py index 0948844..b4d4d4a 100644 --- a/src/hyperview/server/app.py +++ b/src/hyperview/server/app.py @@ -5,16 +5,21 @@ import json import os from pathlib import Path -from typing import Any, Literal +from typing import Any import numpy as np from fastapi import Depends, FastAPI, HTTPException, Query from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import FileResponse, Response, StreamingResponse from fastapi.staticfiles import StaticFiles -from pydantic import BaseModel +from pydantic import BaseModel, ConfigDict from hyperview._version import __version__ +from hyperview.control import ( + CommandEnvelope, + ControlService, + create_default_command_registry, +) from hyperview.core.dataset import Dataset from hyperview.core.selection import ( OrbitViewState3D, @@ -26,7 +31,7 @@ LayoutViewState, ) from hyperview.storage.metrics import distance_metric_for_space -from hyperview.storage.schema import parse_layout_dimension +from hyperview.storage.schema import parse_layout_dimension, space_key_from_index_ref # Extensions whose content is handed off to esbuild for JSX transformation. _JSX_SUFFIXES = {".jsx"} @@ -60,6 +65,7 @@ class LassoSelectionRequest(BaseModel): viewport_width: int | None = None viewport_height: int | None = None label_filter: str | None = None + missing_label_filter: bool = False offset: int = 0 limit: int = 100 include_thumbnails: bool = False @@ -93,36 +99,15 @@ class UiSelectionRequest(BaseModel): sample_ids: list[str] -class UiSimilarityQueryRequest(BaseModel): - workspace_id: str - sample_id: str - layout_key: str | None = None - space_key: str | None = None - k: int = 18 - source: str | None = None - - -class UiSimilarityClearRequest(BaseModel): - workspace_id: str - - -class UiStatePatchSimilarity(BaseModel): - sample_id: str - layout_key: str | None = None - space_key: str | None = None - k: int = 18 - source: str | None = None - - class UiStatePatchRequest(BaseModel): + model_config = ConfigDict(extra="forbid") + workspace_id: str client_id: str | None = None set_active_layout: bool = False active_layout_key: str | None = None set_selection: bool = False selected_ids: list[str] | None = None - set_similarity_query: bool = False - similarity_query: UiStatePatchSimilarity | None = None class UiLayoutViewRequest(BaseModel): @@ -131,52 +116,6 @@ class UiLayoutViewRequest(BaseModel): camera_3d: OrbitViewState3D | None = None -class UiPanelRequest(BaseModel): - workspace_id: str - panel_id: str - title: str | None = None - kind: Literal["extension", "scatter", "module", "builtin"] = "extension" - builtin_panel: Literal["samples"] | None = None - extension: str | None = None - extension_panel: str | None = None - module_file: str | None = None - layout_key: str | None = None - position: str = "right" - reference_panel_id: str | None = None - direction: str | None = None - width: int | None = None - height: int | None = None - min_width: int | None = None - min_height: int | None = None - max_width: int | None = None - max_height: int | None = None - visible: bool = True - props: dict[str, Any] | None = None - - -class UiPanelRemoveRequest(BaseModel): - workspace_id: str - panel_id: str - - -class UiPanelUpdateRequest(BaseModel): - workspace_id: str - panel_id: str - title: str | None = None - position: Literal["center", "right", "bottom"] | None = None - reference_panel_id: str | None = None - direction: Literal["right", "left", "above", "below", "within"] | None = None - width: int | None = None - height: int | None = None - min_width: int | None = None - min_height: int | None = None - max_width: int | None = None - max_height: int | None = None - visible: bool | None = None - active: bool | None = None - props: dict[str, Any] | None = None - - class SamplesQueryRequest(BaseModel): workspace_id: str | None = None ids: list[str] | None = None @@ -246,6 +185,10 @@ class ExtensionRemoveRequest(BaseModel): name: str +def _control_service(runtime: HyperViewRuntime) -> ControlService: + return ControlService(runtime, create_default_command_registry()) + + class SampleResponse(BaseModel): """Response model for a sample.""" @@ -253,6 +196,8 @@ class SampleResponse(BaseModel): filepath: str filename: str label: str | None + text: str | None = None + modality: str = "image" thumbnail: str | None media_url: str | None = None thumbnail_url: str | None = None @@ -284,6 +229,30 @@ class SpaceInfoResponse(BaseModel): config: dict[str, Any] | None +class RepresentationInfoResponse(BaseModel): + """A derived vector field, independent of how it is searched.""" + + id: str + entity_set_id: str + field_path: str + kind: str + shape: list[int] + model_id: str + provider: str + modality: str + geometry: str + count: int + + +class IndexInfoResponse(BaseModel): + """A searchable access path over a representation.""" + + id: str + representation_id: str + query_modes: list[str] + scorer: str + + class DatasetResponse(BaseModel): """Response model for dataset info.""" @@ -291,6 +260,8 @@ class DatasetResponse(BaseModel): num_samples: int labels: list[str] spaces: list[SpaceInfoResponse] + representations: list[RepresentationInfoResponse] = [] + indexes: list[IndexInfoResponse] = [] layouts: list[LayoutInfoResponse] @@ -313,7 +284,8 @@ class SimilarSampleResponse(SampleResponse): class SimilaritySearchResponse(BaseModel): """Response model for similarity search results.""" - query_id: str + query_id: str | None = None + query_text: str | None = None query_sample: SampleResponse | None space_key: str | None metric: str @@ -321,6 +293,15 @@ class SimilaritySearchResponse(BaseModel): results: list[SimilarSampleResponse] +class TextSearchRequest(BaseModel): + query_text: str + k: int = 10 + space_key: str | None = None + index_id: str | None = None + layout_key: str | None = None + include_thumbnails: bool = False + + def serialize_sample_for_response( sample: Any, include_thumbnail: bool = False, @@ -391,6 +372,77 @@ def _query_samples(ds: Dataset, request: SamplesQueryRequest | SamplesAggregateR return samples +def _resolve_collection_items( + ds: Dataset, + collection: Any, + *, + offset: int, + limit: int, +) -> tuple[list[tuple[Any, float | None]], int, bool]: + """Materialize a page of a collection's members as (sample, score) pairs. + + Reuses the same retrieval/filtering methods the collection.* commands used + to build the collection, rather than storing membership row-by-row, so a + collection always reflects current data. + """ + query = collection.query or {} + + if collection.kind == "neighbors": + anchor = query.get("anchor") or {} + sample_id = str(anchor.get("entityId") or "") + if not sample_id: + raise ValueError("Neighbors collection is missing an anchor entity id") + k = int(query.get("k") or 18) + space_key = query.get("spaceKey") or space_key_from_index_ref(query.get("indexId")) + results = ds.find_similar(sample_id, k=k, space_key=space_key) + total = len(results) + page = results[offset : offset + limit] + return ( + [(sample, float(distance)) for sample, distance in page], + total, + offset + limit < total, + ) + + if collection.kind == "search": + query_text = str(query.get("queryText") or "") + if not query_text: + raise ValueError("Search collection is missing queryText") + k = int(query.get("k") or 18) + space_key = query.get("spaceKey") or space_key_from_index_ref(query.get("indexId")) + results = ds.find_similar_by_text(query_text, k=k, space_key=space_key) + total = len(results) + page = results[offset : offset + limit] + return ( + [(sample, float(distance)) for sample, distance in page], + total, + offset + limit < total, + ) + + if collection.kind == "filter": + field = str(query.get("field") or "label") + op = str(query.get("op") or "eq") + value = query.get("value") + if field == "label" and op == "eq": + samples, total = ds.get_samples_paginated(offset=offset, limit=limit, label=value) + return [(sample, None) for sample in samples], total, offset + limit < total + # Less common field/op combinations: filter in memory. Every sample is + # still loaded once per request in this path, unlike the label fast path. + all_samples = ds.samples + if field == "label": + matches = [s for s in all_samples if s.label == value] + else: + matches = [s for s in all_samples if _metadata_value(s.metadata, field) == value] + total = len(matches) + page = matches[offset : offset + limit] + return [(sample, None) for sample in page], total, offset + limit < total + + if collection.kind == "all": + samples, total = ds.get_samples_paginated(offset=offset, limit=limit) + return [(sample, None) for sample in samples], total, offset + limit < total + + raise ValueError(f"Collection kind '{collection.kind}' is not yet materializable") + + def create_app( dataset: Dataset | None = None, runtime: HyperViewRuntime | None = None, @@ -694,56 +746,11 @@ async def set_ui_selection_query_endpoint( workspace = runtime_dep.set_selection(request.workspace_id, [sample.id for sample in samples]) return {"workspace": workspace.to_dict()} - @app.post("/api/control/ui/similarity") - async def set_ui_similarity_query_endpoint( - request: UiSimilarityQueryRequest, - runtime_dep: HyperViewRuntime = Depends(get_runtime), - ): - try: - query = runtime_dep.resolve_similarity_query( - request.workspace_id, - request.sample_id, - layout_key=request.layout_key, - space_key=request.space_key, - k=request.k, - source=request.source, - ) - except (KeyError, LookupError) as exc: - raise HTTPException(status_code=404, detail=str(exc)) from exc - except ValueError as exc: - raise HTTPException(status_code=400, detail=str(exc)) from exc - workspace = runtime_dep.set_similarity_query(request.workspace_id, query) - return {"workspace": workspace.to_dict()} - - @app.delete("/api/control/ui/similarity") - async def clear_ui_similarity_query_endpoint( - request: UiSimilarityClearRequest, - runtime_dep: HyperViewRuntime = Depends(get_runtime), - ): - workspace = runtime_dep.set_similarity_query(request.workspace_id, None) - return {"workspace": workspace.to_dict()} - @app.patch("/api/control/ui/state") async def patch_ui_state_endpoint( request: UiStatePatchRequest, runtime_dep: HyperViewRuntime = Depends(get_runtime), ): - query = None - if request.set_similarity_query and request.similarity_query is not None: - try: - query = runtime_dep.resolve_similarity_query( - request.workspace_id, - request.similarity_query.sample_id, - layout_key=request.similarity_query.layout_key, - space_key=request.similarity_query.space_key, - k=request.similarity_query.k, - source=request.similarity_query.source, - ) - except (KeyError, LookupError) as exc: - raise HTTPException(status_code=404, detail=str(exc)) from exc - except ValueError as exc: - raise HTTPException(status_code=400, detail=str(exc)) from exc - try: workspace = runtime_dep.patch_ui_state( request.workspace_id, @@ -751,8 +758,6 @@ async def patch_ui_state_endpoint( active_layout_key=request.active_layout_key, set_selection=request.set_selection, selected_ids=request.selected_ids, - set_similarity_query=request.set_similarity_query, - similarity_query=query, source_client_id=request.client_id, ) except ValueError as exc: @@ -773,82 +778,18 @@ async def set_ui_layout_view_endpoint( ) return {"workspace": workspace.to_dict()} - @app.post("/api/control/ui/panels") - async def add_ui_panel_endpoint( - request: UiPanelRequest, - runtime_dep: HyperViewRuntime = Depends(get_runtime), - ): - try: - workspace = runtime_dep.add_runtime_panel( - request.workspace_id, - panel_id=request.panel_id, - title=request.title, - kind=request.kind, - builtin_panel=request.builtin_panel, - extension=request.extension, - extension_panel=request.extension_panel, - layout_key=request.layout_key, - position=request.position, - reference_panel_id=request.reference_panel_id, - direction=request.direction, - width=request.width, - height=request.height, - min_width=request.min_width, - min_height=request.min_height, - max_width=request.max_width, - max_height=request.max_height, - visible=request.visible, - props=request.props, - ) - except LookupError as exc: - raise HTTPException(status_code=404, detail=str(exc)) from exc - except ValueError as exc: - raise HTTPException(status_code=400, detail=str(exc)) from exc - return {"workspace": workspace.to_dict()} - - @app.patch("/api/control/ui/panels") - async def update_ui_panel_endpoint( - request: UiPanelUpdateRequest, + @app.get("/api/control/commands") + async def list_control_commands_endpoint( runtime_dep: HyperViewRuntime = Depends(get_runtime), ): - fields = request.model_fields_set - patch: dict[str, Any] = {} - for field_name in ( - "title", - "position", - "reference_panel_id", - "direction", - "width", - "height", - "min_width", - "min_height", - "max_width", - "max_height", - "visible", - "active", - "props", - ): - if field_name in fields: - patch[field_name] = getattr(request, field_name) - try: - workspace = runtime_dep.update_custom_panel( - request.workspace_id, - request.panel_id, - **patch, - ) - except KeyError as exc: - raise HTTPException(status_code=404, detail=str(exc)) from exc - except ValueError as exc: - raise HTTPException(status_code=400, detail=str(exc)) from exc - return {"workspace": workspace.to_dict()} + return {"commands": _control_service(runtime_dep).list_commands()} - @app.delete("/api/control/ui/panels") - async def remove_ui_panel_endpoint( - request: UiPanelRemoveRequest, + @app.post("/api/control/commands/run") + async def run_control_command_endpoint( + request: CommandEnvelope, runtime_dep: HyperViewRuntime = Depends(get_runtime), ): - workspace = runtime_dep.remove_custom_panel(request.workspace_id, request.panel_id) - return {"workspace": workspace.to_dict()} + return _control_service(runtime_dep).run(request).to_dict() @app.get("/api/tools") async def list_tools_endpoint( @@ -884,6 +825,16 @@ async def list_extensions_endpoint( "extensions": [item.to_dict() for item in runtime_dep.list_extensions()], } + @app.get("/api/panel-definitions") + async def list_panel_definitions_endpoint( + runtime_dep: HyperViewRuntime = Depends(get_runtime), + ): + return { + "panel_definitions": [ + definition.to_dict() for definition in runtime_dep.list_panel_definitions() + ], + } + @app.post("/api/control/extensions/install") async def install_extension_endpoint( request: ExtensionInstallRequest, @@ -926,6 +877,8 @@ async def get_dataset_info(ds: Dataset = Depends(get_dataset)): num_samples=len(ds), labels=ds.labels, spaces=space_dicts, + representations=[s.to_representation_dict() for s in spaces], + indexes=[s.to_index_dict() for s in spaces], layouts=layout_dicts, ) @@ -935,11 +888,15 @@ async def get_samples( offset: int = Query(0, ge=0), limit: int = Query(100, ge=1, le=MAX_SAMPLE_PAGE_SIZE), label: str | None = None, + missing_label: bool = Query(False), include_thumbnails: bool = Query(False), ): """Get paginated sample metadata.""" samples, total = ds.get_samples_paginated( - offset=offset, limit=limit, label=label + offset=offset, + limit=limit, + label=None if missing_label else label, + missing_label=missing_label, ) return { @@ -1078,6 +1035,76 @@ async def aggregate_samples( ] return {"total": len(samples), "group_by": request.group_by, "groups": groups} + @app.get("/api/collections/{collection_id}") + async def get_collection( + collection_id: str, + runtime_dep: HyperViewRuntime = Depends(get_runtime), + workspace_id: str | None = Query(None), + ): + """Get collection metadata (kind, query, dataset scope).""" + try: + workspace = runtime_dep.get_workspace(workspace_id) + except ValueError as exc: + raise HTTPException(status_code=404, detail=str(exc)) from exc + collection = workspace.collections.get(collection_id) + if collection is None: + raise HTTPException(status_code=404, detail=f"Unknown collection: {collection_id}") + return collection.to_dict() + + @app.get("/api/collections/{collection_id}/items") + async def get_collection_items( + collection_id: str, + runtime_dep: HyperViewRuntime = Depends(get_runtime), + workspace_id: str | None = Query(None), + offset: int = Query(0, ge=0), + limit: int = Query(100, ge=1, le=MAX_SAMPLE_PAGE_SIZE), + include_thumbnails: bool = Query(False), + ): + """Get a paged, materialized slice of a collection's member samples. + + This is the read path collections were introduced for: panels resolve + `collection_id` to rows here instead of owning retrieval/filter logic + themselves. Membership is (re)computed from the collection's stored + `query`, not stored row-by-row, so it always reflects current data. + """ + try: + workspace = runtime_dep.get_workspace(workspace_id) + except ValueError as exc: + raise HTTPException(status_code=404, detail=str(exc)) from exc + collection = workspace.collections.get(collection_id) + if collection is None: + raise HTTPException(status_code=404, detail=f"Unknown collection: {collection_id}") + + try: + ds = runtime_dep.get_dataset( + workspace_id=workspace.id, dataset_name=collection.dataset_id or None + ) + except ValueError as exc: + raise HTTPException(status_code=404, detail=str(exc)) from exc + + try: + items, total, has_more = _resolve_collection_items( + ds, collection, offset=offset, limit=limit + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + return { + "collection_id": collection.id, + "kind": collection.kind, + "offset": offset, + "limit": limit, + "total": total, + "has_more": has_more, + "items": [ + { + **serialize_sample_for_response(sample, include_thumbnail=include_thumbnails), + "score": score, + } + for sample, score in items + ], + } + @app.get("/api/embeddings", response_model=EmbeddingsResponse) async def get_embeddings(ds: Dataset = Depends(get_dataset), layout_key: str | None = None): """Get embedding coordinates for visualization.""" @@ -1161,7 +1188,8 @@ async def lasso_selection(request: LassoSelectionRequest, ds: Dataset = Depends( x_max=x_max, y_min=y_min, y_max=y_max, - label_filter=request.label_filter, + label_filter=None if request.missing_label_filter else request.label_filter, + missing_label_filter=request.missing_label_filter, ) if candidate_coords.size == 0: @@ -1265,7 +1293,8 @@ async def lasso_selection(request: LassoSelectionRequest, ds: Dataset = Depends( view=view_3d, viewport_width=request.viewport_width, viewport_height=request.viewport_height, - label_filter=request.label_filter, + label_filter=None if request.missing_label_filter else request.label_filter, + missing_label_filter=request.missing_label_filter, ) else: raise HTTPException( @@ -1299,11 +1328,12 @@ async def search_similar( ds: Dataset = Depends(get_dataset), k: int = Query(10, ge=1, le=100), space_key: str | None = None, + index_id: str | None = None, layout_key: str | None = None, include_thumbnails: bool = Query(False), ): """Return k nearest neighbors for a given sample.""" - resolved_space_key = space_key + resolved_space_key = space_key or space_key_from_index_ref(index_id) if layout_key is not None: layout = next((item for item in ds.list_layouts() if item.layout_key == layout_key), None) if layout is None: @@ -1357,6 +1387,69 @@ async def search_similar( results=results, ) + @app.post("/api/search/text", response_model=SimilaritySearchResponse) + async def search_by_text( + request: TextSearchRequest, + ds: Dataset = Depends(get_dataset), + ): + """Return k nearest neighbors for a natural-language text query.""" + query_text = request.query_text.strip() + if not query_text: + raise HTTPException(status_code=400, detail="query_text must be a non-empty string") + + resolved_space_key = request.space_key or space_key_from_index_ref(request.index_id) + if request.layout_key is not None: + layout = next( + (item for item in ds.list_layouts() if item.layout_key == request.layout_key), + None, + ) + if layout is None: + raise HTTPException(status_code=404, detail=f"Layout not found: {request.layout_key}") + if resolved_space_key is not None and resolved_space_key != layout.space_key: + raise HTTPException( + status_code=400, + detail="space_key does not match the requested layout_key", + ) + resolved_space_key = layout.space_key + + spaces = ds.list_spaces() + if resolved_space_key is None: + if not spaces: + raise HTTPException(status_code=400, detail="No embedding spaces available") + resolved_space_key = spaces[0].space_key + space = next((s for s in spaces if s.space_key == resolved_space_key), None) + metric = distance_metric_for_space(space) if space is not None else "cosine" + + try: + similar = ds.find_similar_by_text( + query_text, + k=request.k, + space_key=resolved_space_key, + layout_key=request.layout_key, + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + results = [] + for sample, distance in similar: + results.append( + SimilarSampleResponse( + **serialize_sample_for_response( + sample, include_thumbnail=request.include_thumbnails + ), + distance=distance, + ) + ) + + return SimilaritySearchResponse( + query_text=query_text, + query_sample=None, + space_key=resolved_space_key, + metric=metric, + k=request.k, + results=results, + ) + # Serve static frontend files static_dir = Path(__file__).parent / "static" if static_dir.exists(): diff --git a/src/hyperview/static_export.py b/src/hyperview/static_export.py new file mode 100644 index 0000000..ea98e33 --- /dev/null +++ b/src/hyperview/static_export.py @@ -0,0 +1,401 @@ +"""Static workspace export for read-only HyperView demos.""" + +from __future__ import annotations + +import io +import json +import shutil +from dataclasses import dataclass +from pathlib import Path +from typing import Any +from urllib.parse import quote + +from fastapi import HTTPException + +from hyperview.core.dataset import Dataset +from hyperview.runtime import CollectionState, CustomPanelSpec, HyperViewRuntime +from hyperview.server.app import ( + DEFAULT_THUMBNAIL_SIZE, + MAX_SAMPLE_PAGE_SIZE, + serialize_sample_for_response, +) +from hyperview.storage.metrics import distance_metric_for_space +from hyperview.storage.schema import parse_layout_dimension, space_key_from_index_ref + +SAMPLE_SHARD_SIZE = 500 +SIMILARITY_EXPORT_K = 100 + + +@dataclass(frozen=True) +class StaticExportResult: + workspace_id: str + dataset_name: str + output_dir: Path + num_samples: int + num_sample_shards: int + num_layouts: int + num_collections: int + num_media_files: int + + def to_dict(self) -> dict[str, Any]: + return { + "workspace_id": self.workspace_id, + "dataset_name": self.dataset_name, + "output_dir": str(self.output_dir), + "num_samples": self.num_samples, + "num_sample_shards": self.num_sample_shards, + "num_layouts": self.num_layouts, + "num_collections": self.num_collections, + "num_media_files": self.num_media_files, + } + + +def _write_json(path: Path, payload: Any) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(payload, indent=2, sort_keys=True), encoding="utf-8") + + +def _copy_static_frontend(out_dir: Path) -> None: + static_dir = Path(__file__).parent / "server" / "static" + if not static_dir.exists(): + raise RuntimeError(f"Packaged frontend assets are missing: {static_dir}") + shutil.copytree(static_dir, out_dir, dirs_exist_ok=True) + + +def _inject_static_flag(index_path: Path) -> None: + marker = "window.__HYPERVIEW_STATIC__ = true;" + script = f"" + if not index_path.exists(): + raise RuntimeError(f"Frontend index.html is missing from export: {index_path}") + html = index_path.read_text(encoding="utf-8") + if marker in html: + return + if "" in html: + html = html.replace("", f"{script}", 1) + else: + html = f"{script}\n{html}" + index_path.write_text(html, encoding="utf-8") + + +def _dataset_payload(dataset: Dataset) -> dict[str, Any]: + return { + "name": dataset.name, + "num_samples": len(dataset), + "labels": dataset.labels, + "spaces": [space.to_api_dict() for space in dataset.list_spaces()], + "layouts": [layout.to_api_dict() for layout in dataset.list_layouts()], + } + + +def _embedding_payload(dataset: Dataset, layout_key: str) -> dict[str, Any]: + layout = next((item for item in dataset.list_layouts() if item.layout_key == layout_key), None) + if layout is None: + raise ValueError(f"Layout not found: {layout_key}") + ids, labels, coords = dataset.get_visualization_data(layout_key) + return { + "layout_key": layout_key, + "geometry": layout.geometry, + "ids": ids, + "labels": labels, + "coords": coords.tolist(), + } + + +def _sample_path_segment(sample_id: str) -> str: + return quote(sample_id, safe="") + + +def _write_sample_media(out_dir: Path, sample: Any) -> int: + media_count = 0 + source = Path(sample.filepath).expanduser() + if source.exists() and source.is_file(): + sample_dir = out_dir / "api" / "samples" / _sample_path_segment(sample.id) + sample_dir.mkdir(parents=True, exist_ok=True) + shutil.copy2(source, sample_dir / "content") + + media_dir = out_dir / "media" / "samples" / _sample_path_segment(sample.id) + media_dir.mkdir(parents=True, exist_ok=True) + shutil.copy2(source, media_dir / source.name) + media_count += 2 + + try: + thumb = sample.get_thumbnail((DEFAULT_THUMBNAIL_SIZE, DEFAULT_THUMBNAIL_SIZE)) + if thumb.mode in ("RGBA", "P"): + thumb = thumb.convert("RGB") + buffer = io.BytesIO() + thumb.save(buffer, format="JPEG", quality=85) + thumb_bytes = buffer.getvalue() + (sample_dir / "thumbnail").write_bytes(thumb_bytes) + thumb_dir = out_dir / "media" / "thumbnails" + thumb_dir.mkdir(parents=True, exist_ok=True) + (thumb_dir / f"{_sample_path_segment(sample.id)}.jpg").write_bytes(thumb_bytes) + media_count += 2 + except Exception: + pass + return media_count + + +def _write_samples(out_dir: Path, dataset: Dataset) -> tuple[list[dict[str, Any]], int, int]: + samples = [serialize_sample_for_response(sample, include_thumbnail=False) for sample in dataset.samples] + shards: list[str] = [] + for offset in range(0, len(samples), SAMPLE_SHARD_SIZE): + shard_index = len(shards) + shard_path = f"shards/{shard_index:06d}.json" + shard_samples = samples[offset : offset + SAMPLE_SHARD_SIZE] + _write_json( + out_dir / "api" / "samples" / shard_path, + { + "total": len(samples), + "offset": offset, + "limit": SAMPLE_SHARD_SIZE, + "samples": shard_samples, + }, + ) + shards.append(shard_path) + + _write_json( + out_dir / "api" / "samples" / "index.json", + { + "total": len(samples), + "shard_size": SAMPLE_SHARD_SIZE, + "shards": shards, + }, + ) + + media_count = 0 + for sample in dataset.samples: + payload = serialize_sample_for_response(sample, include_thumbnail=False, ensure_dimensions=True) + _write_json(out_dir / "api" / "samples" / f"{_sample_path_segment(sample.id)}.json", payload) + media_count += _write_sample_media(out_dir, sample) + + return samples, len(shards), media_count + + +def _resolve_collection_ids(dataset: Dataset, collection: CollectionState) -> tuple[list[str], dict[str, float] | None]: + query = collection.query or {} + if collection.kind == "all": + return [sample.id for sample in dataset.samples], None + if collection.kind == "filter": + field = query.get("field") + op = query.get("op") + value = query.get("value") + if field == "label" and op == "eq": + return [sample.id for sample in dataset.samples if sample.label == value], None + return [], None + if collection.kind in {"neighbors", "search"}: + anchor = query.get("anchor") + k = int(query.get("k") or SIMILARITY_EXPORT_K) + space_key = query.get("spaceKey") or space_key_from_index_ref(query.get("indexId")) + layout_key = query.get("layoutId") + if isinstance(anchor, dict): + sample_id = str(anchor.get("entityId") or anchor.get("entity_id") or "") + try: + results = dataset.find_similar( + sample_id, + k=max(1, min(k, SIMILARITY_EXPORT_K)), + space_key=space_key if isinstance(space_key, str) else None, + layout_key=layout_key if isinstance(layout_key, str) else None, + ) + except Exception: + return [], None + scores = {sample.id: float(distance) for sample, distance in results} + return [sample.id for sample, _distance in results], scores + return [], collection.scores + + +def _write_collections(out_dir: Path, dataset: Dataset, snapshot: dict[str, Any]) -> int: + collections = [ + CollectionState.from_dict(item) + for item in snapshot.get("workspace", {}).get("collections", []) + if isinstance(item, dict) + ] + for collection in collections: + ids, scores = _resolve_collection_ids(dataset, collection) + rows = [ + { + "sample_id": sample_id, + "rank": rank, + "score": scores.get(sample_id) if scores is not None else None, + "sample": serialize_sample_for_response(sample, include_thumbnail=False), + } + for rank, (sample_id, sample) in enumerate( + (item.id, item) for item in dataset.get_samples_by_ids(ids) + ) + ] + collection_dir = out_dir / "api" / "collections" / quote(collection.id, safe="") + _write_json( + collection_dir / "items.json", + {"collection_id": collection.id, "total": len(rows), "offset": 0, "limit": len(rows), "items": rows}, + ) + _write_json( + collection_dir / "index.json", + {"collection": collection.to_dict(), "total": len(rows), "shards": ["items.json"]}, + ) + return len(collections) + + +def _write_embeddings(out_dir: Path, dataset: Dataset) -> int: + layouts = dataset.list_layouts() + if not layouts: + return 0 + default_layout = next( + (layout for layout in layouts if parse_layout_dimension(layout.layout_key) == 2), + layouts[0], + ) + _write_json(out_dir / "api" / "embeddings" / "default.json", _embedding_payload(dataset, default_layout.layout_key)) + for layout in layouts: + _write_json( + out_dir / "api" / "embeddings" / f"{quote(layout.layout_key, safe='')}.json", + _embedding_payload(dataset, layout.layout_key), + ) + return len(layouts) + + +def _similarity_payload( + dataset: Dataset, + sample_id: str, + *, + space_key: str | None, + layout_key: str | None = None, +) -> dict[str, Any] | None: + spaces = dataset.list_spaces() + resolved_space_key = space_key + if layout_key is not None: + layout = next((item for item in dataset.list_layouts() if item.layout_key == layout_key), None) + if layout is not None: + resolved_space_key = layout.space_key + if resolved_space_key is None and spaces: + resolved_space_key = spaces[0].space_key + space = next((item for item in spaces if item.space_key == resolved_space_key), None) + metric = distance_metric_for_space(space) if space is not None else "cosine" + try: + query_sample = dataset[sample_id] + similar = dataset.find_similar(sample_id, k=SIMILARITY_EXPORT_K, space_key=resolved_space_key) + except Exception: + return None + return { + "query_id": sample_id, + "query_sample": serialize_sample_for_response(query_sample, include_thumbnail=False), + "space_key": resolved_space_key, + "metric": metric, + "k": SIMILARITY_EXPORT_K, + "results": [ + { + **serialize_sample_for_response(sample, include_thumbnail=False), + "distance": float(distance), + } + for sample, distance in similar + ], + } + + +def _write_similarity(out_dir: Path, dataset: Dataset) -> None: + spaces = dataset.list_spaces() + if not spaces: + return + for sample in dataset.samples: + sample_dir = out_dir / "api" / "search" / "similar" / _sample_path_segment(sample.id) + for space in spaces: + payload = _similarity_payload(dataset, sample.id, space_key=space.space_key) + if payload is not None: + _write_json(sample_dir / f"{quote(space.space_key, safe='')}.json", payload) + default_payload = _similarity_payload(dataset, sample.id, space_key=None) + if default_payload is not None: + _write_json(sample_dir / "default.json", default_payload) + + +def _copy_panel_modules(out_dir: Path, runtime: HyperViewRuntime, workspace_id: str) -> None: + workspace = runtime.get_workspace(workspace_id) + for panel in workspace.ui.custom_panels: + if panel.kind != "module": + continue + _copy_panel_module(out_dir, panel, workspace_id) + + +def _copy_panel_module(out_dir: Path, panel: CustomPanelSpec, workspace_id: str) -> None: + module_file = panel.resolved_module_file() + if module_file is None or not module_file.exists(): + return + target_dir = out_dir / "api" / "panels" / "content" / quote(workspace_id, safe="") / quote(panel.id, safe="") + target_dir.mkdir(parents=True, exist_ok=True) + for item in module_file.parent.iterdir(): + target = target_dir / item.name + if item.is_file(): + if item.suffix.lower() == ".jsx": + try: + from esbuild_py import transform as esbuild_transform + + transformed = esbuild_transform(item.read_text(encoding="utf-8")) + except Exception as exc: + raise RuntimeError(f"Failed to transform JSX panel module {item}: {exc}") from exc + if transformed is not None: + target.write_text(transformed, encoding="utf-8") + else: + shutil.copy2(item, target) + + +def export_runtime_workspace( + runtime: HyperViewRuntime, + workspace_id: str, + out: str | Path, +) -> StaticExportResult: + out_dir = Path(out).expanduser().resolve() + out_dir.mkdir(parents=True, exist_ok=True) + + workspace = runtime.get_workspace(workspace_id) + if not workspace.dataset_name: + raise RuntimeError(f"Workspace '{workspace_id}' has no dataset") + dataset = runtime.get_dataset(workspace_id, workspace.dataset_name) + + _copy_static_frontend(out_dir) + _inject_static_flag(out_dir / "index.html") + + snapshot = runtime.snapshot(workspace_id) + _write_json(out_dir / "api" / "runtime.json", snapshot) + _write_json(out_dir / "api" / "dataset.json", _dataset_payload(dataset)) + _write_json( + out_dir / "api" / "panel-definitions.json", + {"panel_definitions": snapshot.get("panel_definitions", [])}, + ) + + _samples, num_shards, num_media_files = _write_samples(out_dir, dataset) + num_layouts = _write_embeddings(out_dir, dataset) + num_collections = _write_collections(out_dir, dataset, snapshot) + _write_similarity(out_dir, dataset) + _copy_panel_modules(out_dir, runtime, workspace_id) + + manifest = { + "static": True, + "workspace_id": workspace_id, + "dataset_name": workspace.dataset_name, + "api": { + "runtime": "api/runtime.json", + "dataset": "api/dataset.json", + "samples": "api/samples/index.json", + "embeddings": "api/embeddings/default.json", + }, + } + _write_json(out_dir / "hyperview-static.json", manifest) + + return StaticExportResult( + workspace_id=workspace_id, + dataset_name=workspace.dataset_name, + output_dir=out_dir, + num_samples=len(dataset), + num_sample_shards=num_shards, + num_layouts=num_layouts, + num_collections=num_collections, + num_media_files=num_media_files, + ) + + +def export_workspace(workspace_id: str, out: str | Path) -> StaticExportResult: + runtime = HyperViewRuntime() + try: + runtime.get_workspace(workspace_id) + except ValueError as exc: + raise RuntimeError(str(exc)) from exc + try: + return export_runtime_workspace(runtime, workspace_id, out) + except HTTPException as exc: + raise RuntimeError(str(exc.detail)) from exc diff --git a/src/hyperview/storage/backend.py b/src/hyperview/storage/backend.py index b44801a..d9d6b11 100644 --- a/src/hyperview/storage/backend.py +++ b/src/hyperview/storage/backend.py @@ -30,6 +30,7 @@ def get_samples_paginated( offset: int = 0, limit: int = 100, label: str | None = None, + missing_label: bool = False, ) -> tuple[list[Sample], int]: """Get paginated samples. Returns (samples, total_count).""" @@ -171,6 +172,7 @@ def get_lasso_candidates_aabb( y_min: float, y_max: float, label_filter: str | None = None, + missing_label_filter: bool = False, ) -> tuple[list[str], np.ndarray]: """Return candidate (id, xy) rows within an axis-aligned bounding box.""" @@ -192,6 +194,26 @@ def find_similar_by_vector( ) -> list[tuple[Sample, float]]: """Find k nearest neighbors to a query vector.""" + def find_similar_by_text( + self, + text: str, + k: int = 10, + space_key: str | None = None, + *, + query_vector: np.ndarray | None = None, + ) -> list[tuple[Sample, float]]: + """Find k nearest neighbors to a text query vector. + + Storage backends do not encode text themselves. Callers should pass a + precomputed ``query_vector`` from the active embedding space. + """ + if query_vector is None: + raise ValueError( + "query_vector is required for find_similar_by_text; " + "encode text with the dataset's embedding model first" + ) + return self.find_similar_by_vector(query_vector, k, space_key) + @abstractmethod def close(self) -> None: """Close the storage connection.""" diff --git a/src/hyperview/storage/lancedb_backend.py b/src/hyperview/storage/lancedb_backend.py index be6bad5..31900cb 100644 --- a/src/hyperview/storage/lancedb_backend.py +++ b/src/hyperview/storage/lancedb_backend.py @@ -105,9 +105,21 @@ def _ensure_scalar_index( ) +def _ensure_text_search_index(table: lancedb.table.Table) -> None: + if "text" not in table.schema.names: + return + for index in table.list_indices(): + if "text" in list(getattr(index, "columns", []) or []): + return + table.create_fts_index("text", replace=False) + + def _ensure_samples_indices(table: lancedb.table.Table) -> None: _ensure_scalar_index(table, "id") _ensure_scalar_index(table, "label", index_type="BITMAP") + if "modality" in table.schema.names: + _ensure_scalar_index(table, "modality", index_type="BITMAP") + _ensure_text_search_index(table) def _ensure_layout_indices(table: lancedb.table.Table, *, layout_dimension: int) -> None: @@ -187,11 +199,21 @@ def get_samples_paginated( offset: int = 0, limit: int = 100, label: str | None = None, + missing_label: bool = False, ) -> tuple[list[Sample], int]: if self._samples_table is None: return [], 0 - if label: + if missing_label: + total = self._samples_table.count_rows("label IS NULL") + results = ( + self._samples_table.search() + .where("label IS NULL") + .offset(offset) + .limit(limit) + .to_list() + ) + elif label: total = self._samples_table.count_rows(_eq_sql("label", label)) results = ( self._samples_table.search() @@ -624,6 +646,7 @@ def get_lasso_candidates_aabb( y_min: float, y_max: float, label_filter: str | None = None, + missing_label_filter: bool = False, ) -> tuple[list[str], np.ndarray]: layout_dimension = parse_layout_dimension(layout_key) if layout_dimension != 2: @@ -650,7 +673,7 @@ def get_lasso_candidates_aabb( ) rows = table.search().select(["id", "x", "y"]).where(where).to_list() - if label_filter is not None and rows: + if (label_filter is not None or missing_label_filter) and rows: if self._samples_table is None: return [], np.empty((0, 2), dtype=np.float32) @@ -660,7 +683,8 @@ def get_lasso_candidates_aabb( chunk = candidate_ids[i : i + 1000] if not chunk: continue - where = f"{_in_sql('id', chunk)} AND {_eq_sql('label', label_filter)}" + label_where = "label IS NULL" if missing_label_filter else _eq_sql("label", label_filter) + where = f"{_in_sql('id', chunk)} AND {label_where}" for r in self._samples_table.search().select(["id"]).where(where).to_list(): matching_ids.add(r["id"]) diff --git a/src/hyperview/storage/memory_backend.py b/src/hyperview/storage/memory_backend.py index dc16098..f09a9b7 100644 --- a/src/hyperview/storage/memory_backend.py +++ b/src/hyperview/storage/memory_backend.py @@ -46,9 +46,12 @@ def get_samples_paginated( offset: int = 0, limit: int = 100, label: str | None = None, + missing_label: bool = False, ) -> tuple[list[Sample], int]: samples = list(self._samples.values()) - if label: + if missing_label: + samples = [s for s in samples if not s.label] + elif label: samples = [s for s in samples if s.label == label] total = len(samples) return samples[offset : offset + limit], total @@ -246,6 +249,7 @@ def get_lasso_candidates_aabb( y_min: float, y_max: float, label_filter: str | None = None, + missing_label_filter: bool = False, ) -> tuple[list[str], np.ndarray]: if parse_layout_dimension(layout_key) != 2: raise ValueError( @@ -255,7 +259,11 @@ def get_lasso_candidates_aabb( layout_store = self._layouts.get(layout_key, {}) ids, coords = [], [] for id_, coord in layout_store.items(): - if label_filter is not None: + if missing_label_filter: + sample = self._samples.get(id_) + if sample is None or sample.label: + continue + elif label_filter is not None: sample = self._samples.get(id_) if sample is None or sample.label != label_filter: continue diff --git a/src/hyperview/storage/schema.py b/src/hyperview/storage/schema.py index 374d326..fda9d59 100644 --- a/src/hyperview/storage/schema.py +++ b/src/hyperview/storage/schema.py @@ -50,6 +50,8 @@ def create_sample_schema() -> pa.Schema: pa.field("id", pa.utf8(), nullable=False), pa.field("filepath", pa.utf8(), nullable=False), pa.field("label", pa.utf8(), nullable=True), + pa.field("text", pa.utf8(), nullable=True), + pa.field("modality", pa.utf8(), nullable=False), pa.field("metadata_json", pa.utf8(), nullable=True), pa.field("thumbnail_base64", pa.utf8(), nullable=True), ] @@ -130,6 +132,50 @@ def provider(self) -> str: def geometry(self) -> str: return (self.config or {}).get("geometry", "euclidean") + @property + def modality(self) -> str: + return (self.config or {}).get("modality", "image") + + @property + def index_id(self) -> str: + return index_id_for_space_key(self.space_key) + + def to_representation_dict(self) -> dict[str, Any]: + """Representation view of this space (architecture.md vocabulary). + + The representation is the derived vector field itself, independent of + how it is searched; `space_key` doubles as the representation id until + storage keys the two separately. + """ + return { + "id": self.space_key, + "entity_set_id": "samples", + "field_path": f"embeddings.{self.space_key}", + "kind": "vector", + "shape": [self.dim], + "model_id": self.model_id, + "provider": self.provider, + "modality": self.modality, + "geometry": self.geometry, + "count": self.count, + } + + def to_index_dict(self) -> dict[str, Any]: + """Index view of this space: the searchable access path over the + representation, addressable as `space:` in retrieval + queries and collection payloads.""" + from hyperview.storage.metrics import distance_metric_for_space + + query_modes = ["nearest"] + if self.modality in ("text", "multimodal"): + query_modes.append("text") + return { + "id": self.index_id, + "representation_id": self.space_key, + "query_modes": query_modes, + "scorer": distance_metric_for_space(self), + } + def to_dict(self) -> dict[str, Any]: return { "space_key": self.space_key, @@ -237,6 +283,28 @@ def from_dict(cls, row: dict[str, Any]) -> "LayoutInfo": ) +INDEX_ID_PREFIX = "space:" + + +def index_id_for_space_key(space_key: str) -> str: + return f"{INDEX_ID_PREFIX}{space_key}" + + +def space_key_from_index_ref(value: Any) -> str | None: + """Resolve an index reference to a space_key. + + Accepts the canonical `space:` index id as well as a bare + space_key, so retrieval can be addressed by index id while storage still + keys spaces by space_key. + """ + if not isinstance(value, str) or not value.strip(): + return None + ref = value.strip() + if ref.startswith(INDEX_ID_PREFIX): + ref = ref[len(INDEX_ID_PREFIX) :] + return ref or None + + def slugify_model_id(model_id: str) -> str: """Convert a model ID to a safe table name component. @@ -290,6 +358,8 @@ def sample_to_dict(sample: Sample) -> dict[str, Any]: "id": sample.id, "filepath": sample.filepath, "label": sample.label, + "text": sample.text, + "modality": sample.modality, "metadata_json": json.dumps(sample.metadata) if sample.metadata else None, "thumbnail_base64": sample.thumbnail_base64, } @@ -304,6 +374,8 @@ def dict_to_sample(row: dict[str, Any]) -> Sample: id=row["id"], filepath=row["filepath"], label=row.get("label"), + text=row.get("text"), + modality=str(row.get("modality") or "image"), metadata=metadata, thumbnail_base64=row.get("thumbnail_base64"), ) diff --git a/src/hyperview/ui.py b/src/hyperview/ui.py index 59d4db7..9b1c4ba 100644 --- a/src/hyperview/ui.py +++ b/src/hyperview/ui.py @@ -56,7 +56,7 @@ class ExtensionPanel: extension: str panel: str title: str | None = None - position: PanelPosition = "right" + position: PanelPosition | None = None reference_panel_id: str | None = None direction: PanelDirection | None = None layout: PanelLayout | None = None @@ -228,7 +228,7 @@ def _compile_item( workspace_id=workspace_id, ) - position = default_position or item.position + position = default_position if default_position is not None else item.position spec = _panel_to_spec( item, position=position, @@ -280,7 +280,7 @@ def _container_direction(kind: ContainerKind) -> PanelDirection: def _panel_to_spec( panel: ExtensionPanel | Scatter | Samples, *, - position: PanelPosition, + position: PanelPosition | None, runtime: HyperViewRuntime | None, workspace_id: str | None, ) -> CustomPanelSpec: diff --git a/tests/test_cli_control.py b/tests/test_cli_control.py index c6c4410..734252c 100644 --- a/tests/test_cli_control.py +++ b/tests/test_cli_control.py @@ -265,21 +265,24 @@ def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dic output = json.loads(capsys.readouterr().out) assert output["workspace"]["id"] == "default" assert recorded["method"] == "POST" - assert recorded["url"] == "http://127.0.0.1:6262/api/control/ui/panels" + assert recorded["url"] == "http://127.0.0.1:6262/api/control/commands/run" assert recorded["payload"] == { - "workspace_id": "default", - "panel_id": "agent-panel", - "title": None, - "kind": "extension", - "builtin_panel": None, - "extension": "agent-tools", - "extension_panel": "agent-panel", - "layout_key": None, - "position": "right", - "reference_panel_id": None, - "direction": None, - "visible": True, - "props": None, + "command": "workspace.panel.add", + "target": {"workspace_id": "default"}, + "args": { + "panel_id": "agent-panel", + "title": None, + "kind": "extension", + "builtin_panel": None, + "extension": "agent-tools", + "extension_panel": "agent-panel", + "layout_key": None, + "position": "right", + "reference_panel_id": None, + "direction": None, + "visible": True, + "props": None, + }, } @@ -322,21 +325,24 @@ def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dic output = json.loads(capsys.readouterr().out) assert output["workspace"]["id"] == "default" assert recorded["method"] == "POST" - assert recorded["url"] == "http://127.0.0.1:6262/api/control/ui/panels" + assert recorded["url"] == "http://127.0.0.1:6262/api/control/commands/run" assert recorded["payload"] == { - "workspace_id": "default", - "panel_id": "uncha-poincare", - "title": "UNCHA", - "kind": "scatter", - "builtin_panel": None, - "extension": None, - "extension_panel": None, - "layout_key": "uncha__poincare_umap__2d", - "position": "center", - "reference_panel_id": "hycoclip-poincare", - "direction": "right", - "visible": True, - "props": None, + "command": "workspace.panel.add", + "target": {"workspace_id": "default"}, + "args": { + "panel_id": "uncha-poincare", + "title": "UNCHA", + "kind": "scatter", + "builtin_panel": None, + "extension": None, + "extension_panel": None, + "layout_key": "uncha__poincare_umap__2d", + "position": "center", + "reference_panel_id": "hycoclip-poincare", + "direction": "right", + "visible": True, + "props": None, + }, } @@ -373,24 +379,52 @@ def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dic output = json.loads(capsys.readouterr().out) assert output["workspace"]["id"] == "default" assert recorded["method"] == "POST" - assert recorded["url"] == "http://127.0.0.1:6262/api/control/ui/panels" + assert recorded["url"] == "http://127.0.0.1:6262/api/control/commands/run" assert recorded["payload"] == { - "workspace_id": "default", - "panel_id": "samples", - "title": None, - "kind": "builtin", - "builtin_panel": "samples", - "extension": None, - "extension_panel": None, - "layout_key": None, - "position": "right", - "reference_panel_id": None, - "direction": None, - "visible": True, - "props": None, + "command": "workspace.panel.add", + "target": {"workspace_id": "default"}, + "args": { + "panel_id": "samples", + "title": None, + "kind": "builtin", + "builtin_panel": "samples", + "extension": None, + "extension_panel": None, + "layout_key": None, + "position": "right", + "reference_panel_id": None, + "direction": None, + "visible": True, + "props": None, + }, } +def test_cli_panel_definitions_lists_runtime_panel_contracts(monkeypatch, capsys) -> None: + recorded: dict[str, str] = {} + + def fake_get(url: str) -> dict[str, object]: + recorded["url"] = url + return { + "panel_definitions": [ + { + "panel_type": "samples", + "label": "Samples", + "title": "Samples", + "source": "builtin", + } + ] + } + + monkeypatch.setattr("hyperview.cli._http_get_json", fake_get) + + main(["ui", "panel", "definitions", "--json"]) + + output = json.loads(capsys.readouterr().out) + assert recorded["url"] == "http://127.0.0.1:6262/api/panel-definitions" + assert output["panel_definitions"][0]["panel_type"] == "samples" + + def test_cli_panel_update_posts_runtime_panel_props(monkeypatch, capsys) -> None: recorded: dict[str, object] = {} @@ -419,16 +453,18 @@ def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dic output = json.loads(capsys.readouterr().out) assert output["workspace"]["id"] == "default" - assert recorded["method"] == "PATCH" - assert recorded["url"] == "http://127.0.0.1:6262/api/control/ui/panels" + assert recorded["method"] == "POST" + assert recorded["url"] == "http://127.0.0.1:6262/api/control/commands/run" assert recorded["payload"] == { - "workspace_id": "default", - "panel_id": "ranked-clip", - "props": { - "mode": "ranked", - "rank": { - "anchorSampleId": "sample-1", - "k": 24, + "command": "workspace.panel.update", + "target": {"workspace_id": "default", "panel_id": "ranked-clip"}, + "args": { + "props": { + "mode": "ranked", + "rank": { + "anchorSampleId": "sample-1", + "k": 24, + }, }, }, } @@ -439,7 +475,7 @@ def test_cli_panel_layout_commands_patch_runtime_panel_state(monkeypatch, capsys def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: recorded.append({"url": url, "payload": payload, "method": method}) - return {"workspace": {"id": "default"}} + return {"ok": True, "workspace": {"id": "default"}} monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) @@ -501,93 +537,427 @@ def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dic "--json", ] ) + main( + [ + "ui", + "panel", + "show", + "--workspace", + "default", + "--panel-id", + "readout", + "--json", + ] + ) _ = capsys.readouterr() - assert [entry["method"] for entry in recorded] == ["PATCH", "PATCH", "PATCH", "PATCH"] + assert [entry["method"] for entry in recorded] == ["POST", "POST", "POST", "POST", "POST"] + assert {entry["url"] for entry in recorded} == { + "http://127.0.0.1:6262/api/control/commands/run" + } assert recorded[0]["payload"] == { - "workspace_id": "default", - "panel_id": "readout", - "width": 360, - "min_width": 280, + "command": "workspace.panel.resize", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": {"width": 360, "min_width": 280}, } assert recorded[1]["payload"] == { - "workspace_id": "default", - "panel_id": "readout", - "position": "right", - "reference_panel_id": "samples", - "direction": "right", + "command": "workspace.panel.move", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": { + "position": "right", + "reference_panel_id": "samples", + "direction": "right", + }, } assert recorded[2]["payload"] == { - "workspace_id": "default", - "panel_id": "readout", - "active": True, - "visible": True, + "command": "workspace.panel.focus", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": {}, } assert recorded[3]["payload"] == { - "workspace_id": "default", - "panel_id": "readout", - "visible": False, + "command": "workspace.panel.close", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": {}, + } + assert recorded[4]["payload"] == { + "command": "workspace.panel.show", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": {}, + } + + +def test_cli_generic_command_runner_posts_command_envelope(monkeypatch, capsys) -> None: + recorded: dict[str, object] = {} + + def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: + recorded["url"] = url + recorded["payload"] = payload + recorded["method"] = method + return { + "ok": True, + "command": "workspace.panel.resize", + "workspace": {"id": "default"}, + "revision": 2, + } + + monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) + + main( + [ + "commands", + "run", + "workspace.panel.resize", + "--target", + "workspace_id=default", + "--target", + "panel_id=readout", + "--arg", + "width=360", + "--arg", + "min_width=null", + "--json", + ] + ) + + output = json.loads(capsys.readouterr().out) + assert output["ok"] is True + assert recorded["method"] == "POST" + assert recorded["url"] == "http://127.0.0.1:6262/api/control/commands/run" + assert recorded["payload"] == { + "command": "workspace.panel.resize", + "target": {"workspace_id": "default", "panel_id": "readout"}, + "args": {"width": 360, "min_width": None}, } -def test_cli_similarity_commands_post_explicit_neighbor_context(monkeypatch, capsys) -> None: +def test_cli_generic_command_runner_prints_error_envelope(monkeypatch, capsys) -> None: + def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: + return { + "ok": False, + "command": "workspace.panel.resize", + "error": { + "code": "not_found", + "message": "Panel not found: missing", + }, + } + + monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) + + main( + [ + "commands", + "run", + "workspace.panel.resize", + "--target", + "workspace_id=default", + "--target", + "panel_id=missing", + "--arg", + "width=360", + "--json", + ] + ) + + output = json.loads(capsys.readouterr().out) + assert output == { + "ok": False, + "command": "workspace.panel.resize", + "error": { + "code": "not_found", + "message": "Panel not found: missing", + }, + } + + +def test_cli_panel_state_and_samples_retrieval_commands_post_command_envelopes( + monkeypatch, + capsys, +) -> None: recorded: list[dict[str, object]] = [] def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: recorded.append({"url": url, "payload": payload, "method": method}) - return {"workspace": {"id": "default"}} + return { + "ok": True, + "command": payload["command"], + "result": {"panel_id": "samples", "state_revision": 1}, + "workspace": {"id": "default"}, + } monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) main( [ "ui", - "similarity", - "set", + "panel", + "state", + "get", + "--workspace", + "default", + "--panel-id", + "samples", + "--json", + ] + ) + panel_state_get = json.loads(capsys.readouterr().out) + assert panel_state_get["command"] == "workspace.panel.state.get" + + main( + [ + "ui", + "panel", + "state", + "patch", + "--workspace", + "default", + "--panel-id", + "samples", + "--state-json", + '{"settings":{"density":"compact"}}', + "--expected-revision", + "0", + "--replace", + "--json", + ] + ) + panel_state_patch = json.loads(capsys.readouterr().out) + assert panel_state_patch["command"] == "workspace.panel.state.patch" + + main( + [ + "ui", + "samples", + "retrieval", + "set-anchor", "--workspace", "default", "--sample-id", "sample-2", - "--layout-key", - "clip__pca__2d", + "--space-key", + "clip", "--k", "12", "--json", ] ) - set_output = json.loads(capsys.readouterr().out) - assert set_output["workspace"]["id"] == "default" + retrieval_set = json.loads(capsys.readouterr().out) + assert retrieval_set["workspace"]["id"] == "default" main( [ "ui", - "similarity", + "samples", + "retrieval", + "set-k", + "--workspace", + "default", + "--k", + "24", + "--json", + ] + ) + retrieval_k = json.loads(capsys.readouterr().out) + assert retrieval_k["workspace"]["id"] == "default" + + main( + [ + "ui", + "samples", + "retrieval", "clear", "--workspace", "default", "--json", ] ) - clear_output = json.loads(capsys.readouterr().out) - assert clear_output["workspace"]["id"] == "default" + retrieval_clear = json.loads(capsys.readouterr().out) + assert retrieval_clear["workspace"]["id"] == "default" + + assert [entry["method"] for entry in recorded] == ["POST", "POST", "POST", "POST", "POST"] + assert {entry["url"] for entry in recorded} == { + "http://127.0.0.1:6262/api/control/commands/run" + } + assert recorded[0]["payload"] == { + "command": "workspace.panel.state.get", + "target": {"workspace_id": "default", "panel_id": "samples"}, + "args": {}, + } + assert recorded[1]["payload"] == { + "command": "workspace.panel.state.patch", + "target": {"workspace_id": "default", "panel_id": "samples"}, + "args": { + "state": {"settings": {"density": "compact"}}, + "replace_state": True, + "expected_revision": 0, + }, + } + assert recorded[2]["payload"] == { + "command": "panel.samples.retrieval.set-anchor", + "target": {"workspace_id": "default"}, + "args": { + "sample_id": "sample-2", + "layout_key": None, + "space_key": "clip", + "k": 12, + "source": "cli", + }, + } + assert recorded[3]["payload"] == { + "command": "panel.samples.retrieval.set-k", + "target": {"workspace_id": "default"}, + "args": {"k": 24}, + } + assert recorded[4]["payload"] == { + "command": "panel.samples.retrieval.clear", + "target": {"workspace_id": "default"}, + "args": {}, + } + + +def test_cli_panel_state_patch_prints_error_envelope(monkeypatch, capsys) -> None: + def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: + return { + "ok": False, + "command": payload["command"], + "error": { + "code": "conflict", + "message": "panel state revision conflict: expected 1, got 2", + }, + } + + monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) + + main( + [ + "ui", + "panel", + "state", + "patch", + "--workspace", + "default", + "--panel-id", + "samples", + "--state-json", + '{"settings":{"density":"loose"}}', + "--expected-revision", + "1", + "--json", + ] + ) + + output = json.loads(capsys.readouterr().out) + assert output == { + "ok": False, + "command": "workspace.panel.state.patch", + "error": { + "code": "conflict", + "message": "panel state revision conflict: expected 1, got 2", + }, + } + + +def test_cli_panel_collection_shortcuts_post_command_envelopes(monkeypatch, capsys) -> None: + recorded: list[dict[str, object]] = [] + + def fake_send(url: str, payload: dict[str, object], method: str = "POST") -> dict[str, object]: + recorded.append({"url": url, "payload": payload, "method": method}) + return { + "ok": True, + "command": payload["command"], + "result": {"collection_id": "collection-1"}, + "workspace": {"id": "default"}, + } + + monkeypatch.setattr("hyperview.cli._http_send_json", fake_send) + + main( + [ + "panel", + "samples", + "show-neighbors", + "--workspace", + "default", + "--sample-id", + "sample-2", + "--space-key", + "clip", + "--k", + "12", + "--json", + ] + ) + neighbors_output = json.loads(capsys.readouterr().out) + assert neighbors_output["result"]["collection_id"] == "collection-1" + + main( + [ + "panel", + "labels", + "filter", + "--workspace", + "default", + "--value", + "cat", + "--json", + ] + ) + filter_output = json.loads(capsys.readouterr().out) + assert filter_output["result"]["collection_id"] == "collection-1" + + main( + [ + "panel", + "labels", + "filter", + "--workspace", + "default", + "--clear", + "--json", + ] + ) + _ = capsys.readouterr() assert recorded == [ { - "url": "http://127.0.0.1:6262/api/control/ui/similarity", + "url": "http://127.0.0.1:6262/api/control/commands/run", + "payload": { + "command": "collection.neighbors.create", + "target": {"workspace_id": "default"}, + "args": { + "sample_id": "sample-2", + "layout_key": None, + "space_key": "clip", + "k": 12, + "source": "cli", + }, + }, + "method": "POST", + }, + { + "url": "http://127.0.0.1:6262/api/control/commands/run", "payload": { - "workspace_id": "default", - "sample_id": "sample-2", - "layout_key": "clip__pca__2d", - "space_key": None, - "k": 12, - "source": "cli", + "command": "collection.filter.set", + "target": {"workspace_id": "default"}, + "args": { + "field": "label", + "source": "cli", + "value": "cat", + }, }, "method": "POST", }, { - "url": "http://127.0.0.1:6262/api/control/ui/similarity", - "payload": {"workspace_id": "default"}, - "method": "DELETE", + "url": "http://127.0.0.1:6262/api/control/commands/run", + "payload": { + "command": "collection.filter.set", + "target": {"workspace_id": "default"}, + "args": { + "field": "label", + "source": "cli", + "clear": True, + }, + }, + "method": "POST", }, ] diff --git a/tests/test_collection_items_api.py b/tests/test_collection_items_api.py new file mode 100644 index 0000000..7f223f7 --- /dev/null +++ b/tests/test_collection_items_api.py @@ -0,0 +1,131 @@ +from __future__ import annotations + +from pathlib import Path + +import numpy as np +from fastapi.testclient import TestClient + +from hyperview import Dataset +from hyperview.control import CommandEnvelope, ControlService, create_default_command_registry +from hyperview.core.sample import Sample +from hyperview.runtime import HyperViewRuntime, ProviderRegistry, WorkspaceRegistry +from hyperview.server.app import create_app + + +def _service_with_dataset(tmp_path: Path) -> ControlService: + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + dataset = Dataset("collection_items", persist=False) + labels = ["cat", "cat", "dog", "cat", "dog"] + for i, label in enumerate(labels): + dataset.add_sample( + Sample(id=f"s{i}", filepath=f"/virtual/s{i}.png", label=label) + ) + dataset._storage.ensure_space( + model_id="test-model", + dim=2, + config={"provider": "test", "geometry": "euclidean"}, + space_key="test_space", + ) + dataset._storage.add_embeddings( + "test_space", + [f"s{i}" for i in range(len(labels))], + np.asarray( + [[1.0, 0.0], [0.9, 0.1], [-1.0, 0.0], [0.8, 0.2], [-0.9, -0.1]], + dtype=np.float32, + ), + ) + runtime.attach_dataset_instance("default", dataset) + return ControlService(runtime, create_default_command_registry()) + + +def test_neighbors_collection_items_are_paged_and_ordered(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + result = service.run( + CommandEnvelope( + command="collection.neighbors.create", + target={"workspace_id": "default"}, + args={"sample_id": "s0", "k": 3, "source": "test"}, + ) + ) + assert result.ok is True + collection_id = result.result["collection"]["id"] + + client = TestClient(create_app(runtime=service.runtime)) + + page1 = client.get( + f"/api/collections/{collection_id}/items", params={"offset": 0, "limit": 2} + ) + assert page1.status_code == 200 + body1 = page1.json() + assert body1["kind"] == "neighbors" + assert body1["total"] == 3 + assert body1["has_more"] is True + assert len(body1["items"]) == 2 + assert body1["items"][0]["score"] <= body1["items"][1]["score"] + + page2 = client.get( + f"/api/collections/{collection_id}/items", params={"offset": 2, "limit": 2} + ) + body2 = page2.json() + assert body2["has_more"] is False + assert len(body2["items"]) == 1 + + all_ids = [item["id"] for item in body1["items"]] + [item["id"] for item in body2["items"]] + assert len(set(all_ids)) == 3 + + +def test_filter_collection_items_are_paged(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + result = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"value": "cat", "source": "test"}, + ) + ) + assert result.ok is True + collection_id = result.result["collection"]["id"] + + client = TestClient(create_app(runtime=service.runtime)) + + response = client.get( + f"/api/collections/{collection_id}/items", params={"offset": 0, "limit": 2} + ) + assert response.status_code == 200 + body = response.json() + assert body["kind"] == "filter" + assert body["total"] == 3 + assert body["has_more"] is True + assert all(item["label"] == "cat" for item in body["items"]) + assert all(item["score"] is None for item in body["items"]) + + +def test_collection_items_404_for_unknown_collection(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + client = TestClient(create_app(runtime=service.runtime)) + + response = client.get("/api/collections/does-not-exist/items") + assert response.status_code == 404 + + +def test_get_collection_metadata(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + result = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"value": "dog", "source": "test"}, + ) + ) + collection_id = result.result["collection"]["id"] + client = TestClient(create_app(runtime=service.runtime)) + + response = client.get(f"/api/collections/{collection_id}") + assert response.status_code == 200 + body = response.json() + assert body["id"] == collection_id + assert body["kind"] == "filter" + assert body["query"]["value"] == "dog" diff --git a/tests/test_control_command_api.py b/tests/test_control_command_api.py new file mode 100644 index 0000000..e307bf6 --- /dev/null +++ b/tests/test_control_command_api.py @@ -0,0 +1,118 @@ +from __future__ import annotations + +from pathlib import Path + +from fastapi.testclient import TestClient + +from hyperview.runtime import HyperViewRuntime, ProviderRegistry, WorkspaceRegistry +from hyperview.server.app import create_app + + +def _client_with_panel(tmp_path: Path) -> TestClient: + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + runtime.add_runtime_panel( + "default", + panel_id="samples", + kind="builtin", + builtin_panel="samples", + position="right", + width=320, + min_width=240, + ) + return TestClient(create_app(runtime=runtime)) + + +def test_control_commands_endpoint_lists_backend_panel_commands(tmp_path: Path) -> None: + client = _client_with_panel(tmp_path) + + response = client.get("/api/control/commands") + + assert response.status_code == 200 + command_ids = {command["id"] for command in response.json()["commands"]} + assert { + "workspace.panel.resize", + "workspace.panel.move", + "workspace.panel.close", + "workspace.panel.show", + "workspace.panel.focus", + "workspace.panel.add", + "workspace.panel.update", + "workspace.panel.remove", + "workspace.panel.state.get", + "workspace.panel.state.patch", + "panel.samples.retrieval.set-anchor", + "panel.samples.retrieval.set-text-query", + "panel.samples.retrieval.clear", + "panel.samples.retrieval.set-k", + "collection.filter.set", + "collection.neighbors.create", + }.issubset(command_ids) + commands = {command["id"]: command for command in response.json()["commands"]} + add_kind_schema = commands["workspace.panel.add"]["args_schema"]["properties"]["kind"] + assert "module" not in add_kind_schema["enum"] + builtin_panel_schema = commands["workspace.panel.add"]["args_schema"]["properties"][ + "builtin_panel" + ] + assert "samples" not in str(builtin_panel_schema.get("enum", "")) + + +def test_control_command_run_mutates_runtime_panel_state(tmp_path: Path) -> None: + client = _client_with_panel(tmp_path) + + resize_response = client.post( + "/api/control/commands/run", + json={ + "command": "workspace.panel.resize", + "target": {"workspace_id": "default", "panel_id": "samples"}, + "args": {"width": 420, "min_width": None}, + }, + ) + + assert resize_response.status_code == 200 + resize_payload = resize_response.json() + assert resize_payload["ok"] is True + assert resize_payload["workspace"]["ui"]["custom_panels"][0]["width"] == 420 + assert resize_payload["snapshot"]["workspace"]["ui"]["custom_panels"][0]["width"] == 420 + assert resize_payload["snapshot"]["panel_definitions"] + assert resize_payload["workspace"]["ui"]["custom_panels"][0]["min_width"] is None + + focus_response = client.post( + "/api/control/commands/run", + json={ + "command": "workspace.panel.focus", + "target": {"workspace_id": "default", "panel_id": "samples"}, + }, + ) + + assert focus_response.status_code == 200 + focus_payload = focus_response.json() + assert focus_payload["ok"] is True + assert focus_payload["workspace"]["ui"]["active_panel_id"] == "samples" + + +def test_control_command_run_returns_machine_readable_errors(tmp_path: Path) -> None: + client = _client_with_panel(tmp_path) + + response = client.post( + "/api/control/commands/run", + json={ + "command": "workspace.panel.resize", + "target": {"workspace_id": "default", "panel_id": "missing"}, + "args": {"width": 420}, + }, + ) + + assert response.status_code == 200 + payload = response.json() + assert payload["ok"] is False + assert payload["error"]["code"] == "not_found" + + +def test_panel_rest_adapter_routes_are_not_registered(tmp_path: Path) -> None: + client = _client_with_panel(tmp_path) + + route_paths = {getattr(route, "path", "") for route in client.app.routes} + assert not any(path.startswith("/api/control/ui/panels") for path in route_paths) diff --git a/tests/test_control_commands.py b/tests/test_control_commands.py new file mode 100644 index 0000000..afb3f95 --- /dev/null +++ b/tests/test_control_commands.py @@ -0,0 +1,526 @@ +from __future__ import annotations + +import logging +from pathlib import Path + +import numpy as np + +from hyperview import Dataset +from hyperview.control import CommandEnvelope, ControlService, create_default_command_registry +from hyperview.core.sample import Sample +from hyperview.runtime import HyperViewRuntime, ProviderRegistry, WorkspaceRegistry + + +def _service_with_panel(tmp_path: Path) -> ControlService: + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + runtime.add_runtime_panel( + "default", + panel_id="samples", + kind="builtin", + builtin_panel="samples", + position="right", + width=320, + min_width=240, + ) + return ControlService(runtime, create_default_command_registry()) + + +def _service_with_dataset(tmp_path: Path) -> ControlService: + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + dataset = Dataset("control_commands", persist=False) + for sample_id, label in (("s0", "cat"), ("s1", "cat"), ("s2", "dog")): + dataset.add_sample(Sample(id=sample_id, filepath=f"/virtual/{sample_id}.png", label=label)) + dataset._storage.ensure_space( + model_id="test-model", + dim=2, + config={"provider": "test", "geometry": "euclidean"}, + space_key="test_space", + ) + dataset._storage.add_embeddings( + "test_space", + ["s0", "s1", "s2"], + np.asarray([[1.0, 0.0], [0.9, 0.1], [-1.0, 0.0]], dtype=np.float32), + ) + runtime.attach_dataset_instance("default", dataset) + return ControlService(runtime, create_default_command_registry()) + + +def test_panel_resize_command_mutates_runtime_panel_layout(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + result = service.run( + CommandEnvelope( + command="workspace.panel.resize", + target={"workspace_id": "default", "panel_id": "samples"}, + args={"width": 420, "min_width": None}, + ) + ) + + assert result.ok is True + assert result.revision == 2 + assert result.workspace is not None + panel = result.workspace["ui"]["custom_panels"][0] + assert panel["width"] == 420 + assert panel["min_width"] is None + + +def test_deprecated_command_alias_dispatches_and_warns(tmp_path: Path, caplog) -> None: + service = _service_with_panel(tmp_path) + + with caplog.at_level(logging.WARNING, logger="hyperview.control.aliases"): + result = service.run( + CommandEnvelope( + command="ui.panel.resize", + target={"workspace_id": "default", "panel_id": "samples"}, + args={"width": 420}, + ) + ) + + assert result.ok is True + assert result.command == "workspace.panel.resize" + assert result.workspace is not None + assert result.workspace["ui"]["custom_panels"][0]["width"] == 420 + assert "Deprecated HyperView command 'ui.panel.resize' used" in caplog.text + + +def test_panel_move_focus_close_show_commands_share_dispatch_path(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + move_result = service.run( + CommandEnvelope( + command="workspace.panel.move", + target={"workspace_id": "default", "panel_id": "samples"}, + args={ + "position": "bottom", + "reference_panel_id": None, + "direction": None, + }, + ) + ) + assert move_result.ok is True + assert move_result.workspace is not None + assert move_result.workspace["ui"]["custom_panels"][0]["position"] == "bottom" + + focus_result = service.run( + CommandEnvelope( + command="workspace.panel.focus", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert focus_result.ok is True + assert focus_result.workspace is not None + assert focus_result.workspace["ui"]["active_panel_id"] == "samples" + + close_result = service.run( + CommandEnvelope( + command="workspace.panel.close", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert close_result.ok is True + assert close_result.workspace is not None + assert close_result.workspace["ui"]["custom_panels"][0]["visible"] is False + assert close_result.workspace["ui"]["active_panel_id"] is None + + show_result = service.run( + CommandEnvelope( + command="workspace.panel.show", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert show_result.ok is True + assert show_result.workspace is not None + assert show_result.workspace["ui"]["custom_panels"][0]["visible"] is True + + +def test_panel_add_update_remove_commands_share_dispatch_path(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + add_result = service.run( + CommandEnvelope( + command="workspace.panel.add", + target={"workspace_id": "default"}, + args={ + "panel_id": "samples", + "kind": "builtin", + "builtin_panel": "samples", + "position": "bottom", + "width": 480, + }, + ) + ) + assert add_result.ok is True + assert add_result.workspace is not None + panel = add_result.workspace["ui"]["custom_panels"][0] + assert panel["position"] == "bottom" + assert panel["width"] == 480 + + update_result = service.run( + CommandEnvelope( + command="workspace.panel.update", + target={"workspace_id": "default", "panel_id": "samples"}, + args={ + "title": "Sample Browser", + "visible": False, + "props": {"mode": "compact"}, + }, + ) + ) + assert update_result.ok is True + assert update_result.workspace is not None + panel = update_result.workspace["ui"]["custom_panels"][0] + assert panel["title"] == "Sample Browser" + assert panel["visible"] is False + assert panel["props"] == {"mode": "compact"} + + remove_result = service.run( + CommandEnvelope( + command="workspace.panel.remove", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert remove_result.ok is True + assert remove_result.workspace is not None + assert remove_result.workspace["ui"]["custom_panels"] == [] + + +def test_panel_command_errors_are_machine_readable(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + missing_panel = service.run( + CommandEnvelope( + command="workspace.panel.resize", + target={"workspace_id": "default", "panel_id": "missing"}, + args={"width": 420}, + ) + ) + assert missing_panel.ok is False + assert missing_panel.error is not None + assert missing_panel.error.code == "not_found" + + invalid_resize = service.run( + CommandEnvelope( + command="workspace.panel.resize", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert invalid_resize.ok is False + assert invalid_resize.error is not None + assert invalid_resize.error.code == "validation_error" + + unknown = service.run( + CommandEnvelope( + command="workspace.panel.unknown", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert unknown.ok is False + assert unknown.error is not None + assert unknown.error.code == "unknown_command" + + +def test_panel_dimension_commands_reject_non_positive_values(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + resize_result = service.run( + CommandEnvelope( + command="workspace.panel.resize", + target={"workspace_id": "default", "panel_id": "samples"}, + args={"width": 0}, + ) + ) + assert resize_result.ok is False + assert resize_result.error is not None + assert resize_result.error.code == "validation_error" + + add_result = service.run( + CommandEnvelope( + command="workspace.panel.add", + target={"workspace_id": "default"}, + args={ + "panel_id": "invalid", + "kind": "builtin", + "builtin_panel": "samples", + "width": -1, + }, + ) + ) + assert add_result.ok is False + assert add_result.error is not None + assert add_result.error.code == "validation_error" + + +def test_panel_state_commands_merge_patch_and_check_revision(tmp_path: Path) -> None: + service = _service_with_panel(tmp_path) + + initial = service.run( + CommandEnvelope( + command="workspace.panel.state.get", + target={"workspace_id": "default", "panel_id": "samples"}, + ) + ) + assert initial.ok is True + assert initial.result == { + "panel_id": "samples", + "state": {}, + "state_revision": 0, + } + + first_patch = service.run( + CommandEnvelope( + command="workspace.panel.state.patch", + target={"workspace_id": "default", "panel_id": "samples"}, + args={ + "state": { + "settings": {"density": "compact"}, + "sort": "label", + }, + "expected_revision": 0, + }, + ) + ) + assert first_patch.ok is True + assert first_patch.revision == 1 + assert first_patch.result == { + "panel_id": "samples", + "state": { + "settings": {"density": "compact"}, + "sort": "label", + }, + "state_revision": 1, + } + + second_patch = service.run( + CommandEnvelope( + command="workspace.panel.state.patch", + target={"workspace_id": "default", "panel_id": "samples"}, + args={ + "state": { + "settings": {"density": "comfortable", "columns": 4}, + "sort": None, + }, + "expected_revision": 1, + }, + ) + ) + assert second_patch.ok is True + assert second_patch.revision == 2 + assert second_patch.result["state"] == { + "settings": { + "density": "comfortable", + "columns": 4, + } + } + + conflict = service.run( + CommandEnvelope( + command="workspace.panel.state.patch", + target={"workspace_id": "default", "panel_id": "samples"}, + args={ + "state": {"settings": {"density": "loose"}}, + "expected_revision": 1, + }, + ) + ) + assert conflict.ok is False + assert conflict.error is not None + assert conflict.error.code == "conflict" + + +def test_labels_filter_command_creates_filter_collection(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + + result = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"value": "cat", "source": "test"}, + ) + ) + + assert result.ok is True + assert result.workspace is not None + collection = result.result["collection"] + assert collection["kind"] == "filter" + assert collection["query"] == { + "field": "label", + "op": "eq", + "source": "test", + "value": "cat", + } + samples_state = result.workspace["ui"]["panels"]["samples"]["state"] + assert samples_state["mode"] == "collection" + assert samples_state["collection_id"] == collection["id"] + assert samples_state["collection"]["scores"] is None + assert result.workspace["collections"][0]["id"] == collection["id"] + + clear_result = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"clear": True}, + ) + ) + assert clear_result.ok is True + assert clear_result.result["collection_id"] is None + assert clear_result.workspace is not None + assert clear_result.workspace["ui"]["panels"]["samples"]["state"] == {} + + +def test_samples_neighbors_command_creates_neighbors_collection(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + + result = service.run( + CommandEnvelope( + command="collection.neighbors.create", + target={"workspace_id": "default"}, + args={"sample_id": "s0", "k": 2, "source": "test"}, + ) + ) + + assert result.ok is True + assert result.workspace is not None + collection = result.result["collection"] + assert collection["kind"] == "neighbors" + assert collection["query"]["anchor"] == { + "datasetId": "control_commands", + "entitySetId": "samples", + "entityId": "s0", + } + assert collection["query"]["indexId"] == "space:test_space" + assert collection["query"]["k"] == 2 + samples_state = result.workspace["ui"]["panels"]["samples"]["state"] + assert samples_state["mode"] == "retrieval" + assert samples_state["collection_id"] == collection["id"] + assert samples_state["collection"]["scores"] is None + assert "layoutId" in samples_state["collection"]["query"] + assert samples_state["retrieval"]["space_key"] == "test_space" + assert "similarity_query" not in result.workspace["ui"] + + +def test_labels_filter_clear_does_not_clear_neighbors_collection(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + + neighbors = service.run( + CommandEnvelope( + command="collection.neighbors.create", + target={"workspace_id": "default"}, + args={"sample_id": "s0", "k": 2, "source": "test"}, + ) + ) + assert neighbors.ok is True + assert neighbors.workspace is not None + neighbors_state = neighbors.workspace["ui"]["panels"]["samples"]["state"] + neighbors_collection_id = neighbors_state["collection_id"] + + cleared_filter = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"clear": True}, + ) + ) + + assert cleared_filter.ok is True + assert cleared_filter.workspace is not None + samples_state = cleared_filter.workspace["ui"]["panels"]["samples"]["state"] + assert samples_state["mode"] == "retrieval" + assert samples_state["collection_id"] == neighbors_collection_id + assert samples_state["collection"]["kind"] == "neighbors" + + +def test_retrieval_clear_does_not_clear_filter_collection(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + + filtered = service.run( + CommandEnvelope( + command="collection.filter.set", + target={"workspace_id": "default"}, + args={"value": "cat", "source": "test"}, + ) + ) + assert filtered.ok is True + assert filtered.workspace is not None + filter_state = filtered.workspace["ui"]["panels"]["samples"]["state"] + filter_collection_id = filter_state["collection_id"] + + cleared_retrieval = service.run( + CommandEnvelope( + command="panel.samples.retrieval.clear", + target={"workspace_id": "default"}, + args={}, + ) + ) + + assert cleared_retrieval.ok is True + assert cleared_retrieval.workspace is not None + samples_state = cleared_retrieval.workspace["ui"]["panels"]["samples"]["state"] + assert samples_state["mode"] == "collection" + assert samples_state["collection_id"] == filter_collection_id + assert samples_state["collection"]["kind"] == "filter" + + +def test_samples_retrieval_commands_own_samples_panel_state(tmp_path: Path) -> None: + service = _service_with_dataset(tmp_path) + service.runtime.set_selection("default", ["s1"]) + + result = service.run( + CommandEnvelope( + command="panel.samples.retrieval.set-anchor", + target={"workspace_id": "default"}, + args={"sample_id": "s0", "k": 2, "source": "test"}, + ) + ) + + assert result.ok is True + assert result.workspace is not None + assert result.workspace["ui"]["selected_ids"] == [] + expected_retrieval = { + "anchor_sample_id": "s0", + "layout_key": None, + "space_key": "test_space", + "k": 2, + "source": "test", + } + samples_state = result.workspace["ui"]["panels"]["samples"]["state"] + assert samples_state["mode"] == "retrieval" + assert samples_state["retrieval"] == expected_retrieval + assert "similarity_query" not in result.workspace["ui"] + assert samples_state["collection"]["kind"] == "neighbors" + assert result.result["panel_id"] == "samples" + assert result.result["collection_id"] == samples_state["collection_id"] + + set_k = service.run( + CommandEnvelope( + command="panel.samples.retrieval.set-k", + target={"workspace_id": "default"}, + args={"k": 3}, + ) + ) + assert set_k.ok is True + assert set_k.workspace is not None + set_k_retrieval = set_k.workspace["ui"]["panels"]["samples"]["state"]["retrieval"] + assert set_k_retrieval["k"] == 3 + assert "similarity_query" not in set_k.workspace["ui"] + + clear = service.run( + CommandEnvelope( + command="panel.samples.retrieval.clear", + target={"workspace_id": "default"}, + ) + ) + assert clear.ok is True + assert clear.workspace is not None + assert clear.result == { + "panel_id": "samples", + "collection_id": None, + "collection": None, + } + assert "similarity_query" not in clear.workspace["ui"] + assert clear.workspace["ui"]["panels"]["samples"]["state"] == {} diff --git a/tests/test_dataset_ingestion.py b/tests/test_dataset_ingestion.py index 1a0512c..7d88feb 100644 --- a/tests/test_dataset_ingestion.py +++ b/tests/test_dataset_ingestion.py @@ -153,6 +153,7 @@ def test_lancedb_backend_indexes_filter_columns_and_counts_filtered_rows(tmp_pat Sample(id="a", filepath="/tmp/a.png", label="cat"), Sample(id="b", filepath="/tmp/b.png", label="dog"), Sample(id="c", filepath="/tmp/c.png", label="kid's label"), + Sample(id="d", filepath="/tmp/d.png"), ] ) @@ -160,6 +161,9 @@ def test_lancedb_backend_indexes_filter_columns_and_counts_filtered_rows(tmp_pat assert total == 1 assert [sample.id for sample in filtered] == ["c"] + missing_filtered, missing_total = storage.get_samples_paginated(missing_label=True) + assert missing_total == 1 + assert [sample.id for sample in missing_filtered] == ["d"] sample_indices = { index.columns[0]: index.index_type.lower() for index in storage._samples_table.list_indices() @@ -170,8 +174,8 @@ def test_lancedb_backend_indexes_filter_columns_and_counts_filtered_rows(tmp_pat storage.ensure_layout("demo_space__euclidean_umap__2d", "demo_space", "umap", "euclidean") storage.add_layout_coords( "demo_space__euclidean_umap__2d", - ["a", "b", "c"], - [[0.0, 0.0], [5.0, 5.0], [1.0, 1.0]], + ["a", "b", "c", "d"], + [[0.0, 0.0], [5.0, 5.0], [1.0, 1.0], [1.5, 1.5]], ) candidate_ids, coords = storage.get_lasso_candidates_aabb( @@ -185,6 +189,18 @@ def test_lancedb_backend_indexes_filter_columns_and_counts_filtered_rows(tmp_pat assert candidate_ids == ["c"] assert coords.tolist() == [[1.0, 1.0]] + + missing_candidate_ids, missing_coords = storage.get_lasso_candidates_aabb( + layout_key="demo_space__euclidean_umap__2d", + x_min=-1.0, + x_max=2.0, + y_min=-1.0, + y_max=2.0, + missing_label_filter=True, + ) + + assert missing_candidate_ids == ["d"] + assert missing_coords.tolist() == [[1.5, 1.5]] layout_table = storage._db.open_table("layouts__demo_space__euclidean_umap__2d") layout_indices = { index.columns[0]: index.index_type.lower() for index in layout_table.list_indices() diff --git a/tests/test_huggingface_config.py b/tests/test_huggingface_config.py index 1bc301f..747fefe 100644 --- a/tests/test_huggingface_config.py +++ b/tests/test_huggingface_config.py @@ -260,6 +260,7 @@ def add_from_huggingface(self, dataset_name: str, **kwargs) -> tuple[int, int]: "image_key": "image", "label_key": "label", "label_names_key": None, + "text_key": None, "max_samples": 100, "shuffle": True, "seed": 42, diff --git a/tests/test_public_ui_api.py b/tests/test_public_ui_api.py index 230a734..3563f41 100644 --- a/tests/test_public_ui_api.py +++ b/tests/test_public_ui_api.py @@ -240,6 +240,21 @@ def test_public_ui_panel_layout_helpers_update_runtime_view_state() -> None: assert panel.height == 360 assert panel.min_width == 240 + control_result = session.control.run( + "workspace.panel.resize", + target={"workspace_id": workspace_id, "panel_id": "map"}, + args={"height": 400}, + ) + assert control_result["ok"] is True + assert control_result["workspace"]["ui"]["custom_panels"][0]["height"] == 400 + props_result = session.control.run( + "workspace.panel.update-props", + target={"workspace_id": workspace_id, "panel_id": "map"}, + args={"props": {"mode": "compact"}}, + ) + assert props_result["ok"] is True + assert props_result["workspace"]["ui"]["custom_panels"][0]["props"] == {"mode": "compact"} + session.ui.resize_panel( "map", workspace_id=workspace_id, @@ -260,7 +275,7 @@ def test_public_ui_panel_layout_helpers_update_runtime_view_state() -> None: workspace = runtime.get_workspace(workspace_id) panel = workspace.ui.custom_panels[0] assert panel.width == 620 - assert panel.height == 360 + assert panel.height == 400 assert panel.min_width == 240 assert panel.min_height == 220 assert panel.max_width == 900 @@ -291,6 +306,10 @@ def test_public_ui_extension_panel_resolves_installed_extension(tmp_path: Path) title = "Summary" position = "right" file = "panel.js" + +[panels.default_layout] +position = "bottom" +height = 260 """.strip() + "\n", encoding="utf-8", @@ -322,6 +341,8 @@ def test_public_ui_extension_panel_resolves_installed_extension(tmp_path: Path) assert len(panels) == 1 assert panels[0].id == "summary-instance" assert panels[0].title == "Summary" + assert panels[0].position == "bottom" + assert panels[0].height == 260 assert panels[0].extension == "readout" assert panels[0].extension_panel == "summary" assert panels[0].module_file == str(panel_file.resolve()) @@ -358,6 +379,43 @@ def test_public_ui_add_panels_preserves_extension_identity(tmp_path: Path) -> No assert panels[0].extension_panel == "summary" +def test_public_ui_lists_builtin_and_extension_panel_definitions(tmp_path: Path) -> None: + workspace_id = f"definitions-{uuid4().hex}" + extension_dir = tmp_path / ".hyperview" / "extensions" / "readout" + extension_dir.mkdir(parents=True) + (extension_dir / "extension.toml").write_text( + """ +name = "readout" + +[[panels]] +id = "summary" +title = "Summary" +label = "Summary card" +panel_type = "analysis.summary" +position = "right" +file = "panel.js" +""".strip() + + "\n", + encoding="utf-8", + ) + (extension_dir / "panel.js").write_text("export default function Panel() { return null; }\n") + + runtime = HyperViewRuntime() + runtime.attach_dataset_instance(workspace_id, _make_dataset(), activate_workspace=True) + session = hv.Session(runtime, "127.0.0.1", 6262) + session.ui.add_extension(extension_dir, workspace_id=workspace_id) + + definitions = { + definition["panel_type"]: definition + for definition in session.ui.list_panel_definitions() + } + + assert {"samples", "scatter", "analysis.summary"}.issubset(definitions) + assert definitions["analysis.summary"]["source"] == "extension" + assert definitions["analysis.summary"]["extension"] == "readout" + assert definitions["analysis.summary"]["label"] == "Summary card" + + def test_public_ui_show_similar_resolves_layout_context() -> None: dataset = _make_dataset() sample_ids = [sample.id for sample in dataset] @@ -382,14 +440,68 @@ def test_public_ui_show_similar_resolves_layout_context() -> None: workspace = runtime.get_workspace("demo") assert workspace.ui.selected_ids == [] - assert workspace.ui.similarity_query is not None - assert workspace.ui.similarity_query.to_dict() == { + expected_retrieval = { "anchor_sample_id": "sample-2", "layout_key": layout_key, "space_key": space_key, "k": 100, "source": "test", } + samples_state = workspace.ui.panels["samples"].state + assert samples_state["mode"] == "retrieval" + assert samples_state["retrieval"] == expected_retrieval + + +def test_public_ui_panel_state_helpers_patch_durable_state() -> None: + workspace_id = f"panel-state-{uuid4().hex}" + runtime = HyperViewRuntime() + runtime.attach_dataset_instance(workspace_id, _make_dataset(), activate_workspace=True) + session = hv.Session(runtime, "127.0.0.1", 6262) + + initial = session.ui.get_panel_state("samples", workspace_id=workspace_id) + assert initial == { + "panel_id": "samples", + "state": {}, + "state_revision": 0, + } + + patched = session.ui.patch_panel_state( + "samples", + { + "settings": {"density": "compact"}, + "sort": "label", + }, + workspace_id=workspace_id, + expected_revision=0, + ) + assert patched == { + "panel_id": "samples", + "state": { + "settings": {"density": "compact"}, + "sort": "label", + }, + "state_revision": 1, + } + + merged = session.ui.patch_panel_state( + "samples", + { + "settings": {"columns": 4}, + "sort": None, + }, + workspace_id=workspace_id, + expected_revision=1, + ) + assert merged == { + "panel_id": "samples", + "state": { + "settings": { + "density": "compact", + "columns": 4, + }, + }, + "state_revision": 2, + } def test_public_ui_state_helpers_update_workspace() -> None: @@ -439,6 +551,11 @@ def test_reused_session_ui_control_fails_explicitly() -> None: with pytest.raises(RuntimeError, match="attached to an existing HyperView server"): session.ui.set_selection(["sample-1"], workspace_id="demo") + with pytest.raises(RuntimeError, match="attached to an existing HyperView server"): + session.control.run( + "workspace.panel.focus", + target={"workspace_id": "demo", "panel_id": "samples"}, + ) def test_samples_query_and_aggregate_endpoints() -> None: diff --git a/tests/test_representation_index_api.py b/tests/test_representation_index_api.py new file mode 100644 index 0000000..0626545 --- /dev/null +++ b/tests/test_representation_index_api.py @@ -0,0 +1,161 @@ +from __future__ import annotations + +from types import SimpleNamespace + +import numpy as np +from fastapi.testclient import TestClient + +from hyperview import Dataset +from hyperview.core.sample import Sample +from hyperview.runtime import SimilarityQueryState +from hyperview.server.app import _resolve_collection_items, create_app +from hyperview.storage.schema import ( + index_id_for_space_key, + space_key_from_index_ref, +) + + +def _make_dataset() -> tuple[Dataset, str]: + dataset = Dataset("representation_index_api", persist=False) + ids = ["s0", "s1", "s2"] + + for index, sample_id in enumerate(ids): + dataset.add_sample( + Sample( + id=sample_id, + filepath=f"/missing/{index}.png", + label="cat" if index < 2 else "dog", + ) + ) + + space_key = "test_space" + dataset._storage.ensure_space( + model_id="test-model", + dim=2, + config={"provider": "test", "geometry": "euclidean", "modality": "multimodal"}, + space_key=space_key, + ) + dataset._storage.add_embeddings( + space_key, + ids, + np.array( + [ + [1.0, 0.0], + [0.9, 0.1], + [-1.0, 0.0], + ], + dtype=np.float32, + ), + ) + + return dataset, space_key + + +def test_space_key_from_index_ref_accepts_index_id_and_bare_key() -> None: + assert space_key_from_index_ref("space:clip_b32") == "clip_b32" + assert space_key_from_index_ref("clip_b32") == "clip_b32" + assert space_key_from_index_ref(" space:clip_b32 ") == "clip_b32" + assert space_key_from_index_ref("space:") is None + assert space_key_from_index_ref("") is None + assert space_key_from_index_ref(None) is None + assert space_key_from_index_ref(42) is None + + +def test_dataset_info_exposes_representations_and_indexes() -> None: + dataset, space_key = _make_dataset() + client = TestClient(create_app(dataset)) + + response = client.get("/api/dataset") + + assert response.status_code == 200 + payload = response.json() + + representations = payload["representations"] + assert len(representations) == 1 + representation = representations[0] + assert representation["id"] == space_key + assert representation["entity_set_id"] == "samples" + assert representation["field_path"] == f"embeddings.{space_key}" + assert representation["kind"] == "vector" + assert representation["shape"] == [2] + assert representation["model_id"] == "test-model" + assert representation["modality"] == "multimodal" + assert representation["geometry"] == "euclidean" + + indexes = payload["indexes"] + assert len(indexes) == 1 + index = indexes[0] + assert index["id"] == index_id_for_space_key(space_key) + assert index["representation_id"] == space_key + assert index["query_modes"] == ["nearest", "text"] + assert index["scorer"] == "cosine" + + +def test_image_only_space_index_has_no_text_query_mode() -> None: + dataset = Dataset("representation_index_image_only", persist=False) + dataset.add_sample(Sample(id="s0", filepath="/missing/0.png")) + dataset._storage.ensure_space( + model_id="image-model", + dim=2, + config={"provider": "test", "geometry": "euclidean", "modality": "image"}, + space_key="image_space", + ) + client = TestClient(create_app(dataset)) + + payload = client.get("/api/dataset").json() + + assert payload["indexes"][0]["query_modes"] == ["nearest"] + + +def test_similarity_endpoint_accepts_index_id() -> None: + dataset, space_key = _make_dataset() + client = TestClient(create_app(dataset)) + + by_space_key = client.get( + "/api/search/similar/s0", params={"k": 2, "space_key": space_key} + ) + by_index_id = client.get( + "/api/search/similar/s0", + params={"k": 2, "index_id": index_id_for_space_key(space_key)}, + ) + + assert by_space_key.status_code == 200 + assert by_index_id.status_code == 200 + assert by_index_id.json()["space_key"] == space_key + assert [item["id"] for item in by_index_id.json()["results"]] == [ + item["id"] for item in by_space_key.json()["results"] + ] + + +def test_similarity_query_state_accepts_index_id() -> None: + state = SimilarityQueryState.from_dict( + {"anchor_sample_id": "s0", "index_id": "space:test_space"} + ) + assert state is not None + assert state.space_key == "test_space" + + explicit = SimilarityQueryState.from_dict( + {"anchor_sample_id": "s0", "space_key": "explicit", "index_id": "space:other"} + ) + assert explicit is not None + assert explicit.space_key == "explicit" + + +def test_collection_items_resolver_accepts_index_id_only_query() -> None: + dataset, space_key = _make_dataset() + collection = SimpleNamespace( + kind="neighbors", + query={ + "anchor": {"entityId": "s0"}, + "indexId": index_id_for_space_key(space_key), + "k": 2, + }, + ) + + items, total, has_more = _resolve_collection_items( + dataset, collection, offset=0, limit=10 + ) + + assert total == 2 + assert has_more is False + assert [sample.id for sample, _score in items] == ["s1", "s2"] diff --git a/tests/test_runtime_control_api.py b/tests/test_runtime_control_api.py index e56dc83..4cff7e7 100644 --- a/tests/test_runtime_control_api.py +++ b/tests/test_runtime_control_api.py @@ -180,6 +180,25 @@ def _wait_for_job(client: TestClient, job_id: str) -> dict: raise AssertionError(f"Job {job_id} did not finish in time") +def _run_control_command( + client: TestClient, + command: str, + *, + target: dict[str, object], + args: dict[str, object] | None = None, +): + response = client.post( + "/api/control/commands/run", + json={ + "command": command, + "target": target, + "args": args or {}, + }, + ) + assert response.status_code == 200 + return response + + def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( tmp_path: Path, monkeypatch, @@ -275,10 +294,11 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( runtime.install_extension("default", tmp_path / "label-histogram-ext") runtime.install_extension("default", tmp_path / "notes-ext") - histogram_panel_response = client.post( - "/api/control/ui/panels", - json={ - "workspace_id": "default", + histogram_panel_response = _run_control_command( + client, + "workspace.panel.add", + target={"workspace_id": "default"}, + args={ "panel_id": "label-histogram", "kind": "extension", "extension": "label-histogram-ext", @@ -288,12 +308,13 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( "min_width": 240, }, ) - assert histogram_panel_response.status_code == 200 + assert histogram_panel_response.json()["ok"] is True - text_panel_response = client.post( - "/api/control/ui/panels", - json={ - "workspace_id": "default", + text_panel_response = _run_control_command( + client, + "workspace.panel.add", + target={"workspace_id": "default"}, + args={ "panel_id": "notes", "kind": "extension", "extension": "notes-ext", @@ -301,13 +322,13 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( "position": "bottom", }, ) - assert text_panel_response.status_code == 200 + assert text_panel_response.json()["ok"] is True - update_panel_response = client.patch( - "/api/control/ui/panels", - json={ - "workspace_id": "default", - "panel_id": "notes", + update_panel_response = _run_control_command( + client, + "workspace.panel.update", + target={"workspace_id": "default", "panel_id": "notes"}, + args={ "title": "Ranked Notes", "position": "right", "reference_panel_id": "label-histogram", @@ -324,13 +345,14 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( }, }, ) - assert update_panel_response.status_code == 200 + assert update_panel_response.json()["ok"] is True target_layout = job_b["result"]["layout_keys"][0] - scatter_panel_response = client.post( - "/api/control/ui/panels", - json={ - "workspace_id": "default", + scatter_panel_response = _run_control_command( + client, + "workspace.panel.add", + target={"workspace_id": "default"}, + args={ "panel_id": "experiment-b-scatter", "title": "Experiment B", "kind": "scatter", @@ -340,7 +362,7 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( "direction": "right", }, ) - assert scatter_panel_response.status_code == 200 + assert scatter_panel_response.json()["ok"] is True set_layout_response = client.post( "/api/control/ui/layout", @@ -421,15 +443,12 @@ def test_runtime_control_api_supports_checkpoint_jobs_panels_and_ui_state( assert panel_asset_response.status_code == 200 assert "Label Histogram" in panel_asset_response.text - remove_scatter_response = client.request( - "DELETE", - "/api/control/ui/panels", - json={ - "workspace_id": "default", - "panel_id": "experiment-b-scatter", - }, + remove_scatter_response = _run_control_command( + client, + "workspace.panel.remove", + target={"workspace_id": "default", "panel_id": "experiment-b-scatter"}, ) - assert remove_scatter_response.status_code == 200 + assert remove_scatter_response.json()["ok"] is True after_remove_response = client.get("/api/runtime") assert after_remove_response.status_code == 200 @@ -446,10 +465,11 @@ def test_runtime_panel_patch_omits_or_clears_placement_fields() -> None: runtime.attach_dataset_instance(workspace_id, _make_dataset(), activate_workspace=True) client = TestClient(create_app(runtime=runtime)) - response = client.post( - "/api/control/ui/panels", - json={ - "workspace_id": workspace_id, + response = _run_control_command( + client, + "workspace.panel.add", + target={"workspace_id": workspace_id}, + args={ "panel_id": "samples", "kind": "builtin", "builtin_panel": "samples", @@ -460,28 +480,26 @@ def test_runtime_panel_patch_omits_or_clears_placement_fields() -> None: "min_width": 240, }, ) - assert response.status_code == 200 + assert response.json()["ok"] is True - response = client.patch( - "/api/control/ui/panels", - json={ - "workspace_id": workspace_id, - "panel_id": "samples", - "props": {"mode": "ranked"}, - }, + response = _run_control_command( + client, + "workspace.panel.update", + target={"workspace_id": workspace_id, "panel_id": "samples"}, + args={"props": {"mode": "ranked"}}, ) - assert response.status_code == 200 + assert response.json()["ok"] is True panel = runtime.get_workspace(workspace_id).ui.custom_panels[0] assert panel.reference_panel_id == "map" assert panel.direction == "right" assert panel.width == 320 assert panel.min_width == 240 - response = client.patch( - "/api/control/ui/panels", - json={ - "workspace_id": workspace_id, - "panel_id": "samples", + response = _run_control_command( + client, + "workspace.panel.update", + target={"workspace_id": workspace_id, "panel_id": "samples"}, + args={ "position": "right", "reference_panel_id": None, "direction": None, @@ -489,7 +507,7 @@ def test_runtime_panel_patch_omits_or_clears_placement_fields() -> None: "min_width": None, }, ) - assert response.status_code == 200 + assert response.json()["ok"] is True panel = runtime.get_workspace(workspace_id).ui.custom_panels[0] assert panel.position == "right" assert panel.reference_panel_id is None @@ -498,6 +516,182 @@ def test_runtime_panel_patch_omits_or_clears_placement_fields() -> None: assert panel.min_width is None +def test_runtime_snapshot_panel_contract_includes_state_and_layout(tmp_path: Path) -> None: + workspace_id = f"panel-contract-{time.time_ns()}" + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + runtime.attach_dataset_instance(workspace_id, _make_dataset(), activate_workspace=True) + runtime.add_runtime_panel( + workspace_id, + panel_id="samples", + kind="builtin", + builtin_panel="samples", + position="right", + width=360, + min_width=260, + props={"mode": "browse"}, + ) + runtime.patch_panel_state( + workspace_id, + "samples", + {"view": {"density": "compact"}}, + source_client_id="test-client", + ) + + snapshot = runtime.snapshot(workspace_id) + panel = snapshot["workspace"]["ui"]["custom_panels"][0] + + assert panel["id"] == "samples" + assert panel["panel_type"] == "samples" + assert panel["source"] == "builtin" + assert panel["props"] == {"mode": "browse"} + assert panel["state"] == {"view": {"density": "compact"}} + assert panel["state_revision"] == 1 + assert panel["layout"] == { + "position": "right", + "reference_panel_id": None, + "direction": None, + "width": 360, + "height": None, + "min_width": 260, + "min_height": None, + "max_width": None, + "max_height": None, + } + assert snapshot["workspace"]["ui"]["panels"]["samples"] == { + "state": {"view": {"density": "compact"}}, + "state_revision": 1, + } + + +def test_extension_panel_definition_drives_runtime_panel_defaults( + tmp_path: Path, +) -> None: + extension_dir = tmp_path / "readout-ext" + extension_dir.mkdir() + (extension_dir / "extension.toml").write_text( + """ +name = "readout" + +[[panels]] +id = "summary" +title = "Summary" +label = "Summary card" +panel_type = "analysis.summary" +position = "right" +file = "panel.js" +commands = ["workspace.panel.state.get", "custom.refresh"] +queries = ["samples.query"] +allow_multiple = false +icon = "chart" +category = "analysis" + +[panels.default_props] +mode = "compact" + +[panels.default_state] +collapsed = false +threshold = 0.75 + +[panels.default_layout] +position = "bottom" +height = 240 +min_height = 180 +""".strip() + + "\n", + encoding="utf-8", + ) + (extension_dir / "panel.js").write_text( + "export default function Panel() { return null; }\n", + encoding="utf-8", + ) + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + client = TestClient(create_app(runtime=runtime)) + + installation = runtime.install_extension("default", extension_dir, add_panels=True) + definition = runtime.get_panel_definition( + "analysis.summary", + source="extension", + extension="readout", + ) + assert definition is not None + + definition_payload = definition.to_dict() + assert definition_payload == installation.to_dict()["panel_definitions"][0] + assert definition_payload["label"] == "Summary card" + assert definition_payload["default_props"] == {"mode": "compact"} + assert definition_payload["default_state"] == { + "collapsed": False, + "threshold": 0.75, + } + assert definition_payload["default_layout"] == { + "position": "bottom", + "height": 240, + "min_height": 180, + } + assert definition_payload["commands"] == ["workspace.panel.state.get", "custom.refresh"] + assert definition_payload["queries"] == ["samples.query"] + assert definition_payload["allow_multiple"] is False + + workspace = runtime.get_workspace("default") + panel = workspace.ui.custom_panels[0] + assert panel.id == "summary" + assert panel.panel_type == "analysis.summary" + assert panel.source == "extension" + assert panel.extension == "readout" + assert panel.extension_panel == "summary" + assert panel.position == "bottom" + assert panel.height == 240 + assert panel.min_height == 180 + assert panel.props == {"mode": "compact"} + assert workspace.ui.panels["summary"].state == { + "collapsed": False, + "threshold": 0.75, + } + assert workspace.ui.panels["summary"].state_revision == 0 + + snapshot = runtime.snapshot("default") + definitions_by_type = { + item["panel_type"]: item for item in snapshot["panel_definitions"] + } + assert {"samples", "scatter", "analysis.summary"}.issubset(definitions_by_type) + assert snapshot["workspace"]["ui"]["custom_panels"][0]["state"] == { + "collapsed": False, + "threshold": 0.75, + } + + response = client.get("/api/panel-definitions") + assert response.status_code == 200 + endpoint_definitions = { + item["panel_type"]: item for item in response.json()["panel_definitions"] + } + assert endpoint_definitions["analysis.summary"] == definition_payload + + runtime.remove_custom_panel("default", "summary") + runtime.add_runtime_panel( + "default", + panel_id="summary-manual", + kind="extension", + extension="readout", + extension_panel="summary", + ) + manual_panel = runtime.get_workspace("default").ui.custom_panels[0] + assert manual_panel.id == "summary-manual" + assert manual_panel.position == "bottom" + assert manual_panel.height == 240 + assert manual_panel.min_height == 180 + assert manual_panel.props == {"mode": "compact"} + assert runtime.get_workspace("default").ui.panels["summary-manual"].state == { + "collapsed": False, + "threshold": 0.75, + } + + def test_runtime_panel_builder_matches_public_ui_compilation(tmp_path: Path) -> None: dataset = _make_dataset() sample_ids = [sample.id for sample in dataset] @@ -520,7 +714,7 @@ def test_runtime_panel_builder_matches_public_ui_compilation(tmp_path: Path) -> ) runtime.install_extension("default", tmp_path / "readout-ext") - rest_specs = [ + runtime_specs = [ runtime.build_custom_panel( "default", panel_id="map", @@ -578,13 +772,13 @@ def test_runtime_panel_builder_matches_public_ui_compilation(tmp_path: Path) -> workspace_id="default", ) - assert [spec.to_dict() for spec in rest_specs] == [spec.to_dict() for spec in ui_specs] - assert rest_specs[0].geometry == "euclidean" - assert rest_specs[0].layout_dimension == 2 - assert rest_specs[2].module_file == str(panel_file.resolve()) + assert [spec.to_dict() for spec in runtime_specs] == [spec.to_dict() for spec in ui_specs] + assert runtime_specs[0].geometry == "euclidean" + assert runtime_specs[0].layout_dimension == 2 + assert runtime_specs[2].module_file == str(panel_file.resolve()) -def test_rest_scatter_panel_requires_existing_layout(tmp_path: Path) -> None: +def test_panel_add_command_requires_existing_layout(tmp_path: Path) -> None: runtime = HyperViewRuntime( provider_registry=ProviderRegistry(tmp_path / "providers.json"), workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), @@ -592,10 +786,11 @@ def test_rest_scatter_panel_requires_existing_layout(tmp_path: Path) -> None: runtime.attach_dataset_instance("default", _make_dataset()) client = TestClient(create_app(runtime=runtime)) - response = client.post( - "/api/control/ui/panels", - json={ - "workspace_id": "default", + response = _run_control_command( + client, + "workspace.panel.add", + target={"workspace_id": "default"}, + args={ "panel_id": "missing-layout", "title": "Missing Layout", "kind": "scatter", @@ -603,8 +798,15 @@ def test_rest_scatter_panel_requires_existing_layout(tmp_path: Path) -> None: }, ) - assert response.status_code == 404 - assert response.json()["detail"] == "Layout not found: missing-layout" + assert response.json() == { + "ok": False, + "command": "workspace.panel.add", + "result": {}, + "error": { + "code": "not_found", + "message": "Layout not found: missing-layout", + }, + } def test_runtime_embedding_job_uses_injected_provider_registry(tmp_path: Path) -> None: @@ -661,7 +863,8 @@ def test_health_reports_package_version() -> None: assert response.json()["version"] == __version__ -def test_ui_similarity_query_is_explicit_and_cleared_with_selection() -> None: +def test_ui_similarity_query_is_explicit_and_cleared_with_selection(tmp_path: Path) -> None: + workspace_id = f"similarity-ui-{time.time_ns()}" dataset = _make_dataset() ids = [sample.id for sample in dataset] layout_key = dataset.set_coords( @@ -671,15 +874,19 @@ def test_ui_similarity_query_is_explicit_and_cleared_with_selection() -> None: ) space_key = dataset.list_layouts()[0].space_key - runtime = HyperViewRuntime() - runtime.attach_dataset_instance("default", dataset) - runtime.set_selection("default", ["sample-2"]) + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + runtime.attach_dataset_instance(workspace_id, dataset, activate_workspace=True) + runtime.set_selection(workspace_id, ["sample-2"]) client = TestClient(create_app(runtime=runtime)) - response = client.post( - "/api/control/ui/similarity", - json={ - "workspace_id": "default", + response = _run_control_command( + client, + "panel.samples.retrieval.set-anchor", + target={"workspace_id": workspace_id}, + args={ "sample_id": "sample-2", "layout_key": layout_key, "k": 12, @@ -687,31 +894,38 @@ def test_ui_similarity_query_is_explicit_and_cleared_with_selection() -> None: }, ) - assert response.status_code == 200 + assert response.json()["ok"] is True ui = response.json()["workspace"]["ui"] assert ui["selected_ids"] == [] - assert ui["similarity_query"] == { + expected_retrieval = { "anchor_sample_id": "sample-2", "layout_key": layout_key, "space_key": space_key, "k": 12, "source": "test", } + samples_state = ui["panels"]["samples"]["state"] + assert samples_state["mode"] == "retrieval" + assert samples_state["retrieval"] == expected_retrieval + assert "similarity_query" not in ui + assert samples_state["collection"]["kind"] == "neighbors" selection_response = client.post( "/api/control/ui/selection", - json={"workspace_id": "default", "sample_ids": ["sample-3"]}, + json={"workspace_id": workspace_id, "sample_ids": ["sample-3"]}, ) assert selection_response.status_code == 200 - assert selection_response.json()["workspace"]["ui"]["similarity_query"] is None + selection_ui = selection_response.json()["workspace"]["ui"] + assert "similarity_query" not in selection_ui + assert selection_ui["panels"]["samples"]["state"] == {} - clear_response = client.request( - "DELETE", - "/api/control/ui/similarity", - json={"workspace_id": "default"}, + clear_response = _run_control_command( + client, + "panel.samples.retrieval.clear", + target={"workspace_id": workspace_id}, ) - assert clear_response.status_code == 200 - assert clear_response.json()["workspace"]["ui"]["similarity_query"] is None + assert clear_response.json()["ok"] is True + assert "similarity_query" not in clear_response.json()["workspace"]["ui"] def test_workspace_load_drops_legacy_similarity_anchor_selection(tmp_path: Path) -> None: @@ -745,8 +959,12 @@ def test_workspace_load_drops_legacy_similarity_anchor_selection(tmp_path: Path) assert workspace is not None assert workspace.ui.selected_ids == [] - assert workspace.ui.similarity_query is not None - assert workspace.ui.similarity_query.anchor_sample_id == "sample-2" + samples_retrieval = workspace.ui.panels["samples"].state["retrieval"] + assert samples_retrieval["anchor_sample_id"] == "sample-2" + assert workspace.ui.panels["samples"].state == { + "mode": "retrieval", + "retrieval": samples_retrieval, + } def test_ui_similarity_query_rejects_mismatched_layout_and_space() -> None: @@ -762,21 +980,29 @@ def test_ui_similarity_query_rejects_mismatched_layout_and_space() -> None: runtime.attach_dataset_instance("default", dataset) client = TestClient(create_app(runtime=runtime)) - response = client.post( - "/api/control/ui/similarity", - json={ - "workspace_id": "default", + response = _run_control_command( + client, + "panel.samples.retrieval.set-anchor", + target={"workspace_id": "default"}, + args={ "sample_id": "sample-2", "layout_key": layout_key, "space_key": "different-space", }, ) - assert response.status_code == 400 - assert response.json()["detail"] == "space_key does not match the requested layout_key" + assert response.json() == { + "ok": False, + "command": "panel.samples.retrieval.set-anchor", + "result": {}, + "error": { + "code": "validation_error", + "message": "space_key does not match the requested layout_key", + }, + } -def test_ui_state_patch_batches_layout_selection_and_similarity() -> None: +def test_ui_state_patch_batches_layout_and_selection() -> None: dataset = _make_dataset() ids = [sample.id for sample in dataset] layout_key = dataset.set_coords( @@ -784,10 +1010,15 @@ def test_ui_state_patch_batches_layout_selection_and_similarity() -> None: ids, [[float(index), float(index % 2)] for index, _ in enumerate(ids)], ) - space_key = dataset.list_layouts()[0].space_key - runtime = HyperViewRuntime() runtime.attach_dataset_instance("default", dataset) + runtime.patch_ui_state( + "default", + set_active_layout=True, + active_layout_key=layout_key, + set_selection=True, + selected_ids=["sample-1"], + ) before_version = runtime.version client = TestClient(create_app(runtime=runtime)) @@ -796,16 +1027,9 @@ def test_ui_state_patch_batches_layout_selection_and_similarity() -> None: json={ "workspace_id": "default", "set_active_layout": True, - "active_layout_key": layout_key, + "active_layout_key": None, "set_selection": True, - "selected_ids": ["sample-2"], - "set_similarity_query": True, - "similarity_query": { - "sample_id": "sample-2", - "layout_key": layout_key, - "k": 12, - "source": "test", - }, + "selected_ids": ["sample-5"], }, ) @@ -813,15 +1037,9 @@ def test_ui_state_patch_batches_layout_selection_and_similarity() -> None: assert runtime.version == before_version + 1 assert runtime.version_source_client_id is None ui = response.json()["workspace"]["ui"] - assert ui["active_layout_key"] == layout_key - assert ui["selected_ids"] == [] - assert ui["similarity_query"] == { - "anchor_sample_id": "sample-2", - "layout_key": layout_key, - "space_key": space_key, - "k": 12, - "source": "test", - } + assert ui["active_layout_key"] is None + assert ui["selected_ids"] == ["sample-5"] + assert "similarity_query" not in ui sourced_response = client.patch( "/api/control/ui/state", @@ -829,14 +1047,14 @@ def test_ui_state_patch_batches_layout_selection_and_similarity() -> None: "workspace_id": "default", "client_id": "client-1", "set_active_layout": True, - "active_layout_key": None, + "active_layout_key": layout_key, }, ) assert sourced_response.status_code == 200 assert runtime.version_source_client_id == "client-1" -def test_ui_state_patch_similarity_clears_selection() -> None: +def test_ui_state_patch_rejects_samples_retrieval_fields() -> None: dataset = _make_dataset() ids = [sample.id for sample in dataset] layout_key = dataset.set_coords( @@ -847,6 +1065,7 @@ def test_ui_state_patch_similarity_clears_selection() -> None: runtime = HyperViewRuntime() runtime.attach_dataset_instance("default", dataset) + initial_selected_ids = list(runtime.get_workspace("default").ui.selected_ids) client = TestClient(create_app(runtime=runtime)) response = client.patch( @@ -863,10 +1082,10 @@ def test_ui_state_patch_similarity_clears_selection() -> None: }, ) - assert response.status_code == 200 - ui = response.json()["workspace"]["ui"] - assert ui["selected_ids"] == [] - assert ui["similarity_query"]["anchor_sample_id"] == "sample-2" + assert response.status_code == 422 + workspace = runtime.get_workspace("default") + assert workspace.ui.selected_ids == initial_selected_ids + assert runtime.get_samples_retrieval_query("default") is None def test_sample_responses_include_media_url_and_content_endpoint_serves_file( @@ -927,6 +1146,29 @@ def test_sample_responses_include_media_url_and_content_endpoint_serves_file( assert inline_batch_response.json()["samples"][0]["thumbnail"] is not None +def test_samples_endpoint_filters_missing_labels() -> None: + dataset = Dataset("runtime_missing_labels", persist=False) + dataset.add_samples( + [ + Sample(id="sample-1", filepath="/virtual/sample-1.png", label="cat"), + Sample(id="sample-2", filepath="/virtual/sample-2.png"), + Sample(id="sample-3", filepath="/virtual/sample-3.png", label="dog"), + ] + ) + + runtime = HyperViewRuntime() + runtime.attach_dataset_instance("default", dataset) + client = TestClient(create_app(runtime=runtime)) + + response = client.get("/api/samples", params={"missing_label": "true"}) + + assert response.status_code == 200 + payload = response.json() + assert payload["total"] == 1 + assert [sample["id"] for sample in payload["samples"]] == ["sample-2"] + assert payload["samples"][0]["label"] is None + + def test_samples_batch_allows_exact_id_reads_larger_than_page_limit() -> None: sample_count = MAX_SAMPLE_PAGE_SIZE + 1 dataset = Dataset("runtime_large_batch", persist=False) @@ -995,23 +1237,25 @@ def test_set_workspace_dataset_clears_dataset_scoped_ui_state(tmp_path: Path, mo runtime.attach_dataset_instance("default", _make_dataset()) runtime.set_active_layout("default", "layout-a") runtime.set_selection("default", ["sample-1", "sample-3"]) - runtime.set_similarity_query( + runtime.set_samples_retrieval( "default", runtime.resolve_similarity_query("default", "sample-1", source="test"), ) + assert runtime.get_workspace("default").ui.panels["samples"].state["mode"] == "retrieval" workspace = runtime.set_workspace_dataset("default", "second-dataset") assert workspace.dataset_name == "second-dataset" assert workspace.ui.active_layout_key is None assert workspace.ui.selected_ids == [] - assert workspace.ui.similarity_query is None + assert workspace.ui.panels == {} snapshot = runtime.snapshot() assert snapshot["workspace"]["dataset_name"] == "second-dataset" assert snapshot["workspace"]["ui"]["active_layout_key"] is None assert snapshot["workspace"]["ui"]["selected_ids"] == [] - assert snapshot["workspace"]["ui"]["similarity_query"] is None + assert "similarity_query" not in snapshot["workspace"]["ui"] + assert snapshot["workspace"]["ui"]["panels"] == {} def test_runtime_panel_module_src_changes_when_module_file_changes( diff --git a/tests/test_selection_3d_lasso.py b/tests/test_selection_3d_lasso.py index c9e2a37..514ea7c 100644 --- a/tests/test_selection_3d_lasso.py +++ b/tests/test_selection_3d_lasso.py @@ -89,3 +89,41 @@ def test_3d_lasso_label_filter_excludes_hidden_occluders() -> None: ) assert selected_ids == ["back-cat"] + + +def test_3d_lasso_missing_label_filter_excludes_labeled_occluders() -> None: + coords = np.array( + [ + [0.0, 0.0, 1.0], + [0.0, 0.0, 0.0], + ], + dtype=np.float32, + ) + view = OrbitViewState3D( + yaw=0.0, + pitch=0.0, + distance=4.0, + target_x=0.0, + target_y=0.0, + target_z=0.0, + ortho_scale=1.5, + ) + + mvp = build_mvp_for_orbit(view, coords, 200, 200) + screen_x, screen_y, _, _ = project_points_3d_to_screen(mvp, coords, 200, 200) + polygon = _square_around(float(screen_x[1]), float(screen_y[1])) + + selected_ids = select_ids_for_3d_lasso( + ids=["front-dog", "back-unlabeled"], + labels=["dog", None], + coords=coords, + geometry="euclidean", + polygon=polygon, + view=view, + viewport_width=200, + viewport_height=200, + label_filter=None, + missing_label_filter=True, + ) + + assert selected_ids == ["back-unlabeled"] diff --git a/tests/test_static_export.py b/tests/test_static_export.py new file mode 100644 index 0000000..80a2dab --- /dev/null +++ b/tests/test_static_export.py @@ -0,0 +1,115 @@ +from __future__ import annotations + +import json +from pathlib import Path + +import numpy as np +from PIL import Image + +from hyperview import Dataset, Session +from hyperview.core.sample import Sample +from hyperview.runtime import HyperViewRuntime, ProviderRegistry, WorkspaceRegistry +from hyperview.static_export import export_runtime_workspace + + +def _write_extension(folder: Path) -> None: + folder.mkdir(parents=True) + (folder / "extension.toml").write_text( + "\n".join( + [ + 'name = "export-demo"', + "", + "[[panels]]", + 'id = "readout"', + 'title = "Readout"', + 'position = "right"', + 'file = "panel.js"', + ] + ), + encoding="utf-8", + ) + (folder / "panel.js").write_text( + "export default function Panel() { return null; }\n", + encoding="utf-8", + ) + + +def _make_runtime(tmp_path: Path) -> HyperViewRuntime: + dataset = Dataset("static_export_dataset", persist=False) + sample_ids: list[str] = [] + for index, label in enumerate(["cat", "dog", "cat"]): + image_path = tmp_path / f"sample-{index}.png" + Image.new("RGB", (12 + index, 10 + index), (index * 40, 40, 180)).save(image_path) + sample_id = f"sample-{index}" + sample_ids.append(sample_id) + dataset.add_sample( + Sample( + id=sample_id, + filepath=str(image_path), + label=label, + metadata={"index": index}, + ) + ) + layout_key = dataset.set_coords( + "euclidean", + sample_ids, + np.asarray([[0.0, 0.0], [1.0, 0.5], [2.0, 0.25]], dtype=np.float32), + ) + + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + runtime.attach_dataset_instance("demo", dataset, activate_workspace=True) + runtime.set_active_layout("demo", layout_key) + + extension_dir = tmp_path / "export-demo-extension" + _write_extension(extension_dir) + runtime.install_extension("demo", extension_dir, add_panels=True) + return runtime + + +def test_static_export_writes_bundle_snapshot_samples_media_and_flag(tmp_path: Path) -> None: + runtime = _make_runtime(tmp_path) + out_dir = tmp_path / "bundle" + + result = export_runtime_workspace(runtime, "demo", out_dir) + + assert result.workspace_id == "demo" + assert result.num_samples == 3 + assert result.num_layouts == 1 + + index_html = (out_dir / "index.html").read_text(encoding="utf-8") + assert "window.__HYPERVIEW_STATIC__ = true;" in index_html + + snapshot = json.loads((out_dir / "api" / "runtime.json").read_text(encoding="utf-8")) + assert snapshot["workspace"]["id"] == "demo" + assert snapshot["workspace"]["ui"]["active_layout_key"] + assert snapshot["panel_definitions"] + + samples_index = json.loads((out_dir / "api" / "samples" / "index.json").read_text(encoding="utf-8")) + assert samples_index["total"] == 3 + shard = json.loads( + (out_dir / "api" / "samples" / samples_index["shards"][0]).read_text(encoding="utf-8") + ) + assert shard["samples"][0]["media_url"] == "/api/samples/sample-0/content" + assert shard["samples"][0]["thumbnail_url"] == "/api/samples/sample-0/thumbnail" + + assert (out_dir / "api" / "samples" / "sample-0" / "content").is_file() + assert (out_dir / "api" / "samples" / "sample-0" / "thumbnail").is_file() + assert (out_dir / "media" / "samples" / "sample-0" / "sample-0.png").is_file() + assert (out_dir / "media" / "thumbnails" / "sample-0.jpg").is_file() + assert (out_dir / "api" / "embeddings" / "default.json").is_file() + assert (out_dir / "api" / "panels" / "content" / "demo" / "readout" / "panel.js").is_file() + + +def test_session_export_uses_runtime_workspace(tmp_path: Path) -> None: + runtime = _make_runtime(tmp_path) + session = Session(runtime, "127.0.0.1", 6262) + out_dir = tmp_path / "session-bundle" + + payload = session.export(out_dir, workspace_id="demo") + + assert payload["workspace_id"] == "demo" + assert Path(payload["output_dir"]).is_dir() + assert (out_dir / "hyperview-static.json").is_file() diff --git a/tests/test_text_retrieval.py b/tests/test_text_retrieval.py new file mode 100644 index 0000000..96cec07 --- /dev/null +++ b/tests/test_text_retrieval.py @@ -0,0 +1,116 @@ +from __future__ import annotations + +from pathlib import Path +from unittest.mock import patch + +import numpy as np + +from hyperview import Dataset +from hyperview.control import CommandEnvelope, ControlService, create_default_command_registry +from hyperview.core.sample import Sample +from hyperview.runtime import HyperViewRuntime, ProviderRegistry, SimilarityQueryState, WorkspaceRegistry +from hyperview.storage.schema import dict_to_sample, sample_to_dict + + +def _service(tmp_path: Path) -> ControlService: + runtime = HyperViewRuntime( + provider_registry=ProviderRegistry(tmp_path / "providers.json"), + workspace_registry=WorkspaceRegistry(tmp_path / "workspaces.json"), + ) + dataset = Dataset("text_retrieval", persist=False) + dataset.add_sample( + Sample( + id="s0", + filepath="/virtual/s0.png", + label="dog", + text="a dog in the park", + modality="multimodal", + ) + ) + dataset.add_sample( + Sample( + id="s1", + filepath="/virtual/s1.png", + label="cat", + text="a cat on a sofa", + modality="multimodal", + ) + ) + dataset._storage.ensure_space( + model_id="openai/clip-vit-base-patch32", + dim=3, + config={"provider": "embed-anything", "geometry": "euclidean", "modality": "multimodal"}, + space_key="clip_space", + ) + dataset._storage.add_embeddings( + "clip_space", + ["s0", "s1"], + np.asarray([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], dtype=np.float32), + ) + runtime.attach_dataset_instance("default", dataset) + return ControlService(runtime, create_default_command_registry()) + + +def test_sample_schema_roundtrip_includes_text_and_modality() -> None: + sample = Sample( + id="abc", + filepath="/tmp/abc.png", + label="label", + text="hello world", + modality="multimodal", + ) + restored = dict_to_sample(sample_to_dict(sample)) + assert restored.text == "hello world" + assert restored.modality == "multimodal" + + +def test_similarity_query_state_supports_text_only() -> None: + query = SimilarityQueryState.from_dict( + {"query_text": "a dog playing", "k": 12, "space_key": "clip_space"} + ) + assert query is not None + assert query.query_text == "a dog playing" + assert query.anchor_sample_id is None + assert query.k == 12 + + +def test_control_command_sets_text_retrieval_state(tmp_path: Path) -> None: + service = _service(tmp_path) + result = service.run( + CommandEnvelope( + command="panel.samples.retrieval.set-text-query", + target={"workspace_id": "default"}, + args={"query_text": "a dog in the park", "space_key": "clip_space", "k": 5}, + ) + ) + assert result.ok is True + retrieval = result.workspace["ui"]["panels"]["samples"]["state"]["retrieval"] + assert retrieval["query_text"] == "a dog in the park" + assert retrieval["space_key"] == "clip_space" + assert result.workspace["ui"]["panels"]["samples"]["state"]["collection"]["kind"] == "search" + + +def test_dataset_find_similar_by_text_uses_encoded_vector(tmp_path: Path) -> None: + dataset = Dataset("text_query", persist=False) + dataset.add_sample(Sample(id="s0", filepath="/virtual/s0.png", text="dog")) + dataset.add_sample(Sample(id="s1", filepath="/virtual/s1.png", text="cat")) + dataset._storage.ensure_space( + model_id="openai/clip-vit-base-patch32", + dim=3, + config={"provider": "embed-anything", "geometry": "euclidean", "modality": "multimodal"}, + space_key="clip_space", + ) + dataset._storage.add_embeddings( + "clip_space", + ["s0", "s1"], + np.asarray([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]], dtype=np.float32)[:2], + ) + + query_vector = np.asarray([0.95, 0.05, 0.0], dtype=np.float32) + with patch("hyperview.embeddings.engine.get_engine") as get_engine: + engine = get_engine.return_value + engine.embed_texts.return_value = np.asarray([query_vector], dtype=np.float32) + results = dataset.find_similar_by_text("dog in park", k=1, space_key="clip_space") + + assert len(results) == 1 + assert results[0][0].id == "s0"