Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,39 @@ auto-generated per-PR notes; this file is the curated, human-readable history.

## [Unreleased]

### Added
- **Portable open/import/export + a transactional import planner** (#287, Dashboard
v1 Phase 5). The File menu's ambiguous `Append` is gone, replaced by
resource-oriented **Import queries / Import Dashboard / Replace workspace** and
**Export Dashboard / Export workspace** operations, all writing the one canonical
`altinity-sql-browser/portable-bundle` (`PortableBundleV1`) interchange format
through the deterministic canonical encoder. Export never mutates workspace
identity or Dashboard revision; **Export Dashboard** emits a dependency closure
(the Dashboard plus exactly its referenced queries), **Export workspace** emits
the whole catalog plus the zero-or-one Dashboard. Import runs a transactional
planner (`src/workspace/import-planner.ts`): parse → schema validation → select
Dashboard → dependency closure → conflict detection → decisions → complete ID
remapping → central reference rewrite → candidate workspace → whole-workspace
validation → atomic commit, with conflict actions **use-existing** (automatic
only on canonical equality), **copy** (fresh id + full reference rewrite),
**replace**, and **skip** (skipping a required Dashboard dependency invalidates
the Dashboard import rather than silently dropping a tile). A multi-Dashboard
bundle asks which Dashboard to import. Legacy Library v1/v2 files remain
**importable** (normalized to an in-memory bundle with `dashboards: []`); no new
Library-only JSON is written.

### Changed
- **The `StoredWorkspaceV1` aggregate is now the single source of truth for the
saved-query collection** (#287). All query CRUD (create/edit/rename/delete/star)
commits the whole workspace through the atomic `WorkspaceRepository`
(validate-before-publish per #280) instead of the flat `asb:saved` localStorage
key, which is retired (read once only as the legacy-migration source); boot loads
and projects the aggregate. Saved-query writes are serialized so two rapid
concurrent edits can't interleave and resurrect a just-deleted query. The
external-bundle **trust preflight** and durable cross-tab transport are deferred
to Phase 6 (#288); every Phase-5 file operation only commits to the workspace and
never executes queries.

### Fixed
- **Dashboard filter strips no longer wrap, the visible "Clear all" control and
"N active" count are both removed, and the layout switcher moved to the
Expand Down
65 changes: 38 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ See the [**feature tour, deployment guide and screenshots**](https://docs.altini
on the project site. Try it live on the Antalya demo cluster: **https://antalya.demo.altinity.cloud/sql**.
The [**ontime chart demo**](docs/ONTIME-CHART-DEMO.md) is a ready-made library of 10
queries (load [`examples/ontime-charts.json`](examples/ontime-charts.json) via
**File ▾ → Open**) that walks through every chart type and feature against the public
**File ▾ → Import queries**) that walks through every chart type and feature against the public
`ontime` flight dataset. The [**system explorer demo**](docs/SYSTEM-EXPLORER-DEMO.md)
is a 14-query library (load [`examples/system-explorer-charts.json`](examples/system-explorer-charts.json)
via **File ▾ → Append**) that introspects ClickHouse's own `system` database —
via **File ▾ → Import queries**) that introspects ClickHouse's own `system` database —
running queries, merges/replication health, and historical query/part/error
activity — with a shared From/To filter driving every time-ranged Dashboard tile
at once.
Expand Down Expand Up @@ -93,7 +93,7 @@ with no per-favorite name, description, or statistics footer, spanning every
Dashboard layout (Full width/Report/2/3 columns). Consecutive explicit KPI
favorites merge into one shared band. The complete
[`kpi-panel.json`](examples/kpi-panel.json) Library example can be opened from
**File ▾ → Open** to see both.
**File ▾ → Import queries** to see both.
When constructing a named tuple from expressions, either enable alias-derived
member names for the query:

Expand Down Expand Up @@ -422,40 +422,51 @@ read; DDL secrets remain masked unless the role separately holds

## Saved queries & the Library

Queries you save (★ **Save** next to Run, or `⌘S`) land in the sidebar **★ Library**
Queries you save (★ **Save** next to Run, or `⌘S`) land in the sidebar **★ Queries**
panel. Each carries a name, an optional **description**, and — when set — its
remembered result view and chart config. Saving or editing a query opens a small
form with both a name and a description field; the description shows under the
row and is included in Markdown/SQL exports.

The whole collection is treated as a **document — the Library** — with a name and
an unsaved-changes dot, managed from the header **File ▾** menu:
The queries plus the zero-or-one editable Dashboard form one **current workspace**
(a `StoredWorkspaceV1` aggregate). The workspace is persisted **atomically** in the
browser (IndexedDB) — a reload restores queries, Dashboard, layout, and workspace
name together; every saved-query edit commits the whole workspace and only
publishes once persistence succeeds. The header **File ▾** menu carries a name, an
unsaved-changes dot (changes since the last export or import), and these
resource-oriented operations (#287):

- **New Library** — clears to an empty, default-named library (confirms first
- **New workspace…** — clears to an empty, default-named workspace (confirms first
when non-empty). Open editor tabs are unaffected.
- **Save JSON** (`.json`) — downloads the whole Library in the versioned
`altinity-sql-browser/saved-queries` envelope. Version 2 stores each query as
`{id, sql, specVersion, spec}`: `spec` is the complete, lossless query
definition (`name`, `description`, `favorite`, `view`, `panel`, `dashboard`,
and future extension fields). New files include a canonical `$schema` hint
and RFC 3339 `exportedAt` timestamp and validate against the
[complete Library contract](docs/library-json-schema.md). The filename derives from the Library name;
saving clears the unsaved-changes dot. Version 1 Library files remain
importable and are upgraded in memory; new exports always use version 2.
- **Open… / Append…** — load a `.json` file: Open swaps the Library and
adopts the file's base name (confirms when the current Library is non-empty);
Append merges via the existing dedupe and reports `Added N · updated N ·
skipped N`. **JSON is the only importable format**, and imported SQL is never
run automatically.
- **Import queries…** — merge a file's queries into the current collection. When
an incoming query's id collides with an existing one, a **conflict dialog**
offers a default action plus per-row overrides — *use existing*, *copy* (import
under a fresh id), *replace*, or *skip*; a byte-identical incoming query is
reused automatically. The Dashboard is untouched (imported favorite flags never
add tiles).
- **Import Dashboard…** — replace the current Dashboard with one from a file
(confirms first when a Dashboard already exists), importing only its referenced
queries. A multi-Dashboard file asks which one.
- **Replace workspace…** — atomically replace **both** the query collection and
the Dashboard from a file (confirms first). A multi-Dashboard file asks which
Dashboard to adopt (or none).
- **Export Dashboard… / Export workspace…** (`.json`) — write the one canonical
**`altinity-sql-browser/portable-bundle`** interchange format. *Export Dashboard*
emits the selected Dashboard plus exactly its dependency-closure of queries;
*Export workspace* emits every saved query (catalog order) plus the zero-or-one
Dashboard. Both use the deterministic canonical encoder and never mutate the
workspace's identity or Dashboard revision. Legacy Library v1/v2 files remain
**importable** (decoded to an in-memory bundle); no new Library-only JSON is
written. See the [schema contracts](docs/library-json-schema.md). Imported SQL
is never run automatically.
- **Share / publish** — **Download Markdown** (`.md`, a `### heading` + fenced
` ```sql ` cookbook) and **Download SQL** (`.sql`, `/* name + description */`
comment blocks, `;`-delimited). Both are **one-way** — lossy by design (no ids
or Spec metadata), so JSON stays the canonical round-trip format.
or Spec metadata), so the portable bundle stays the canonical round-trip format.

The Library name is editable inline (click it in the header) and is persisted
separately from the queries. The **•** dot appears after any change that hasn't
been written to a file yet (save/rename/delete/favorite/append/rename) and clears
on Save JSON / Open / New.
The workspace name is editable inline (click it in the header). The **•** dot
appears after any change not yet written to a file and clears on export / import /
New workspace.

### Dashboard Filter sources

Expand All @@ -471,7 +482,7 @@ ordinary parameter field when a source, consumer type, or provider conflicts.
Filter sources run and reconcile saved values before any Panel query starts.

The complete [`query-log-explorer.json`](examples/query-log-explorer.json)
Library example (load via **File ▾ → Append**) demonstrates every filter
Library example (load via **File ▾ → Import queries**) demonstrates every filter
variant against `system.query_log` on any cluster: three Filter sources, one
per option shape (`Array(Tuple(value, label))`, `Map(String, String)`, plain
`Array(T)`), alongside plain auto-detected numeric/text fields — a KPI panel,
Expand Down
29 changes: 29 additions & 0 deletions docs/ADR-0001-reactivity.md
Original file line number Diff line number Diff line change
Expand Up @@ -338,3 +338,32 @@ re-opens the decision via a fresh spike:

If re-opened, the candidate is **Preact** (`spike/preact-schema` stands as
evidence, +6.8 KB gzip), never React (+45 KB).

## Addendum — the workspace aggregate is the saved-query source of truth (#287, Dashboard v1 Phase 5)

Phase 5 made the persisted `StoredWorkspaceV1` aggregate (via the atomic
`WorkspaceRepository`, IndexedDB) the single source of truth for the saved-query
collection, retiring the flat `asb:saved` localStorage write path (read once only
as the legacy-migration source). This is a state-flow change worth recording
against this ADR because it re-shapes how `state.savedQueries` relates to
reactivity:

- `state.savedQueries` is now a **projection** of the committed workspace, not a
directly-mutated array. Boot loads the aggregate and projects it (queries,
Dashboard, workspace id/name); every file operation commits and re-projects
through one shared `app.applyCommittedWorkspace` helper.
- All query CRUD is **strict async, validate-before-publish** (#280): each op
computes a candidate, `await`s `WorkspaceRepository.commit` (which validates the
whole candidate, then atomically replaces the record), and only then mutates
in-memory state + tabs. A failed commit mutates nothing and keeps the draft
dirty. The array is not a signal (it never was — see #276); the render surfaces
repaint explicitly after a commit, exactly as the pre-#287 synchronous code did.
- Because commits are async, saved-query writes are **serialized** per app
(`app.serializeWrite`) so two overlapping ops can't each build a candidate from
the same stale snapshot and have the later commit resurrect a just-deleted query
or clobber a concurrent edit. This is last-commit-wins within one tab; #280's
multi-tab "last successful commit wins, no compare-and-swap" policy is unchanged.

No framework pull here: the change is about persistence atomicity and validation,
not a render model. The imperative-islands + signals-for-invalidation decision
stands.
2 changes: 1 addition & 1 deletion docs/ONTIME-CHART-DEMO.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dataset (`ontime`, ~230M rows, 1987–2025) on the Antalya demo cluster.
3. In the header, click **File ▾ → Open…** and pick the file. The library is renamed
**ontime-charts** and fills with 10 saved queries (confirm the replace if you already
had queries saved).
4. Click any query in the **Library** panel — it runs and opens straight into its chart.
4. Click any query in the **Queries** panel — it runs and opens straight into its chart.
Switch **Table / JSON / Chart** at the top of the results, or change the **Type / X / Y /
Series** dropdowns to re-encode any chart live.

Expand Down
56 changes: 38 additions & 18 deletions src/application/saved-query-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ import { queryPanel, withQuerySpec } from '../core/saved-query.js';
import { isQuerylessPanel } from '../core/panel-cfg.js';
import { encodeShare } from '../core/share.js';
import type { SavedQueryV2 } from '../generated/json-schema.types.js';
import type { WorkspaceRepository } from '../workspace/workspace-repository.js';
import type { WorkspaceDiagnostic } from '../dashboard/model/workspace-diagnostics.js';

// ── Construction deps ────────────────────────────────────────────────────────

Expand All @@ -49,7 +51,8 @@ import type { SavedQueryV2 } from '../generated/json-schema.types.js';
* `createSavedQuery`/`commitSavedQuery`/`recordHistory` (state.ts, all three
* narrowed to their own exact reads) need between them. */
export interface SavedQueryServiceDeps {
state: Pick<AppState, 'savedQueries' | 'resultView' | 'libraryDirty' | 'history'>;
state: Pick<AppState,
'savedQueries' | 'resultView' | 'libraryDirty' | 'history' | 'libraryName' | 'workspaceId' | 'dashboard'>;
saveJSON: SaveJSON;
/** `createSavedQuery`'s minting timestamp — a genuine wall-clock read
* (production wires this to `() => Date.now()`, called at the exact
Expand All @@ -64,17 +67,25 @@ export interface SavedQueryServiceDeps {
* `createSavedQuery`/`commitSavedQuery` only ever call `.validate`).
* Production passes `app.specValidators` (structurally assignable). */
specValidators: SpecValidationService;
/** The strict aggregate-commit seam (#287 W4) `createSavedQuery`/
* `commitSavedQuery` await instead of the retired flat `asb:saved` write.
* Production passes `app.workspace` (structurally assignable — only
* `.commit` is ever called). */
workspace: Pick<WorkspaceRepository, 'commit'>;
}

// ── Result types ─────────────────────────────────────────────────────────────

export type CreateSavedResult =
| { ok: true; entry: SavedQueryV2 }
/** `createSavedQuery` itself returned null (already-linked tab, blank SQL
* on a non-text panel, blank name, or a blocking validation diagnostic) —
* the pre-extraction inline code never distinguished a reason here either
* (silent no-op), so neither does this result. */
| { ok: false };
/** `createSavedQuery` itself rejected the entry — either a pre-commit
* compute guard (already-linked tab, blank SQL on a non-text panel, blank
* name, or a blocking validation diagnostic — the pre-#287 inline code
* never distinguished a reason here either, so neither does this result;
* `diagnostics` absent), or the aggregate strictly rejected the whole-
* workspace commit (#287 W4 — `diagnostics` present, straight from
* `WorkspaceRepository.commit`; nothing was mutated). */
| { ok: false; diagnostics?: WorkspaceDiagnostic[] };

export type CommitLinkedResult =
| { ok: true; entry: SavedQueryV2 }
Expand All @@ -85,10 +96,13 @@ export type CommitLinkedResult =
| 'invalid-spec'
/** Blank SQL on a panel type that isn't SQL-optional (#166's text panel). */
| 'empty'
/** `commitSavedQuery` itself returned null (tab no longer linked, or its
* own re-validation against the normalized Spec rejected it) — a
* defensive case the pre-extraction inline code never toasted either. */
/** `commitSavedQuery` itself rejected the commit — either its own
* defensive re-check against the normalized Spec (tab no longer linked —
* the pre-#287 inline code never toasted this either, `diagnostics`
* absent), or the aggregate strictly rejected the whole-workspace commit
* (#287 W4 — `diagnostics` present; nothing was mutated). */
| 'rejected';
diagnostics?: WorkspaceDiagnostic[];
};

export type ShareResult =
Expand Down Expand Up @@ -116,12 +130,12 @@ export interface SavedQueryService {
* saved query from an unsaved tab's current `sqlDraft`/`specParsed` plus
* `name`/`description`. Rejects (silently — see `CreateSavedResult`) an
* already-linked tab, per `createSavedQuery`'s own guard. */
create(tab: QueryTab, name: unknown, description: unknown): CreateSavedResult;
create(tab: QueryTab, name: unknown, description: unknown): Promise<CreateSavedResult>;
/** Update-in-place path (app.ts's `commitLinkedQuery`, the "Save" button on
* an already-linked tab): persist `evaluated` as the linked saved query's
* new Spec. Takes the Spec evaluation as an input (see this module's
* header comment) rather than evaluating it itself. */
commit(tab: QueryTab, evaluated: { parsed: unknown; diagnostics: SpecValidationDiagnostic[] }): CommitLinkedResult;
commit(tab: QueryTab, evaluated: { parsed: unknown; diagnostics: SpecValidationDiagnostic[] }): Promise<CommitLinkedResult>;
/** Record a successful run in history (state.ts's own `recordHistory`) —
* never touches rendering; app.ts's own `app.recordHistory` delegate
* conditionally repaints the History side panel itself after calling
Expand All @@ -136,23 +150,29 @@ export interface SavedQueryService {
* validation, no defaulting; the caller supplies every field exactly as it
* wants it used. */
export function createSavedQueryService(deps: SavedQueryServiceDeps): SavedQueryService {
function create(tab: QueryTab, name: unknown, description: unknown): CreateSavedResult {
const entry = createSavedQuery(deps.state, tab, name, description, deps.saveJSON, deps.now(), deps.specValidators);
return entry ? { ok: true, entry } : { ok: false };
async function create(tab: QueryTab, name: unknown, description: unknown): Promise<CreateSavedResult> {
const result = await createSavedQuery(
deps.state, tab, name, description, deps.workspace.commit, deps.now(), deps.specValidators,
);
return result.ok ? { ok: true, entry: result.entry } : { ok: false, diagnostics: result.diagnostics };
}

function commit(
async function commit(
tab: QueryTab, evaluated: { parsed: unknown; diagnostics: SpecValidationDiagnostic[] },
): CommitLinkedResult {
): Promise<CommitLinkedResult> {
if (!evaluated.parsed || hasBlockingSpecErrors(evaluated.diagnostics)) {
return { ok: false, reason: 'invalid-spec' };
}
const panel = queryPanel({ spec: evaluated.parsed });
if (!String(tab.sqlDraft || '').trim() && !isQuerylessPanel(panel)) {
return { ok: false, reason: 'empty' };
}
const entry = commitSavedQuery(deps.state, tab, evaluated.parsed as QuerySpecDraft | null, deps.saveJSON, deps.specValidators);
return entry ? { ok: true, entry } : { ok: false, reason: 'rejected' };
const result = await commitSavedQuery(
deps.state, tab, evaluated.parsed as QuerySpecDraft | null, deps.workspace.commit, deps.specValidators,
);
return result.ok
? { ok: true, entry: result.entry }
: { ok: false, reason: 'rejected', diagnostics: result.diagnostics };
}

function recordHistoryFn(tab: QueryTab, sqlText?: string): void {
Expand Down
Loading
Loading