Skip to content

storage-controller: keep export descriptions in sync on ALTER SINK and skip no-op connection alters#37653

Draft
ggevay wants to merge 2 commits into
MaterializeInc:mainfrom
ggevay:gabor/sql-517-alter-connection-invalid-alter
Draft

storage-controller: keep export descriptions in sync on ALTER SINK and skip no-op connection alters#37653
ggevay wants to merge 2 commits into
MaterializeInc:mainfrom
ggevay:gabor/sql-517-alter-connection-invalid-alter

Conversation

@ggevay

@ggevay ggevay commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Fixes SQL-517

Motivation

GRANT, REVOKE, ALTER ... OWNER, or ALTER CONNECTION on a connection
panicked the coordinator with

altering export connections after txn must succeed: InvalidAlter(AlterError { id: User(...) })

whenever a dependent sink had previously been changed with
ALTER SINK ... SET FROM. Seen in the Nightly Parallel Workload jobs
"rename + naughty identifiers" and "DDL Only".

Root cause: StorageController::alter_export (the ALTER SINK path) sends the
new sink definition to the cluster and updates the ExportState, but never
wrote the new ExportDescription into the collection's DataSource::Sink. The
controller's stored export description therefore kept the sink definition from
sink creation forever. This update was lost when the description moved from
ExportState into CollectionState in 273b3e4 ("Make sinks a variant of
collections").

Any catalog change to a connection item (including privilege and owner changes)
emits a Connection(Altered) implication, which recomputes the connection of
every dependent sink from the catalog and calls alter_export_connections.
That method diffs the new connection against the stale stored description with
alter_compatible, which requires fields like value_desc and
relation_key_indices to match. After an ALTER SINK ... SET FROM had changed
those, the check failed with InvalidAlter, and the coordinator treats that as
a violated invariant and panics.

Description

Two commits:

  • Keep the stored export description current on ALTER SINK (the bug fix):
    alter_export now stores the new ExportDescription in the collection state
    after sending the RunSink command, mirroring alter_export_connections.
    With the controller's description tracking the catalog, later connection
    alters diff against the sink that is actually running. This also makes
    run_export rerun the current sink definition instead of the stale one.

  • Skip no-op sink connection alters (separable optimization): catalog
    changes that don't affect a connection's runtime config, like GRANT and
    REVOKE, also arrive at alter_export_connections. Previously they
    restarted every dependent sink dataflow for no reason. Skip updates where
    the connection is unchanged, mirroring alter_ingestion_connections. This
    does not mask the bug above: with a stale description the recomputed
    connection differs, so the compatibility check still runs.

User-facing changes:

  • GRANT/REVOKE/ALTER on a connection no longer panics the coordinator
    when a dependent sink had been altered with ALTER SINK ... SET FROM.
  • Privilege- and owner-only changes to a connection no longer needlessly
    restart dependent sink dataflows.

Verification

  • New regression test in test/testdrive/alter-sink.td: GRANT on the sink's
    connection after ALTER SINK ... SET FROM, then verify data still flows.
    Before the fix, a minimal repro (CREATE SINK, ALTER SINK SET FROM, GRANT ON
    CONNECTION) deterministically reproduced the exact panic and backtrace from
    the Nightly failures. After the fix it passes.
  • alter-sink.td, connection-alter.td, and kafka-sink-statistics.td (which
    breaks and restores a sink's connection via real ALTER CONNECTION,
    exercising the non-skipped path) all pass with both commits.
  • bin/cargo-test -p mz-storage-controller passes.

🤖 Generated with Claude Code

ggevay and others added 2 commits July 14, 2026 20:28
alter_export sent the new sink definition to the cluster and updated the
ExportState, but never wrote the new ExportDescription into the
collection's DataSource::Sink. The stored description therefore kept the
definition from sink creation forever. The staleness dates back to the
sinks-as-collections refactor (273b3e4), which moved the description
from ExportState into CollectionState but dropped the update here.

alter_export_connections reads that stored description as the currently
running sink and checks a newly computed connection against it with
alter_compatible. Any catalog change to a connection (ALTER CONNECTION,
GRANT, REVOKE, owner change) emits a Connection Altered implication,
which recomputes the connection of every dependent sink from the catalog
and calls alter_export_connections. After an ALTER SINK ... SET FROM had
changed the sink's value_desc or relation_key_indices, that check failed
with InvalidAlter and panicked the coordinator with "altering export
connections after txn must succeed" (SQL-517).

Fix by storing the new description in the collection state after sending
the RunSink command, mirroring alter_export_connections. This also makes
run_export rerun the current sink definition instead of the stale one.

Add a regression test to alter-sink.td that grants on the sink's
connection after ALTER SINK ... SET FROM.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A Connection Altered implication re-alters every dependent sink, also
for catalog changes that don't affect the connection's runtime config,
like GRANT and REVOKE. Restarting the sink dataflow in that case is
wasted work and briefly interrupts the sink. Skip updates where the
connection is unchanged, mirroring alter_ingestion_connections.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ggevay ggevay added the T-sinks Theme: Sinks label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-sinks Theme: Sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant