Skip to content

adapter: bound read-then-write dependency validation#37654

Merged
mtabebe merged 1 commit into
MaterializeInc:mainfrom
mtabebe:ma/SQL-426
Jul 15, 2026
Merged

adapter: bound read-then-write dependency validation#37654
mtabebe merged 1 commit into
MaterializeInc:mainfrom
mtabebe:ma/SQL-426

Conversation

@mtabebe

@mtabebe mtabebe commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Problem:
validate_read_then_write_dependencies recursed over the transitive uses() of a read-then-write's read set, one coordinator-thread frame per object. A statement whose selection reads from a deep chain of stacked views overflowed the coordinator stack

Solution:
Replace the recursion with an iterative worklist. Dedup at enqueue time so each object is validated once and the worklist stays proportional to the number of distinct objects, not the number of dependency edges. Cap the traversal at read_then_write_max_dependencies (a dyncfg, default 100k) and reject larger graphs with a clean error instead of walking an unbounded graph.

Tests:

  • a unit test validating a 100k-deep chain (the old recursive code overflowed the stack on it)
  • a unit test for the dependency bound
  • a small read-then-write limits smoke test.

Fixes SQL-426

Problem:
A DELETE or UPDATE whose selection reads from a deep chain of stacked
views could crash the coordinator, and the process would take the whole
environment down with it. Such a statement now runs normally. A read set
with an extreme number of transitive dependencies gets a clean error
instead of an unbounded walk.

validate_read_then_write_dependencies recursed over the transitive
uses() of a read-then-write's read set, one coordinator-thread frame per
object. A statement whose selection reads from a deep chain of stacked
views overflowed the coordinator stack.

Solution:
Replace the recursion with an iterative worklist. Dedup at enqueue time
so each object is validated once and the worklist stays proportional to
the number of distinct objects, not the number of dependency edges. Cap
the traversal at read_then_write_max_dependencies (a dyncfg, default
100k) and reject larger graphs with a clean error instead of walking an
unbounded graph.

Tests:
- a unit test validating a 100k-deep chain (the old recursive code
  overflowed the stack on it)
- a unit test for the dependency bound
- a small read-then-write limits smoke test.
@mtabebe mtabebe marked this pull request as ready for review July 14, 2026 21:10
@mtabebe mtabebe requested review from a team as code owners July 14, 2026 21:10

@aljoscha aljoscha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

use crate::catalog::Catalog;
use crate::error::AdapterError;

/// Adds `id` to the worklist the first time it is seen, enforcing the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this is really an implementation detail of the validation method, so maybe could declare in there. But fine if you just want to move on and merge

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just going to leave it and close the issue

@mtabebe mtabebe merged commit 3713eed into MaterializeInc:main Jul 15, 2026
372 of 376 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants