Skip to content

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675

Open
indietyp wants to merge 7 commits into
bm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and
Open

BE-523: HashQL: Suspendable interpreter with structured values and input system#8675
indietyp wants to merge 7 commits into
bm/be-522-hashql-expand-stdlib-type-definitionsfrom
bm/be-523-hashql-suspendable-interpreter-with-structured-values-and

Conversation

@indietyp
Copy link
Copy Markdown
Member

@indietyp indietyp commented Apr 29, 2026

🌟 What is the purpose of this PR?

Rework the MIR interpreter to support suspend/resume execution, richer value representations, and a proper input system. Together these let the interpreter pause when it needs external data (e.g. a database query), hand back a structured description of what it needs, and resume once the caller provides the result.

🔍 What does this change?

Suspension model (interpret/suspension/)

  • Introduce a suspend/resume protocol: when the interpreter hits a GraphRead terminator it yields a Suspension instead of blocking. Callers drive a start/resume loop until they get a Return.
  • GraphReadSuspension extracts temporal axes, filter parameters, and entity type from the suspended state.
  • Continuation carries the response value that resumes execution.
  • TemporalAxesInterval / Timestamp give structured representations of the temporal query window.
  • Runtime::run convenience wrapper for callers that can fulfill suspensions synchronously.

Input system (interpret/inputs.rs)

  • Add Inputs<'heap, A>: a typed map from interned symbols to Values, consulted for InputOp::Load and InputOp::Exists. Replaces the unstructured FastHashMap previously threaded through the runtime. Supports both global and custom allocators.

Value representation (interpret/value/)

  • Int: carry bit-width (1-bit boolean vs 128-bit integer) so serialization can distinguish true/false from 0/1 without external type info. Arithmetic promotes to 128-bit; bitwise boolean ops preserve 1-bit.
  • Struct: add StructBuilder<N> for stack-allocated fixed-field construction. Enforce sorted field invariant. Add merge and project_by_name.
  • Minor improvements to Opaque, List allocator parameterization, Str display.

SSA repair fix (pass/transform/ssa_repair)

  • Fix block parameters not being renamed during SSA repair.
  • Extend use-before-def detection to terminators, not just statements.

Entity path type resolution (pass/execution/traversal)

  • Add field_path and resolve_type to EntityPath for navigating from an entity path (e.g. WebId, Properties) to its concrete stdlib type.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

None

🐾 Next steps

  • The orchestrator in hashql-eval consumes the suspension model and input system introduced here.

🛡 What tests cover this?

  • Interpreter unit tests (interpret/tests.rs): cover the suspend/resume loop, input resolution, and value operations.
  • SSA repair UI tests (tests/ui/pass/ssa_repair): cover the block parameter and terminator fixes.
  • Existing pass UI tests re-blessed for snapshot changes from the Int representation change.

❓ How to test this?

  1. cargo nextest run --package hashql-mir
  2. cargo test --package hashql-mir --doc

feat: make Int size aware

feat: postgres serialization

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: errors

feat: psql codec

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: move bridge -> orchestrator

feat: move bridge -> orchestrator

feat: organize the runtime a bit more

feat: organize the runtime a bit more

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: checkpoint

feat: organize the runtime a bit more

feat: finish orchestrator first version

feat: finish orchestrator first version

feat: feat: move out suspension requests to own module

chore: clippy

feat: split out into more manageable functions

feat: split out into more manageable functions

chore: orchestrator docs

feat: organize the runtime a bit more

feat: test orchestrator

feat: better symbol repr

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: orchestrator test harness

feat: test orchestrator

feat: pipeline

feat: bless snapshots

chore: tests

chore: masking

chore: tests

fix: SSA repair

chiore: ssa repair snapshots

feat: orchestrator test harness

chore: remove spec

fix: library features

chore: update snapshots

chore: update docs
@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:14 Inactive
@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 29, 2026

PR Summary

High Risk
High risk because it refactors the MIR interpreter’s core execution API (new suspend/resume flow, GraphRead handling, and Runtime::run signature) and changes fundamental runtime value semantics (boolean/int representation and struct/opaque projections), which can affect query correctness and backend integration.

Overview
Enables a suspend/resume execution model in the MIR interpreter: GraphRead terminators now yield a typed Suspension (with extracted temporal axes) and resume via a Continuation, with new Runtime::start, Runtime::resume, Yield, run_until_suspension, and run_until_transition APIs (and Runtime::run updated to drive suspensions via a callback).

Replaces ad-hoc input passing with a dedicated Inputs container and updates runtime/bench/tests to use it. Refactors interpreter error handling to support suspension-originated diagnostics and adds new runtime invariants/errors (UnexpectedValueType, InvalidConstructor).

Updates runtime value infrastructure: Int now tracks bit-width to distinguish booleans from integers (affecting display/tests/snapshots), adds StructBuilder, makes struct fields sorted, and extends projections/subscripts to work through Opaque wrappers (plus value_mut on Opaque and minor Str/List adjustments). Also adds small execution-planner utilities (IslandGraph::lookup, allocator generic on scheduling) and entity traversal helpers (EntityPath::field_path/resolve_type, TraversalPath::resolve_type), with associated test/snapshot updates and a minor Postgres constant compilation tweak (u32 conversion).

Reviewed by Cursor Bugbot for commit 4082a24. Bugbot is set up for automated code reviews on this repo. Configure here.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment May 15, 2026 7:57am
petrinaut Ready Ready Preview May 15, 2026 7:57am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview May 15, 2026 7:57am

@github-actions github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Apr 29, 2026
@indietyp indietyp changed the title HashQL: Suspendable interpreter with structured values and input system BE-523: HashQL: Suspendable interpreter with structured values and input system Apr 29, 2026
Copy link
Copy Markdown
Member Author

indietyp commented Apr 29, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.21%. Comparing base (9047680) to head (4082a24).

Additional details and impacted files
@@                                 Coverage Diff                                 @@
##           bm/be-522-hashql-expand-stdlib-type-definitions    #8675      +/-   ##
===================================================================================
- Coverage                                            62.25%   62.21%   -0.05%     
===================================================================================
  Files                                                 1354     1354              
  Lines                                               137104   136863     -241     
  Branches                                              5793     5771      -22     
===================================================================================
- Hits                                                 85360    85155     -205     
+ Misses                                               50837    50811      -26     
+ Partials                                               907      897      -10     
Flag Coverage Δ
local.hash-backend-utils 2.81% <ø> (ø)
rust.hash-graph-api 2.52% <ø> (ø)
rust.hashql-compiletest 29.63% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vercel vercel Bot temporarily deployed to Preview – petrinaut April 29, 2026 14:18 Inactive
@indietyp indietyp force-pushed the bm/be-522-hashql-expand-stdlib-type-definitions branch from deb5f19 to 7d9f781 Compare May 4, 2026 11:34
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from b2d61fe to 4cc8b7a Compare May 4, 2026 11:34
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from 4cc8b7a to 7be336d Compare May 8, 2026 08:33
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from 7be336d to e2fda5b Compare May 11, 2026 10:58
@indietyp indietyp force-pushed the bm/be-522-hashql-expand-stdlib-type-definitions branch from 3cb3bf5 to dd10bcf Compare May 12, 2026 13:58
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from e2fda5b to 50368b6 Compare May 12, 2026 13:58
@indietyp indietyp force-pushed the bm/be-522-hashql-expand-stdlib-type-definitions branch from dd10bcf to ffd9997 Compare May 13, 2026 09:38
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from 50368b6 to 6499ac2 Compare May 13, 2026 09:38
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from 6499ac2 to d005e5a Compare May 13, 2026 09:57
@indietyp indietyp force-pushed the bm/be-522-hashql-expand-stdlib-type-definitions branch from 89ae7c2 to 9047680 Compare May 15, 2026 07:49
@indietyp indietyp force-pushed the bm/be-523-hashql-suspendable-interpreter-with-structured-values-and branch from d005e5a to 4082a24 Compare May 15, 2026 07:49
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4082a24. Configure here.


// Interval(Struct { start, end })
let interval_fields = interner.symbols.intern_slice(&[sym::end, sym::start]);
let interval_struct = Struct::new_unchecked(interval_fields, Rc::new([end_bound, start_bound]));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interval field order swaps start and end bounds

Medium Severity

In make_temporal_axes, the interval_fields are interned as [sym::end, sym::start] (sorted alphabetically), but the values Rc::new([end_bound, start_bound]) pair end with end_bound at index 0 and start with start_bound at index 1. While this happens to be correct for this specific ordering, the construction is fragile and easy to misread — the fields array reads [end, start] while semantically one expects [start, end]. More importantly, the Struct::new_unchecked invariant requires fields to be sorted, and the code relies on "end" < "start" alphabetically. If the field names ever changed, or if Symbol ordering changed, the pairing would silently break because the values array order must match the fields array order but is written in a confusing [end_bound, start_bound] order rather than a clear correspondence.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4082a24. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants