feat: RYOW for DuckDB within an explicit txn and MDEV-40379 fix

[drrtuy]

Add optional Read-Your-Own-Writes for cross-engine DuckDB scans

What

In cross-engine joins, DuckDB previously read non-DuckDB tables through a fiber running a synthetic SELECT in a separate background transaction, so uncommitted writes from the parent transaction were invisible. This PR adds an opt-in session variable that lets DuckDB read those uncommitted writes (RYOW) via a direct handler scan in the parent transaction.

Key changes

• ha_duckdb.cc: new session var duckdb_cross_engine_ryow (default OFF) plus accessor get_thd_cross_engine_ryow().
• cross_engine_scan.cc/.h: per-query RYOW toggle; new _mdb_scan_direct table function with filter_pushdown=false; replacement scan routes to _mdb_scan_direct when RYOW is on.
• ha_duckdb_pushdown.cc: registers the RYOW flag during init_scan().
• architecture.md: documents the RYOW path and its tradeoffs.

How to test

Run cross_engine_ryow.test: with the var OFF uncommitted parent-transaction writes to InnoDB are hidden from the join; with it ON they are visible; after ROLLBACK committed state is intact.

[gemini-code-assist]

Code Review

This pull request introduces a new session variable duckdb_cross_engine_ryow to toggle Read-Your-Own-Writes (RYOW) visibility for cross-engine joins in DuckDB, and disables temporary table support to resolve an assertion failure (MDEV-40379). The reviewer suggests a significant simplification to the RYOW implementation: instead of capturing the flag in init_scan() and storing it in a thread-local variable, the code should query current_thd directly during query compilation. This eliminates the need for the thread-local variable, its registration, cleanup logic, and an unnecessary header include, which should also be reflected in the updated documentation.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

This include is no longer needed if we don't call get_thd_cross_engine_ryow here.

[gemini-code-assist]

Since the RYOW flag is now read directly from current_thd during query compilation, we don't need to register it here in init_scan().