Support UDTFs in information_schema.routines / SHOW FUNCTIONS

[zhuqi-lucas]

Which issue does this PR close?

• Closes SHOW FUNCTIONS / information_schema.routines omit table functions (UDTFs) #23437.

Rationale for this change

information_schema.routines and SHOW FUNCTIONS currently enumerate only scalar / aggregate / window UDFs. Table functions (UDTFs) registered via SessionContext::register_udtf are omitted, making them undiscoverable through SQL. Discovery matters because downstream tooling (DataFusion CLI, Massive's atlas SQL surface, dbt-datafusion, etc.) uses these SQL surfaces to list available functions.

What changes are included in this PR?

1. Snapshot table functions into the information_schema provider.

Session::table_functions() cannot exist on the Session trait today: TableFunction lives in datafusion-catalog, which already depends on datafusion-session (both TableProvider::scan and TableFunction take &dyn Session). Adding a trait method returning &HashMap<String, Arc<TableFunction>> would reverse that edge and create a crate-dependency cycle. ScalarUDF / AggregateUDF / WindowUDF don't hit this because they live in datafusion-expr, which sits below datafusion-session.

This is the same underlying constraint as #23348. A follow-up PR will move TableFunction (or hoist the relevant traits) so the builder can be deleted.

For this PR, SessionState.table_functions is snapshotted into InformationSchemaProvider via with_table_functions() at construction. The provider is built per-query, so the snapshot stays fresh.

2. Emit UDTF rows in information_schema.routines only.

make_routines: one row per UDTF with routine_type = "FUNCTION", function_type = "TABLE", data_type = "TABLE".

UDTFs deliberately do NOT appear in information_schema.parameters. A same-named scalar UDF (e.g. generate_series exists as both a scalar UDF in functions-nested and a UDTF in functions-table) would cross-join with a UDTF row keyed only by (name, rid) and produce spurious TABLE-typed variants of every scalar signature in SHOW FUNCTIONS.

3. Rewrite the SHOW FUNCTIONS SQL to UNION UDTFs directly from routines.

The old query joined parameters p (INNER) requiring both IN and OUT rows before joining routines. UDTFs have no IN parameters, so they fell out. The new plan sources scalar / aggregate / window signatures from parameters as before, and UDTFs from a separate UNION branch that reads routines directly (guarded by function_type = TABLE). This avoids the cross-join blowup mentioned above.

Are these changes tested?

Verified locally by registering a UDTF and running SHOW FUNCTIONS:

> SHOW FUNCTIONS LIKE 'stale%'
+---------------+-------------+---------------+
| function_name | return_type | function_type |
| stale_files   | TABLE       | TABLE         |
+---------------+-------------+---------------+

Happy to add sqllogictests in datafusion/sqllogictest/test_files/information_schema.slt if maintainers prefer.

Are there any user-facing changes?

Yes: SHOW FUNCTIONS output now includes table functions. information_schema.routines gains one row per registered UDTF. information_schema.parameters is intentionally unchanged (see rationale in point 2 above). Existing rows are unchanged.

[Copilot]

Pull request overview

This PR extends DataFusion’s SQL discovery surfaces to include table functions (UDTFs) in information_schema.routines and SHOW FUNCTIONS, addressing the gap where only scalar/aggregate/window UDFs were listed.

Changes:

• Snapshot SessionState.table_functions into InformationSchemaProvider so UDTFs can be exposed without introducing crate cycles.
• Emit information_schema.routines and information_schema.parameters rows for UDTFs (synthetic OUT parameter row with data_type = 'TABLE').
• Rewrite SHOW FUNCTIONS SQL to start from OUT parameter rows and LEFT JOIN IN parameter rows so functions with no inputs can appear.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
datafusion/sql/src/statement.rs	Rewrites SHOW FUNCTIONS to be compatible with functions that lack IN parameters (e.g., UDTFs).
datafusion/core/src/execution/session_state.rs	Passes the session’s registered table functions into the per-query InformationSchemaProvider.
datafusion/catalog/src/information_schema.rs	Stores table functions in InformationSchemaConfig and emits corresponding routines / parameters rows.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[alamb]

Makes sense to me -- thank you @zhuqi-lucas

[alamb]

I don't understand why we have to treat TableFunctions specially -- the PR descriptio says it is to avoid a crate dependency cycle, but I don't understand why the same thing doesn't apply to Scalar and Aggregate functons

Mabe we just need to move a definition of TableFunction to datafusion-expr 🤔 But maybe that is not possible

[zhuqi-lucas]

Thanks @alamb for good catch, refreshed the PR description with the concrete cycle explanation.

TableFunction lives in datafusion-catalog, which already depends on datafusion-session — putting table_functions() on the Session trait reverses that edge. Scalar/Agg live in datafusion-expr (below session), so no cycle there. Same root as #23348 — I'll open a follow-up PR that moves TableFunction down and deletes with_table_functions().

[zhuqi-lucas]

Thank you @alamb and @xudong963 for review, merged to main.