Forward telemetry parameters to the kernel binding - #925

🔵 Low — _kernel_telemetry_kwargs builds the phase-7 identity/telemetry kwargs (driver_name, telemetry_enabled, process_name, etc.) and they are spread unconditionally into _kernel.Session(**telemetry_kwargs) at open_session. The kernel wheel constraint is still ^0.2.0 (>=0.2.0,<0.3.0). If these kwargs were introduced in a later 0.2.x than 0.2.0, a user with an older-but-constraint-satisfying wheel installed would hit a TypeError: Session() got an unexpected keyword argument ... at connect time. If phase-7 requires a minimum kernel version, consider bumping the lower bound of the databricks-sql-kernel pin so the wheel and connector stay in lockstep. (Flagged Low — I can't verify the kernel Session signature from this repo.)

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

I've verified the reviewer's concern against the real installed wheel and flagged it for a human. Summary:

• Concern is valid. The installed databricks_sql_kernel 0.2.0 Session signature accepts none of the phase-7 telemetry kwargs that _kernel_telemetry_kwargs builds (nor even request_timeout_secs). A user with 0.2.0 — which satisfies the ^0.2.0 pin — would hit TypeError at connect time, and the @realkernel routing test can't catch it because it mocks open_session.
• Why blocked, not fixed: the fix is a pin lower-bound bump in pyproject.toml, but the correct floor version can't be determined from this environment (only 0.2.0 is present; no network/PyPI access to find which 0.2.x introduced the kwargs). Guessing risks mis-constraining a widely-used connector, s

Pushed 6cba809 (bundled with 4 other thread(s)).

it's okay, the required kernel version will be updated once kernel is released

Thanks — that timing plan makes sense. Just to close the loop for anyone reading the thread later: engineer-bot confirmed the concern is real (the installed 0.2.0 Session signature rejects the phase-7 telemetry kwargs), and the diff at 6cba809 only added an explanatory comment, so the ^0.2.0 lower bound is still in place. Leaving this open as a tracking reminder to bump the databricks-sql-kernel floor in pyproject.toml once the kernel release lands, so the wheel and connector stay in lockstep.

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

Converged / needs-human, and already resolved out-of-band: maintainer @jay-xiao446 decided the databricks-sql-kernel pin floor will be bumped once the kernel is released, so it deliberately cannot ride in this PR. The reviewer's concern is valid (installed 0.2.0 Session rejects the phase-7 telemetry kwargs built by _kernel_telemetry_kwargs), but the fix is a pyproject.toml lower-bound bump gated on an unreleased kernel version — an out-of-band change, not actionable here. Thread is being kept open only as a tracking reminder; no code change warranted.

🔵 Low — Unlike retry_kwargs/http_headers_kwargs (which are conditionally omitted when empty), _kernel_telemetry_kwargs always returns the phase-7 identity fields (driver_name, runtime_*, os_*, process_name, telemetry_enabled), so these kwargs are passed to _kernel.Session(...) on every use_kernel open. If the installed kernel wheel predates phase-7 support for these Session kwargs, construction raises TypeError and every use_kernel=True connection breaks. The dependency floor is still ^0.2.0 and isn't bumped in this PR. If phase-7 requires a newer kernel wheel, consider raising the minimum version so incompatible wheels fail at install time rather than at connect time. (Low because I can't verify the 0.2.0 Session signature from this repo — the kernel is a compiled extension.)

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

I've completed my investigation and reported the outcome as blocked.

Summary: The reviewer flagged that phase-7 telemetry kwargs are passed unconditionally to _kernel.Session(...) on every use_kernel=True open, and would raise TypeError if the installed kernel wheel predates phase-7 support. I verified this empirically using the real 0.2.0 wheel present on the runner:

• Session.__text_signature__ contains none of the phase-7 kwargs.
• Session(..., telemetry_enabled=True) raises TypeError: Session.__new__() got an unexpected keyword argument 'telemetry_enabled'.
• 0.2.0 is both this PR's floor and the latest published version — so every use_kernel=True connection would break at connect time.

I escalated rather than editing because the reviewer's suggested fix (r

Pushed 6cba809 (bundled with 4 other thread(s)).

it's okay, the required kernel version will be updated once kernel is released

Thanks — that matches what the engineer-bot found empirically (the shipped 0.2.0 wheel rejects telemetry_enabled with a TypeError), so this is a real break for use_kernel=True connections until that kernel release lands. Since the version floor isn't bumped in this PR and the fix is deferred to the future kernel release, I'll leave this open so it isn't lost track of — worth bumping the ^0.2.0 floor in lockstep with that release so incompatible wheels fail at install time rather than at connect time.

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

Converged + deferred out-of-band. Maintainer (jay-xiao446) explicitly decided the required kernel version will be bumped once the phase-7 kernel wheel is released; the review-bot is only keeping the thread open as a tracking reminder to raise the ^0.2.0 floor in lockstep. The concern is real (verified previously: the shipped 0.2.0 wheel raises TypeError on telemetry_enabled), but the fix — bumping the dependency floor — can only be actioned when the new kernel wheel is published and cannot ride in this PR. No safe in-PR code change exists (pinning to an unpublished version breaks install; omitting the identity kwargs disables the telemetry this PR ships). Human action: bump the kernel minimum version alongside the phase-7 kernel release.

🔵 Low — This suppresses the wrapper-side connection-failure log for use_kernel=True, delegating failure telemetry to the kernel. But a connect() failure happens inside KernelDatabricksClient.open_session (where the kernel Session is first constructed), so if the kernel never finished initializing/flushing its telemetry, the failure may not be reported by either side. Combined with the kernel path defaulting enable_telemetry to False (session.py), a default use_kernel=True connection that fails to open reports no connection-failure telemetry anywhere — whereas a Thrift connection (default enable_telemetry=True) does. Worth confirming the kernel actually captures pre-session-open connect failures; otherwise this is an observability regression for the kernel path. Design tradeoff, not a correctness bug.

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

The reviewer raises a genuine but design-level observability tradeoff, explicitly "not a correctness bug." The deciding fact — whether the kernel captures/flushes connection-failure telemetry when its own databricks_sql_kernel.Session constructor throws before init completes (client.py open_session:401-423) — lives in the compiled Rust kernel wheel, which is not readable from this repo, and can only be verified against a live warehouse (E2E), which this follow-up job has no credentials to run. The connector side is confirmed to be deliberately silent for use_kernel (is_telemetry_enabled short-circuits False at telemetry_client.py:124-125; _kernel_telemetry_kwargs defaults telemetry_enabled=False "by design" at client.py:194-197), so if the kernel does NOT capture pre-session-open failures, a default use_kernel=True connect that fails would report no failure telemetry anywhere. Resolving this needs a human to (1) confirm the kernel's pre-Session-open failure telemetry behavior, and (2) decide the wrapper-vs-kernel duplication vs. coverage tradeoff — no in-PR code change can settle it here.

🔵 Low — Forwarding an explicit enable_telemetry=True sets the kernel's telemetry_enabled=True unconditionally, whereas the Thrift/SEA path only actually emits when the server-side enableTelemetryForPythonDriver feature flag is on (see is_telemetry_enabled). The inline comment acknowledges this and states the kernel "is expected to mirror the feature-flag-gated wrapper behaviour" — but that gate lives entirely on the kernel side and can't be verified from this repo. If the kernel does not re-implement the feature-flag gate, a caller passing enable_telemetry=True on use_kernel=True will get telemetry emitted that the Thrift path would have suppressed. Worth confirming the kernel honors the flag, or documenting the divergence for users; the None-when-unset default is fine, this is specifically about the explicit-True case.

⛔ NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

Valid concern, but not resolvable with a code change in this repo. The reviewer confirms the None-when-unset default is fine; the open question is only the explicit enable_telemetry=True case, where the correctness of forwarding depends on whether the kernel re-implements the server-side enableTelemetryForPythonDriver gate. That gate lives entirely on the kernel side and cannot be read or verified from this repo, so no local edit or further thread exchange can settle it. The divergence is already heavily documented in the inline comment at session.py:271-283; whether to add user-facing telemetry-parity docs is a public-API/documentation call for the connector maintainers. A human who owns the kernel telemetry path must confirm the kernel honors the feature flag (or decide to document the divergence for users).

it's expected, telemetry for kernel requires opt-in

Thanks for confirming — that the kernel treats enable_telemetry=True as an explicit opt-in (rather than mirroring the server-side enableTelemetryForPythonDriver gate) is exactly the design point I couldn't verify from this repo, so it's good to have it from someone who owns that path. The one thing that's still worth a moment: this is an intentional behavioral divergence from the Thrift/SEA path, where an explicit True can be suppressed by the feature flag. Since the reasoning lives only in the inline comment, a short user-facing note that kernel telemetry is opt-in and not feature-flag-gated would spare callers a surprise. I'll leave this open so that documentation decision stays on a maintainer's radar rather than getting silently resolved.

I added a user-facing telemetry note to the use_kernel parameter docstring in src/databricks/sql/client.py, addressing the reviewer's request. The note documents that on the kernel path telemetry is strictly opt-in — an explicit enable_telemetry=True enables it unconditionally and is not gated by the server-side enableTelemetryForPythonDriver feature flag (an intentional divergence from the Thrift/SEA paths, where an explicit True can still be suppressed by the flag). This moves the reasoning out of the internal inline comment and into public docs where callers will see it. The change is docstring-only with no logic impact, so no tests were affected.

Pushed 02002a1.