Forward telemetry parameters to the kernel binding - #925
Conversation
8816d71 to
ae59e2c
Compare
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the core gate (is_telemetry_enabled returns False for kernel connections, via a safe getattr) is correct and well-tested, and session.use_kernel is guaranteed set before it's read. One low-severity gap: the connection-failure telemetry path (connection_failure_log) is not gated on use_kernel, so a failed kernel connect still emits a Python-side failure log.
Other findings
- 🔵 Low — This PR disables Python-side telemetry for kernel connections by short-circuiting
is_telemetry_enabled(telemetry_client.py:125). However, the connection-failure telemetry path is a separate code path that bypassesis_telemetry_enabledentirely: whensession.open()raises,connection_failure_logis invoked and gated only onenable_telemetry(defaulting toTrue), never onuse_kernel. So a kernel connection that fails to open will still emit a Python-side failure log — inconsistent with the PR's stated goal of disabling Python telemetry for kernel connections (and potentially duplicating what the kernel reports). If failure-before-kernel-open telemetry is intentionally still desired on the Python side, this is fine as-is; otherwise consider passinguse_kernelthrough and skippingconnection_failure_logon that path.
ae59e2c to
449d175
Compare
There was a problem hiding this comment.
Verdict: 2 Low
Looks good overall — the change cleanly disables Python-side telemetry on kernel connections and threads phase-7 identity/telemetry kwargs into the kernel Session, with solid regression tests (including the force-enabled case). Two low-severity notes: an identity-vs-truthiness mismatch between the telemetry bypass and backend routing, and a potential wheel-version compat gap since the new kwargs are passed unconditionally against a ^0.2.0 pin.
449d175 to
8261c66
Compare
There was a problem hiding this comment.
Verdict: 2 Low
Looks good overall — the kernel-path telemetry forwarding and the is_telemetry_enabled short-circuit are correct, defaults line up between connect()/session.py/_kernel_telemetry_kwargs, and the new tests are meaningful. Two low-severity notes: (1) the connection-failure telemetry path isn't gated on use_kernel, so the Python wrapper still emits failure telemetry on the kernel path; (2) the phase-7 Session kwargs are forwarded unconditionally while the kernel dependency floor stays ^0.2.0, a potential connect-time break if an older wheel is installed.
Other findings
- 🔵 Low — The kernel-path guard in
is_telemetry_enableddisables Python-side telemetry only for the successful connection path. When ause_kernel=Trueconnection fails duringsession.open(), thisconnection_failure_log(...)call still fires withenable_telemetry=kwargs.get("enable_telemetry", True)— it is not gated onuse_kernel. So the Python wrapper still emits a connection-failure telemetry event on the kernel path, which is the kind of wrapper-owned telemetry this PR set out to suppress foruse_kernel. It may be intentional (the kernel session never opened, so it couldn't emit its own failure telemetry), but if the goal is "no Python-side connector telemetry on the kernel path," this branch is an unguarded exception worth confirming.
8261c66 to
088ff16
Compare
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — a clean, well-tested change. Python-side telemetry is correctly disabled for kernel connections (session is assigned before the is_telemetry_enabled check), and kernel telemetry_enabled follows enable_telemetry as intended. One low note: the kernel path's default telemetry_batch_size differs from the Thrift/SEA path when the caller leaves it unset.
Addresses: - #3833615986 at src/databricks/sql/telemetry/telemetry_client.py:125 - #3833615990 at src/databricks/sql/backend/kernel/client.py:176 - #3833661667 at src/databricks/sql/backend/kernel/client.py:419 - #3858839800 at src/databricks/sql/session.py:228 - #3876277610 at src/databricks/sql/session.py:260 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the telemetry wiring is correct. Session.__init__ sets use_kernel before Connection calls is_telemetry_enabled, so the kernel-path telemetry suppression and the forwarded kernel kwargs (identity, telemetry_enabled, batch size, circuit breaker) are consistent with the described design and well covered by tests. One low-severity cleanup: two leftover probe files were committed.
Signed-off-by: Jay Xiao <jay.xiao@databricks.com>
Signed-off-by: Jay Xiao <jay.xiao@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — the telemetry parameter forwarding is coherent and well covered by tests. The use_kernel opt-out in is_telemetry_enabled (telemetry_client.py:125) uses getattr(..., False), which correctly handles both real sessions (attribute set in _create_backend before the check at client.py:432) and the MagicMock-based tests. One minor default-parity inconsistency noted inline (Low). Separately, note that the connection_failure_log path (client.py:404-419) is not gated on use_kernel and would still emit Python-side failure telemetry for a failed kernel connection — a small gap in the "no duplicate kernel-path telemetry" goal, though arguably acceptable since it's the failure path.
Addresses: - #3876849398 at src/databricks/sql/session.py:276 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — clean, well-tested change forwarding telemetry config to the kernel Session and disabling duplicate wrapper telemetry on the kernel path. Option threading, defaults (enable_telemetry→False, batch_size/circuit_breaker parity), force_enable_telemetry semantics, and the getattr guard on connection.session all check out, and session creation precedes telemetry init so the new check is safe. One low-severity completeness note: the connect-time connection_failure_log path is not gated by the new kernel opt-out.
Addresses: - #3876884856 at src/databricks/sql/telemetry/telemetry_client.py:125 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — clean, well-commented, and well-tested telemetry forwarding into the kernel Session. The use_kernel gate in is_telemetry_enabled is safe (session.use_kernel is set in _create_backend before the check at client.py:436), key names line up across session.py/client.py/kernel/client.py, and the intentional default divergences (enable_telemetry False on kernel, force_enable_telemetry as wrapper-only) are documented and covered by tests. One low-severity observability note on the failure-path suppression is filed inline; note also that force_enable_telemetry=True has no effect on the kernel path (wrapper off + kernel follows only enable_telemetry), which is intentional but may surprise users.
Signed-off-by: Jay Xiao <jay.xiao@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — well-scoped and thoroughly tested; the kwarg names, defaults, and the auth/common.py:55 citation all check out, and the session→client ordering means session.use_kernel is set before is_telemetry_enabled reads it. One low note: the kernel path forwards enable_telemetry as None when unset (deferring to a kernel default) while the wrapper path gates the same case behind a server feature flag, which may diverge in the never-opted-in case.
Addresses: - #3877217286 at src/databricks/sql/session.py:263 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — clean, well-tested change forwarding telemetry config to the kernel binding and suppressing wrapper-side telemetry on the use_kernel path. Parity/default claims in the comments verify against the code, and there's no circular-import issue. One low-severity note about the explicit enable_telemetry=True case relying on an unverifiable kernel-side feature-flag gate.
Addresses: - #3877271142 at src/databricks/sql/session.py:275 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Medium
Solid, well-tested change — telemetry forwarding, kernel-path suppression of wrapper telemetry, and the is_telemetry_enabled short-circuit all look correct, and self.session is assigned before is_telemetry_enabled(self) reads session.use_kernel. One medium concern: the kernel telemetry_circuit_breaker_enabled default of True diverges from the wrapper's effective default of False, contradicting the parity rationale in the comment.
Addresses: - #3877391550 at src/databricks/sql/session.py:290 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Medium
Mostly solid — the telemetry short-circuit (is_telemetry_enabled → False for use_kernel), the failure-log suppression at client.py:418, and the enable_telemetry/telemetry_batch_size/circuit-breaker mapping all look correct and are well-tested. One medium concern: the new phase-7 Session identity kwargs are forwarded unconditionally to the real PyO3 constructor while the kernel wheel pin stays at ^0.2.0 and every test mocks Session, so nothing proves the installed wheel accepts them.
Addresses: - #3877431848 at src/databricks/sql/backend/kernel/client.py:180 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Medium · 1 Low
Mostly solid, well-tested telemetry-forwarding change — 1 medium concern worth addressing. The kwarg-gating helper falls open on introspection failure, which could break every use_kernel connection against a non-introspectable PyO3 binding (the test only covers a Python-function fake, so this path is unverified). Also an empty _scratch_ test file looks accidentally committed.
Other findings
- 🔵 Low — This appears to be accidentally-committed scratch debris: a brand-new, empty file named
_scratch_kernel_kwargs_probe.pyundertests/unit/. It contains no code and its_scratch_name signals a throwaway probe. pytest won't collect it (name doesn't matchtest_*), so it's harmless functionally, but it shouldn't land in the repo. Please delete it before merge.
Addresses: - #3877505593 at src/databricks/sql/backend/kernel/client.py:194 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks solid — the telemetry-forwarding wiring is coherent and the kwarg-gating (_kernel_session_accepts_kwarg falling closed on introspection failure), the use_kernel short-circuit in is_telemetry_enabled, and the failure-log suppression in client.py are all consistent with each other and well covered by new tests. One cleanup: an empty _scratch_ file was committed. Minor nit (not filed inline): _kernel_telemetry_kwargs re-runs inspect.signature(_kernel.Session) once per candidate kwarg (~14x per open_session); caching the parameter set once would be marginally cleaner, though the cost is negligible since it only runs at session open.
Other findings
- 🔵 Low — This appears to be a leftover debugging artifact: the filename is prefixed with
_scratch_and the file is committed empty (0 bytes). It adds no test coverage and pollutestests/unit/. Please drop it from the PR (git rm) — the actual kwarg-gating coverage already lives intest_kernel_client.pyandtest_telemetry.py.
Signed-off-by: Jay Xiao <jay.xiao@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Nit
Looks good — the telemetry-forwarding wiring is correct and unusually well-tested. The intentional divergences from the Thrift/SEA paths (unset enable_telemetry/circuit-breaker deferring to the kernel, batch-size defaulting to DEFAULT_BATCH_SIZE, wrapper telemetry suppressed on the kernel path) are documented and verified by unit tests, and the fixed-signature/non-introspectable PyO3 gate is exercised directly. Only one minor efficiency nit (repeated inspect.signature calls).
Other findings
- ⚪ Nit —
_kernel_telemetry_kwargscalls_kernel_session_accepts_kwarg(name)once per candidate (up to 14 times), and each call re-runsinspect.signature(_kernel.Session). Since the signature is identical across all candidates, it could be introspected once (e.g. resolve the parameter set / VAR_KEYWORD presence a single time, then filter). This runs only once per session-open so the cost is negligible — flagging as a preference, not a defect.
Summary
databricks_sql_kernel.Session: driver/runtime identity,telemetry_enabled, andtelemetry_batch_size.telemetry_enabledfromenable_telemetry; keepforce_enable_telemetryas a Python wrapper-only feature-flag bypass concept.use_kernel=Trueconnections so wrapper telemetry does not duplicate kernel-owned telemetry.enable_telemetry.Tests
.venv/bin/python -m pytest tests/unit/test_telemetry.py -q.venv/bin/python -m pytest tests/unit/test_session.py -q.venv/bin/python -m pytest tests/unit/test_kernel_client.py -q