[telemetry improvement]: emit sql_operation, auth_type, driver_connection_params - ship even if null#396
Conversation
…tion_params Aligns the Node.js telemetry payload with the receiver schema (JDBC parity). Several fields were being collected by the producer/aggregator but dropped on the floor in the exporter: - `sql_operation.operation_detail.operation_type` (CREATE_SESSION / DELETE_SESSION / EXECUTE_STATEMENT / LIST_*) on both connection and statement events. - `sql_operation.is_compressed` on statement events when CloudFetch compression observability is available. - `sql_operation.execution_result` now resolves to `FORMAT_UNSPECIFIED` when result-set metadata isn't available (DDL/DML, or SELECTs closed without fetching), so the `sql_operation` block fires on every statement-complete event instead of being skipped. - Top-level `auth_type` from `DriverConfiguration.authType`. - New `driver_connection_params` block carrying `host_info.host_url`, `http_path`, `enable_arrow`, `enable_direct_results`, `socket_timeout`, `enable_metric_view_metadata`, `cloud_fetch_enabled`, `lz4_enabled`, `retry_max_attempts`, `cloud_fetch_concurrent_downloads`. Both `auth_type` and `driver_connection_params` are gated behind the existing authenticated-export guard (same path as `system_configuration`), since `host_url` and `http_path` are workspace-correlated identifiers that must not ship on the unauthenticated endpoint. Co-authored-by: Isaac
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Drop four Node-specific fields that aren't in the receiver's `DriverConnectionParameters` proto schema: - cloud_fetch_enabled - lz4_enabled - retry_max_attempts - cloud_fetch_concurrent_downloads These would be ignored at deserialization on the receiver side anyway. Remaining block contains only proto-defined fields: `host_info`, `http_path`, `enable_arrow`, `enable_direct_results`, `socket_timeout`, `enable_metric_view_metadata`. Co-authored-by: Isaac
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
The four Node-specific fields dropped in the previous commit have no
proto equivalent (CloudFetch enablement, LZ4, retry config, concurrent
downloads aren't tracked at the receiver). But two unrelated proto
fields can be filled from existing Node config:
- `mode = "THRIFT"` — Node.js always uses Thrift transport (no SEA
client). JDBC's `DatabricksClientType` enum is `SEA | THRIFT`.
- `use_proxy` — derived from `ConnectionOptions.proxy`. Stored on
DBSQLClient at `connect()` time alongside the existing host /
httpPath / authType fields, then surfaced through a new
`DriverConfiguration.useProxy` field.
Co-authored-by: Isaac
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Summary
Although these can be extracted from close monitoring of logs, it is good to populate these upfront. The producer + aggregator already had the data; the exporter was dropping it on the floor and the receiver-side rows were showing up with empty
sql_operationand missingauth_type/driver_connection_params.sql_operation.operation_detail.operation_type— on both connection (CREATE_SESSION/DELETE_SESSION) and statement (EXECUTE_STATEMENT/LIST_*) events. Already aggregated, just not exported.sql_operation.is_compressed— surfaced from the aggregator's existingcompressedflag (true-on-any-CloudFetch-chunk).sql_operation.execution_result—mapResultFormatToTelemetryType(undefined)now resolves toFORMAT_UNSPECIFIED(matching its existing default-case fallback) instead ofundefined. This makes the existingif (metric.resultFormat || ...)gate light up for DDL/DML and SELECTs that were closed without iterating rows, so every statement-complete row gets a populatedsql_operationblock.auth_type— fromDriverConfiguration.authType.driver_connection_paramsblock —host_info.host_url,http_path,enable_arrow,enable_direct_results,socket_timeout,enable_metric_view_metadata,cloud_fetch_enabled,lz4_enabled,retry_max_attempts,cloud_fetch_concurrent_downloads. Field names mirror the JDBCDriverConnectionParametersschema.auth_typeanddriver_connection_paramsare gated behind the sameincludeCorrelation(authenticated-export) guard assystem_configuration—host_urlandhttp_pathare workspace-correlated identifiers and must not ship on the unauthenticated endpoint. Existing testomits workspace_id, session_id, statement_id from unauth payloadcontinues to enforce that contract.driverConfigonly rides on connection metrics today, so the new top-level blocks ride along withCREATE_SESSION/DELETE_SESSIONevents; receivers can join statement-level rows to the corresponding connection row bysession_id.Test plan
npm run type-check(build) — cleannpx mocha --require ts-node/register tests/unit/telemetry/*.test.ts— 300 passingentry.sql_statement_id is not null) after deploy and confirmsql_operationis non-null on every statement row and carriesoperation_detail.operation_typeentry.auth_typeandentry.driver_connection_params.host_info.host_urlpopulate onCREATE_SESSIONrows in the authenticated stream and remain absent in the unauthenticated streamThis pull request and its description were written by Isaac.