perf(trino-driver): native REST polling to cut coordinator wait/drain latency - #11548
Open
aman-shrivastava01 wants to merge 2 commits into
Open
perf(trino-driver): native REST polling to cut coordinator wait/drain latency#11548aman-shrivastava01 wants to merge 2 commits into
aman-shrivastava01 wants to merge 2 commits into
Conversation
… latency presto-client slept ~800ms on every nextUri poll, including while Trino was still queued and after rows were already flowing. Talk to POST /v1/statement directly with short wait-phase backoff and immediate drain. Use OSS TrinoQuery (AT TIME ZONE + DATE promotion) instead of PrestodbQuery. Keep formatAnsi escaping, poll headers, and X-Trino-Source=nodejs-client. Signed-off-by: Aman Shrivastava <aman.shrivastava@coupa.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Check List
Description of Changes Made
@cubejs-backend/trino-driverno longer wrapspresto-client/PrestoDriver. It talks to Trino's HTTP protocol directly (POST /v1/statement, thenGET nextUri). The protocol is the same; the poll cadence is not.presto-clientslept ~800ms on every poll, including while the query was still queued and after rows were already flowing. That wait dominates interactive latency. PrestoDB is unchanged and still usespresto-client.Polling
drainIntervaldefault0).checkInterval: 800restores constant-interval polling for both phases.Tune with
CUBEJS_DB_TRINO_POLL_*ordriverFactory(pollBackoff,drainInterval,checkInterval). Invalid poll env values are rejected.Compatibility
formatAnsi.nextUripoll, including worker hosts./v1/infotestConnection, andX-Trino-Sourcedefaultnodejs-clientare unchanged.Behavior changes
dialectClass()returnsTrinoQuery(AT TIME ZONEplus DATE promotion), matching QueryBuilder'strinomapping. The previous wrapper selectedPrestodbQuery.CUBEJS_DB_QUERY_TIMEOUT(default10m) is sent asquery_max_run_timeon every statement, not only streams.useris not set,X-Trino-Userusesbasic_auth.userso the session user matches the authenticated principal.release()destroys the agents.TrinoDriverno longer extendsPrestoDriver.Test plan
packages/cubejs-trino-driver: lint and 63 unit testspackages/cubejs-backend-shared: Trino poll/source env parsing testsyarn integration:trinoagainst a live Trino container (11/11)packages/cubejs-testing:yarn smoke:trinotimezone snapshot