Conversation
Add test case 26 (`test_primary_key_dedup_composite`) to ensure records are correctly deduplicated at the API extraction layer. This prevents duplicate-key violations during the merge process by keeping only the first occurrence of each composite key tuple.
Re-render stream SQL queries after start-hooks have executed so that runtime variables (execution.*, store.*, state.*) are resolved. This allows stream SQL to reference dynamic values like state from hooks. Also initializes runtime state earlier during replication compilation and adds the new token prefixes to the evaluator.
Check if `syncKey` is not empty before warning about a missing `SLING_STATE` environment variable. This prevents the warning from being incorrectly triggered for API syncs that do not require incremental state tracking.
…at/csv-skip-lines
Fix nil pointer dereference in error message when MakeAuthenticator fails by using ep.Authentication.Type() instead of calling method on nil authenticator. Add default case to return error for unsupported authentication types instead of silently returning nil.
… tables
When the FE proxy returns a 307 redirect, the previous code saved only
the redirected port (conn.fePort), then reconstructed subsequent chunk
URLs by replacing the port in the original FE-proxy URL. This left the
FE-proxy hostname intact, so chunk 2+ went to <fe-proxy-host>:<cn-port>
instead of <cn-host>:<cn-port>, causing a context deadline exceeded on
every multi-chunk stream load.
Fix: store the full redirected Host ("host:port") in conn.feHost and
replace fu.U.Host in the URL, so all subsequent chunks are sent directly
to the CN/BE node that the FE proxy redirected to.
…reaming Before this change, chunk 1 sent its full body to the FE proxy just to receive a 307 redirect, then re-sent the same body to the CN. For large chunks this caused the FE proxy to time out under load. Send an empty PUT to the FE proxy upfront (before any data goroutines start) to trigger the 307 cheaply and cache the CN host. All chunks including the first then go directly to CN with no proxy buffering. Falls back gracefully to the existing per-chunk redirect handling if the probe returns a non-3xx response. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added `Suite` field to `testCase` struct to specify a nested test suite file path. - Introduced `loadCLITestCases` function to recursively read and merge test cases from nested YAML files. - Updated `TestCLI` to use `loadCLITestCases`, allowing tests to be split into multiple files for better organization.
Previously, http_timeout was only set when the httpfs extension was loaded. This caused operations using DuckDB's local HTTP stream to hit the 30s default timeout (issue #770). Extract session settings into a dedicated getSessionSettingsSQL function applied to all sessions, and allow overriding via the http_timeout property.
- Added a bounded grace period (500ms) in `Describe` to wait for an async stderr error if the result is empty, preventing a misleading "no columns found" error. - Added regression test pipeline (`p.37`) and CLI suite test to verify the real Catalog Error is surfaced.
Update several dependencies in go.mod including AWS SDK S3, pgx, golang.org/x packages, and grpc to their latest versions to ensure stability and access to recent fixes.
…idelity The sqlldr control file template was missing a CHARACTERSET clause, causing SQL*Loader to interpret Sling's UTF-8 CSV using the client NLS_LANG charset (a single-byte default). This mangled multibyte characters into U+FFFD replacement characters or double-encoded them, silently corrupting VARCHAR2/NVARCHAR2/CLOB text on the bulk loader path. Adding `CHARACTERSET AL32UTF8` forces correct UTF-8 interpretation. Also adds a test (r.119) verifying byte-level fidelity of multibyte characters loaded via the sqlldr path.
Guard `ChangeColumn` calls in `CastVal` with a `!col.Sourced` check so that columns with an explicit/source-declared type are no longer silently reverted to string or widened to decimal when individual values fail to parse. Previously, a user-declared datetime column would be reverted to text mid-stream whenever a value couldn't be parsed as time, causing a mismatch between the temp table schema and the final table DDL and breaking valid data. Resolves discussion #763. Adds test `r.120.explicit_cast_datetime.yaml` and fixture `disc763_explicit_cast.csv` to verify the declared timestampz cast survives streaming end-to-end.
fix(redshift): support serverless URLs and fix DELETE alias in merge
- Upgrade OpenTelemetry to v1.43.0 - Upgrade Docker to v28.5.2 - Upgrade go-git to v5.19.1 - Upgrade go-jose to v4.1.4 / v3.0.5 - Upgrade golang.org/x/image to v0.38.0
Redshift does not support table aliases in DELETE/UPDATE statements.
Updated the `merge_change_capture` and `merge_change_capture_soft`
templates to remove the `tgt` alias from the target table and use
`{src_tgt_pk_equal_tbl}` instead of `{src_tgt_pk_equal}`. Also added
test coverage for the `change_capture_soft` merge strategy.
Handle the wrapped/nested list form (e.g. `primary: [[user_id, brand_id]]`) by flattening it into a plain []string, matching the flat form behavior. Previously this shape caused SetKeys to fail with "could not set primary key. Did not find column" because the nested list was treated as a single phantom column name. Empty tokens are now trimmed to avoid trailing separator issues.
…redirect-host fix(starrocks): preserve redirect host in stream load for multi-chunk tables
Fixed SQL server pre-fetched SP token authentication
feat: skip lines for csv files
Add handling for `ActiveDirectoryServicePrincipalAccessToken` in the `BcpImportFile` method. The pre-fetched access token is sourced from the password property, written to a temporary file, and passed to `bcp` via the `-G` and `-P` arguments to prevent it from being treated as a plain SQL password.
When a user explicitly casts a column to a bare `decimal` (type only, no precision), the Coerce function was locking the sample-inferred precision/scale as if the user had pinned it. A limited inference sample can under-size the decimal (e.g. seeing only values up to "899" → decimal(3,1)), causing later rows to silently overflow or saturate on databases that tightly honor decimal(P,S), such as StarRocks. The decimal *type* remains pinned (Sourced=true) so it survives streaming (per discussion #763), but the inferred precision/scale are now cleared so downstream inference applies safe minimums. Sized decimal casts (e.g. decimal(24,6)) are unaffected and still pin their precision/scale as before. Adds TestCoerceUnsizedDecimalCast to cover both the unsized and sized cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.