[FLINK-39797] Support disabling automatic table creation#4420
Open
litiliu wants to merge 1 commit into
Open
Conversation
56f5a77 to
85e3cf9
Compare
85e3cf9 to
86e3414
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Introduce a framework-level sink option
schema.create.modeto control target table creation behavior.The new
ERROR_IF_NOT_EXISTSmode allows users to skip automatic target table creation and fail fast when the target table does not exist. This is useful when target tables are managed by an external platform.This PR currently enables
ERROR_IF_NOT_EXISTSfor Fluss and MaxCompute only. Other pipeline connectors keep their existing create-table behavior and will report unsupported when configured withschema.create.mode=ERROR_IF_NOT_EXISTS.Brief change log
schema.create.modewith defaultCREATE_IF_NOT_EXISTS.ERROR_IF_NOT_EXISTShandling through a framework-levelMetadataApplierwrapper.SupportsExistingSchemaValidationfor sinks that can validate existing target schemas.ERROR_IF_NOT_EXISTSin this phase.Verifying this change
schema.create.mode.schemaExists()returns false when target table is missingvalidateExistingSchema()passes when schema and primary key matchERROR_IF_NOT_EXISTSwrapper fails when target table is missing and does not create the tableSchemaCreateModeMetadataApplierTestMaxComputeMetadataApplierTestspotless:checkgit diff --checkDocumentation
This introduces a new user-facing sink option:
schema.create.modeCREATE_IF_NOT_EXISTSdefaultERROR_IF_NOT_EXISTSDocumentation should mention that
ERROR_IF_NOT_EXISTSis currently supported only by Fluss and MaxCompute. Other connectors will fail during schema application because they do not implement existing target schema validation yet.