Skip to content

[hotfix][runtime] Return UNKNOWN for null regex LIKE predicates#4483

Open
wolfkill wants to merge 1 commit into
apache:masterfrom
wolfkill:codex/flink-cdc-bug-hunt
Open

[hotfix][runtime] Return UNKNOWN for null regex LIKE predicates#4483
wolfkill wants to merge 1 commit into
apache:masterfrom
wolfkill:codex/flink-cdc-bug-hunt

Conversation

@wolfkill

Copy link
Copy Markdown

What is the purpose of this pull request?

This PR fixes nullable transform LIKE / NOT LIKE predicates that are translated to the two-argument regex helpers.

The three-argument SQL LIKE helper already returns SQL UNKNOWN (null) when any operand is null. The regex-style two-argument helper instead called Pattern.matcher(...) directly and threw an NPE when the input string was null. This caused nullable transform predicates such as name like 'A.*' to fail during evaluation instead of producing UNKNOWN.

Brief change log

  • Return Boolean from the two-argument regex like / notLike helpers.
  • Return null when the input string or regex pattern is null.
  • Use LogicalFunctions.not(...) for nullable notLike semantics.
  • Add function-level and Janino-translated predicate regression coverage.

Verifying this change

This change added tests and can be verified as follows:

  • Added unit coverage in StringFunctionsTest for nullable two-argument regex LIKE / NOT LIKE operands.
  • Added regression coverage in JaninoCompilerTest for translated nullable LIKE / NOT LIKE predicates.
  • Verified RED before the fix:
    • mvn -U -pl flink-cdc-runtime -am -Dtest=JaninoCompilerTest#testTranslatedLikePredicateReturnsUnknownForNullOperand -DfailIfNoTests=false -Dspotless.check.skip=true -Dcheckstyle.skip=true test
    • Failed with NullPointerException from StringFunctions.like(StringFunctions.java:70).
  • Verified after the fix with Java 17:
    • mvn -pl flink-cdc-runtime -am -Dtest=JaninoCompilerTest#testTranslatedLikePredicateReturnsUnknownForNullOperand -DfailIfNoTests=false -Dspotless.check.skip=true -Dcheckstyle.skip=true test
    • mvn -pl flink-cdc-runtime -am -Dtest=StringFunctionsTest,JaninoCompilerTest -DfailIfNoTests=false -Dspotless.check.skip=true -Dcheckstyle.skip=true test
    • mvn -pl flink-cdc-runtime -am -DfailIfNoTests=false -Dspotless.check.skip=true -Dcheckstyle.skip=true test
    • mvn -pl flink-cdc-runtime -am -DskipTests test

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: OpenAI Codex

@haruki-830

Copy link
Copy Markdown
Contributor

Hi @wolfkill, I noticed that this PR overlaps with an effort I’m currently working on.

This PR fixes NULL handling for the legacy two-argument regex LIKE / NOT LIKE helpers. My work aims to provide a more complete alignment between YAML Transform predicates and Flink SQL semantics, including three-valued logic for comparison, IN, and BETWEEN, as well as SQL-compatible LIKE / NOT LIKE, ESCAPE, and SIMILAR TO semantics.

Since the changes touch related runtime functions and expression compilation paths, would you be willing to review my PR as well? I’d especially appreciate your feedback on compatibility with the existing regex behavior and how we should coordinate the overlapping changes.

PR: https://github.com/apache/flink-cdc/pull/4484

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants