MDEV-39530 Collation mix issue after update

[gengtianuiowa]

Description

SELECT IF((@p:=(SELECT f.Path FROM file f WHERE f.ID=1 AND f.Path REGEXP "^xx"))!="",1,2); fails with ER_CANT_AGGREGATE_2COLLATIONS ("Illegal mix of collations") when then connection collation (utf8mb3_uca1400_ai_ci, the post-MDEV-25829 default) differs from the column collation (utf8mb3_unicode_ci). The query worked in 11.4.

Root cause

The comparison (@p := subquery) != "" forces the != operator to pick a single collation for both operands. Every string expression carries two attributes that drive this decision:

• a collation, and
• a derivation (a.k.a. coercibility) that expresses how strongly that collation
  is attached to the value. Lower numeric derivation = stronger.

For the failing query the two operands are:

Operand	Collation	Derivation (coercibility)	Source
@p := (SELECT f.Path ...)	utf8mb3_unicode_ci	DERIVATION_COERCIBLE (6)	inherited from column Path
"" (empty string literal)	utf8mb3_uca1400_ai_ci	DERIVATION_COERCIBLE (6)	inherited from collation_connection

Here both operands sit at DERIVATION_COERCIBLE (6) with different collations, so aggregation fails and the error is raised.

Fix

Make Item_func_set_user_var use DERIVATION_USERVAR so that assigning a user variable has the same derivation as reading it back, keeping it weaker than a literal. With the assignment at level 5 and the literal at level 6, the operands are now at different levels, so aggregation simply coerces the empty string to the variable's collation and returns a result — regardless of the connection collation.

Release Notes

N/A

How can this PR be tested?

Execute the main.user_var_collation test in mysql-test-run. It sets a connection collation that differs from the column collation and runs the reporter's query.

Before this change:

The query raised an error instead of returning a result:

SELECT IF((@p:=(SELECT f.Path FROM file f WHERE f.ID=1 AND f.Path REGEXP "^xx"))!="",1,2);
ERROR 1267 (HY000): Illegal mix of collations (utf8mb3_unicode_ci,COERCIBLE) and (utf8mb3_uca1400_ai_ci,COERCIBLE) for operation '<>'

After this change:

The query returns the expected result, matching 11.4 behavior:

SELECT IF((@p:=(SELECT f.Path FROM file f WHERE f.ID=1 AND f.Path REGEXP "^xx"))!="",1,2);
IF((@p:=(SELECT f.Path FROM file f WHERE f.ID=1 AND f.Path REGEXP "^xx"))!="",1,2)
2

The coercibility of a user variable assignment now matches reading it back
(both DERIVATION_USERVAR), reflected in the updated main.user_var result:

select coercibility(@a:=_latin2'test');
coercibility(@a:=_latin2'test')
-6
+5

main.user_var_collation, main.user_var, and the ctype_* / main.variables suites all pass:

main.user_var                            [ pass ]
main.ctype_utf8                          [ pass ]
main.ctype_utf8mb4                       [ pass ]
main.ctype_latin1                        [ pass ]
main.ctype_ucs                           [ pass ]
main.ctype_collate                       [ pass ]
main.ctype_utf8mb3_uca1400_ai_ci         [ pass ]
main.variables                           [ pass ]
main.user_var_collation                  [ pass ]
Completed: All tests were successful.

main suite passed without failure relevant to this change.

Basing the PR against the correct MariaDB version

• This is a bug fix, and the PR is based against the branch 12.3.

Copyright

All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.

[gemini-code-assist]

Code Review

This pull request addresses MDEV-39530 by updating the collation derivation for user variable assignments in sql/item_func.cc. By changing DERIVATION_COERCIBLE to DERIVATION_USERVAR, it ensures consistent collation handling and prevents "Illegal mix of collations" errors. Corresponding test cases and result files have been updated to reflect these changes. I have one piece of feedback regarding a potentially redundant assignment in fix_length_and_dec.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

This line appears to be redundant. The fix_fields function, which is called just before fix_length_and_dec, already sets the correct derivation. Removing this line avoids redundant assignments or incorrect overwriting of DERIVATION_NUMERIC.

[gkodinov]

This is indeed a good suggestion. I'd fix the fix_fields's setting and remove these.

[gkodinov]

This is a preliminary review. Thank you for your contribution!

LGTM, except the cleanup suggestion below.

Please stand by for the final review.