Skip to content

Do not fold dotless i to ASCII i in reference labels - #308

Merged
jgm merged 1 commit into
commonmark:masterfrom
DePasqualeOrg:fix/dotless-i-case-folding
Sep 12, 2026
Merged

jgm merged 1 commit into
commonmark:masterfrom
DePasqualeOrg:fix/dotless-i-case-folding

Conversation

@DePasqualeOrg

Copy link
Copy Markdown
Contributor

This PR was developed with Claude Code, but solves a real issue I encountered with this library and is based on verified evidence. It underwent multiple rounds of automated review. The following description was produced by Claude Code. I’m happy to make any changes as needed.


The spec matches link labels under Unicode case fold. normalizeReference approximates that with .toLowerCase().toUpperCase(), which merges dotless i (U+0131) with ASCII I and i, because 'ı'.toUpperCase() is 'I'. CaseFolding.txt has no entry for U+0131 at all, so it folds to itself. The only nearby mapping goes the other way, and 0049; T; 0131 is the Turkic profile, which CommonMark does not specify.

This is a regression rather than a deliberate behavior. The generated table removed in b8fc0c7 for #168 left dotless i alone, and the changelog entry describes the replacement as an accurate Unicode case fold, which it is for every character but this one.

cmark keeps the two labels distinct:

[I]: /a

[ı]: /b

[a][i] [b][ı]
cmark:         <p><a href="/a">a</a> <a href="/b">b</a></p>
commonmark.js: <p><a href="/a">a</a> <a href="/a">b</a></p>

In commonmark.js the second definition collides with the first, and the earlier one wins.

Dotless i is the only character affected. Comparing normalization output across every BMP codepoint before and after this change, U+0131 is the one that differs, so labels without it keep exactly the old expression.

Three regression cases are added: a bare [ı] against an [I] definition, which no longer resolves; the two-definition collision above; and [FIQ]/[Fıq], which pairs dotless i with characters that still need folding. All three fail against master.

The indexOf guard is purely an optimization; the replace alone is correct for every input. Performance is unchanged, and since #168 weighed it, it was measured here too. Amortized over 2,000 parses per sample, bench/samples/README.md (the default make bench input) moves -0.7%, and the two reference-heavy samples land within a few percent either way across repeated sessions with no consistent direction.

This does change rendering for existing documents: a [ı] reference written against an [I] definition now finds no definition and renders as literal text. That is the intent, but it is a visible difference rather than an internal one. parser.refmap keys change for these labels as well, though normalized labels reach neither the AST nor any renderer.

@wooorm

wooorm commented Sep 11, 2026

Copy link
Copy Markdown

@jgm there was some conversation about this in micromark/micromark#232. Personally I have no strong opinions on where this should go. Spec/cmark. Or all the JS projects.

@jgm

jgm commented Sep 12, 2026

Copy link
Copy Markdown
Member

looks good to me, thanks.

@jgm
jgm merged commit f73088a into commonmark:master Sep 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants