fix: emit glyph marker for symbolic/composite fonts with no usable cmap

[wittjeff]

Follow-up to #299 (thanks @yalsaffar for the report).

The guard added in #299 only fired when a /ToUnicode cmap parsed (cmap_initialized == true) and merely missed the code. A symbolic or Type0/CID font whose /ToUnicode fails to parse (could not find cmap in '/ToUnicode'), or that carries none, sets cmap_initialized = false, skips the guard, and falls through to the Standard/WinAnsi tables — which fabricate unrelated text again (e.g. !"#).

This drops the cmap_initialized requirement for symbolic/composite fonts: their codes bear no relation to the standard encodings, so an unresolved code must surface as a GLYPH<> marker regardless of whether a cmap was parsed. Non-symbolic fonts are unaffected (they legitimately use the declared encoding).

Adds two tests: a symbolic font with no usable /ToUnicode (must emit markers, not fabricate) and a non-symbolic control (encoding preserved). Reproduced against the current build; the change is a one-line guard widening.

[github-actions]

✅ DCO Check Passed

Thanks @wittjeff, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[PeterStaar-IBM]

@wittjeff it seems the regression tests are failing. Any idea why?

[wittjeff]

Yes — my fault, and the CI diff pinpointed it. The widening was on the wrong axis; fixed and re-pushed.

What broke. The 29 failing pages all follow one pattern: real ASCII text turned into markers (Specify → GLYPH<83>GLYPH<112>…, Convert, 2.4.). I pulled the font dictionaries of every affected document (font_01.pdf, deep-mediabox-inheritance.pdf, the five dln_* DocLayNet docs): every broken font is symbolic-flagged but declares an encoding — either /Encoding /WinAnsiEncoding outright or an /Encoding dict with /Differences. For those fonts, codes outside /Differences legitimately resolve through the declared base encoding (9.6.6); producers set the symbolic flag spuriously on such fonts all the time. By dropping the cmap_initialized requirement without checking for a declared encoding, the guard started eating the legitimate fallback.

The case #322 is actually about is narrower. @yalsaffar's logs show both warnings together:

could not find cmap in '/ToUnicode'
font-encoding not defined, falling back to STANDARD

i.e. a symbolic font with neither a usable cmap nor any declared /Encoding. With no statement of intent from the producer at all, falling through to the standard tables is fabrication — that's the only case the guard should now catch beyond the original #299 one.

The fix (one condition, pushed as a follow-up commit):

const bool has_declared_simple_encoding =
  (has_explicit_encoding and encoding != CMAP_RESOURCES) or diff_initialized;

if(subtype==TYPE_0 or
   (is_symbolic and (cmap_initialized or not has_declared_simple_encoding)))

Case table:

font	cmap	declared encoding	result
#3802 (في → #)	parsed, misses code	any	marker (unchanged from #299)
#322 report	failed to parse / absent	none	marker (the new case)
font_01.pdf, dln_*, deep-mediabox	none	WinAnsi or Differences	declared encoding honored (regression fixed)
Type0/CID	—	—	marker (unchanged)

I also fixed the synthetic test to match the reported shape: the no-cmap symbolic test now also omits /Encoding (and uses a subset-style BaseFont name so the core-14 alias table stays out of the way), and there's a new control asserting that a symbolic-flagged font with declared WinAnsi keeps its text — which is exactly the shape that broke CI.

Verified locally against the full regression suite: test_parse.py, test_threaded_parse.py, test_threaded_render.py all green, plus the 7 unit tests in test_tounicode_fallback.py.

One residual gap, deliberately not touched here: a symbolic no-encoding font whose name matches a core-14 alias (e.g. an ABCDEF+Arial subset) resolves through the base-font AFM table before reaching this guard, so it can still fabricate. That path predates #299 and behaves identically before/after this PR; happy to look at it separately if it shows up in the wild.