Unicode Normalization Lab

One visible name. Six byte sequences. Four different dashboards.

This interactive lab shows how Unicode-equivalent or visually confusable attribute values can fragment one logical telemetry identity. The default exact-match dashboard reports a healthy 1% error rate while the incident-carrying population is excluded and the canonical view reaches 6.2%.

The canonicalization model is an executable sed program. A dependency-free Node.js service sends summary records through that program, aggregates the transformed output, renders the result, and exposes correlated OTLP-shaped metrics, logs, and traces.

The failure

The model emits six forms of café-api:

Form	Difference
NFC	Precomposed é (U+00E9)
NFD	e followed by combining acute accent
NFKC case	Fullwidth letters and punctuation
ZWSP	An embedded zero-width space
NBHY	A non-breaking hyphen instead of ASCII -
SHY	An embedded soft hyphen

They look identical or nearly identical in an ordinary dashboard, but their code points and UTF-8 bytes differ. Exact equality therefore creates separate groups.

Unicode Standard Annex #15 defines canonical and compatibility equivalence plus the NFC, NFD, NFKC, and NFKD normalization forms. Compatibility normalization must not be applied blindly to arbitrary prose because it can erase meaningful distinctions. This lab instead demonstrates a controlled mapping for a deliberately restricted identity field.

OpenTelemetry defines service.name as a string that must remain the same across horizontally scaled instances of a service. See the OpenTelemetry service attributes registry.

Why telemetry.sh helps

The lab emits the signals needed to diagnose the split:

• telemetry.attribute.group.count
• telemetry.attribute.coverage
• http.server.request.error_rate
• a warning log containing the raw and canonical values
• an error log containing the incident value's UTF-8 bytes
• query spans showing which dashboard view lost identity coverage

The strongest pattern keeps two fields:

1. a controlled canonical value for grouping;
2. the original value or fingerprint for investigation.

That preserves operational correlation without pretending every Unicode transformation is semantically safe.

Run locally

Requirements:

• sed
• GNU Make
• Node.js 20 or newer

npm ci
make check
make run

Open http://127.0.0.1:3000.

Useful endpoints:

GET /api/simulate
GET /api/telemetry
GET /healthz

All controls are also query parameters:

/api/simulate?variants=5&requests=1000&incident=4&incidentError=45&threshold=6

Run with Docker

docker compose up --build

Then open http://127.0.0.1:3000.

Executable model

model/canonicalize.sed contains the complete lab canonicalizer. It handles only the explicit corpus shown in the UI so every transformation remains inspectable.

The Node.js service generates tab-separated summary records:

variant=5	service.name=café‑api	requests=400	errors=128

It runs:

sed -f model/canonicalize.sed

The output becomes:

variant=5	service.name=café-api	requests=400	errors=128

The service groups that output and reports the exact-match, raw, canonical, and dual-value query views.

Architecture

browser controls
      │
      ▼
Node.js service ── raw records ──▶ sed canonicalizer
      │                                  │
      ├── identity microscope ◀──────────┤
      └── OTLP-shaped evidence ◀─────────┘

No database, framework, credentials, CDN, or third-party JavaScript dependency is required.

Safety

This is a deterministic educational simulation. It sends no external traffic and exports no telemetry.

License

MIT