Add a unit argument to the length validator (utf16 / bytes) - #398
Open
ChrisJr404 wants to merge 1 commit into
Open
Add a unit argument to the length validator (utf16 / bytes)#398ChrisJr404 wants to merge 1 commit into
ChrisJr404 wants to merge 1 commit into
Conversation
The length validator has always counted strings in Unicode scalar values (chars). That does not match how an HTML form's maxlength or JavaScript's String.length count characters, which use UTF-16 code units, so backend validation can disagree with frontend validation for non-ASCII input. Add an optional `unit` argument (`chars` (default), `utf16`, `bytes`) that selects how string-like values are measured. Collections are still measured by their number of elements regardless of the unit. This exposes `LengthUnit` and `ValidateLength::validate_length_with_unit`; `validate_length` now delegates to it with the default `chars` unit, so existing behaviour and custom impls are unchanged. Closes Keats#250
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #250.
Motivation
The
lengthvalidator counts strings in Unicode scalar values (chars, viastr::chars().count()). That doesn't match how an HTML form field'smaxlengthor JavaScript'sString.lengthcount characters — both use UTF-16 code units. So for non-ASCII input the backend and the frontend can disagree about whether a value is valid.Concretely,
"🖐🏽"is 2chars, but 4 UTF-16 code units and 8 UTF-8 bytes. A form that lets the user type up to 4 "characters" of that emoji can produce a value the currentlengthvalidator rejects (or accepts) inconsistently with the browser.In #250 you suggested handling this with a
mode/unit parameter on thelengthvalidator rather than a separatelength_utf16validator, so thatunicode/utf-16/bytesare all covered by one option. This implements that.What this does
Adds an optional
unitargument tolength:unit = "chars"(default) — Unicode scalar values, i.e. today's behaviourunit = "utf16"— UTF-16 code units (matchesmaxlength/ JSString.length)unit = "bytes"— length of the UTF-8 encodingThe unit only changes how string-like values are measured. Collections (
Vec,HashMap, arrays, …) are still measured by their number of elements, and the unit is ignored for them. When a unit is set it's also added to the error as aunitparam, alongsidemin/max/equal.Implementation notes
ValidateLengthgainsvalidate_length_with_unitandlength_with_unit, both with default implementations.length_with_unitdefaults tolength(), so every existing impl (including user-provided ones that only implementlength()) keeps working unchanged. The string impls and the deref/Cow/Optionforwarders override it so the unit is honoured through wrappers like&StringandOption<String>.validate_lengthnow delegates tovalidate_length_with_unit(LengthUnit::Chars, …), so its behaviour is identical to before.LengthUnitis exported from the crate root.unit = "chars" | "utf16" | "bytes"(a couple of aliases likeunicode/utf8are accepted too) and emits the unit-aware call only when the argument is present; otherwise it emits the exact samevalidate_length(...)call as before, so output for existing code is unchanged.Tests
Added unit tests for the trait (
utf16/bytes/chars, ASCII equivalence, forwarding through&String/Option/Cow, and the collection no-op) and derive integration tests invalidator_derive_tests/tests/length.rs(utf16 max, bytes min/max, explicit-chars-equals-default, the exposedunitparam, andOption<String>).cargo test -p validatorandcargo test -p validator_derive_tests --test lengthpass;cargo fmt --checkandcargo clippyare clean. Also updated the READMElengthsection and the changelog.