fix: keep manifestXmlJsonml output well-formed XML#1077
Open
He-Pin wants to merge 1 commit into
Open
Conversation
b41538d to
4b02ba9
Compare
Motivation: Jsonnet documents std.manifestXmlJsonml as converting a JsonML-encoded value to a string containing XML. XML names and characters must satisfy XML 1.0, and a JsonML element is array-shaped. The previous implementation accepted a bare top-level string and silently dropped illegal XML control characters, which could either produce non-JsonML output or lose user data. Modification: Replace scalatags rendering with a small XML renderer, validate XML element and attribute names, reject top-level non-array values, reject text and attribute characters outside the XML 1.0 Char production, escape XML-sensitive text and attribute characters, stringify complex attribute values with compact JSON, and add regression tests for invalid names, invalid XML characters, top-level non-array input, complex attributes, valid XML names, and supplementary Unicode characters. Result: sjsonnet now rejects inputs that cannot produce XML instead of emitting invalid markup or silently dropping data, while preserving valid XML names and Unicode text. The behavior intentionally follows the Jsonnet documentation XML contract even where cpp-jsonnet, go-jsonnet, and jrsonnet currently emit invalid XML for some cases. References: - https://jsonnet.org/ref/stdlib.html#std-manifestXmlJsonml - https://www.w3.org/TR/xml/#NT-Name - https://www.w3.org/TR/xml/#charsets - https://www.w3.org/TR/xml/#sec-starttags - https://www.jsonml.org/syntax/
4b02ba9 to
6fc4356
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
Jsonnet documents
std.manifestXmlJsonml(value)as converting a JsonML-encoded value to a string containing XML. JsonML elements are array-shaped values, and XML 1.0 start-tags, end-tags, empty-element tags, attributes, and character data impose stricter rules than the previous renderer enforced.Before this PR, sjsonnet delegated rendering to
scalatags. That rejected some valid XML names such as dotted element names, surfaced invalid names as generic internal errors, rejected array/object attribute values even though other implementations stringify them, accepted a bare top-level string even though it is not a JsonML element, and silently dropped XML-forbidden control characters.Modification:
scalatags-dependentmanifestXmlJsonmlrendering path with a small XML renderer for JsonML.Nameproduction.Charproduction instead of silently dropping them.scalatagsrejected, array/object attribute values, top-level non-array input, invalid XML characters, and supplementary Unicode text.Result:
sjsonnet now follows the documented "string containing the XML" contract even where other implementations currently emit strings that are not well-formed XML. The table below includes the old sjsonnet behavior from
masterbefore this PR.std.manifestXmlJsonml(["", {}])"<></>"invalid XML"<></>"invalid XML"<></>"invalid XMLInternal Errorfromscalatagsinvalid tag nameinvalid XML tag name ""std.manifestXmlJsonml(["div", {data:{x:1}}])"<div data=\"{\"x\": 1}\"></div>"invalid XML attribute quotingunsupported attribute type object"<div data=\"{"x":1}\"></div>"std.manifestXmlJsonml(["div", {}, "a&b"])"<div>a&b</div>"invalid XML text escaping"<div>a&b</div>""<div>a&b</div>"std.manifestXmlJsonml(["child.node"])"<child.node></child.node>"Internal Errorfromscalatagsinvalid tag name"<child.node></child.node>"std.manifestXmlJsonml(["tag", {"bad attr": "x"}])"<tag bad attr=\"x\"></tag>"invalid XML attribute nameInternal Errorfromscalatagsinvalid attribute nameinvalid XML attribute name "bad attr"std.manifestXmlJsonml("text")Expected a JSONML value (an array), got string"text"Expected a JSONML value (an array), got stringstd.manifestXmlJsonml(["tag", std.char(1)])U+0001in<tag>text, invalid XML character"<tag></tag>"after silently droppingU+0001invalid XML character U+0001std.manifestXmlJsonml(["emoji", {value: std.char(128512)}, std.char(128512)])"<emoji value=\"😀\">😀</emoji>"This PR is intentionally not an "accept empty tags" compatibility change. Empty XML elements are valid only when they still have a valid XML element name, for example
<tag></tag>or<tag/>.References:
std.manifestXmlJsonmldocumentation: https://jsonnet.org/ref/stdlib.html#std-manifestXmlJsonmlNameproduction: https://www.w3.org/TR/xml/#NT-Name