Skip to content

feat(validation): improve user-facing schema validation errors#940

Open
saquibsaifee wants to merge 2 commits into
CycloneDX:mainfrom
saquibsaifee:implement-fix-for-validation-error-messages
Open

feat(validation): improve user-facing schema validation errors#940
saquibsaifee wants to merge 2 commits into
CycloneDX:mainfrom
saquibsaifee:implement-fix-for-validation-error-messages

Conversation

@saquibsaifee

@saquibsaifee saquibsaifee commented Feb 28, 2026

Copy link
Copy Markdown
Contributor

Description

Implements the structured validation error improvement planned in #836 (which remains a WIP draft by the maintainer). PR #840 (already merged) laid the groundwork by introducing JsonValidationError and XmlValidationError subclasses — this PR completes that work.

Changes:

  • ValidationError gains three structured fields: message (human-readable str), path (tuple of path segments to the offending value), and data (raw backend error object). __str__ and __repr__ now use message.
  • JsonValidationError._make_from_jsve recurses into nested jsonschema context errors to surface the most specific leaf failure for oneOf/anyOf checks, instead of emitting the generic parent message.
  • XmlValidationError._make_from_xle extracts e.message and e.path from lxml's _LogEntry, normalising XML errors to the same message/path shape.

Closes #827


Note: This PR was originally based on #836 to build on top of its stub. Since #836 is a WIP draft and #840 (which merged the stubs into main) has already landed, this PR is now retargeted directly to main.

AI Tool Disclosure

  • My contribution includes AI-generated content, as disclosed below:
    • AI Tools: GTP Codex
    • LLMs and versions: GPT-5.3-Codex
    • Prompts: [Summarize the key prompts or instructions given to the AI tools]

Affirmation

@read-the-docs-community

read-the-docs-community Bot commented Feb 28, 2026

Copy link
Copy Markdown

@saquibsaifee saquibsaifee changed the base branch from main to feat/validator_error_useful February 28, 2026 20:55
@saquibsaifee saquibsaifee changed the base branch from feat/validator_error_useful to main February 28, 2026 20:56
@codacy-production

codacy-production Bot commented May 17, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 5 complexity · 0 duplication

Metric Results
Complexity 5
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@saquibsaifee saquibsaifee changed the base branch from main to feat/validator_error_useful June 12, 2026 20:58
@saquibsaifee saquibsaifee marked this pull request as ready for review June 12, 2026 21:00
@saquibsaifee saquibsaifee requested a review from a team as a code owner June 12, 2026 21:00
@saquibsaifee saquibsaifee force-pushed the implement-fix-for-validation-error-messages branch from eb0350d to 40d2d1b Compare July 14, 2026 20:18
@saquibsaifee saquibsaifee changed the base branch from feat/validator_error_useful to main July 14, 2026 20:19
Implements the planned improvement from PR CycloneDX#836 by the maintainer.

- ValidationError gains structured fields: message (str), path (tuple),
  and data (raw backend object) — replacing the bare data-only shape
- JsonValidationError._make_from_jsve recurses into nested jsonschema
  context errors to surface the most specific leaf failure for oneOf/anyOf
- XmlValidationError._make_from_xle maps lxml _LogEntry.message and path
- __repr__ and __str__ now emit the human-readable message field

Closes CycloneDX#827

Signed-off-by: Saquib Saifee <saquibsaifee2@gmail.com>
@saquibsaifee saquibsaifee force-pushed the implement-fix-for-validation-error-messages branch from 40d2d1b to c66c077 Compare July 14, 2026 20:21
PRs already in main that this builds on:
- CycloneDX#834 (merged): added all_errors support and JsonValidationError /
  XmlValidationError subclass stubs
- CycloneDX#840 (merged): refined the subclass stubs further
- CycloneDX#836 (WIP by maintainer): placeholder for this exact work

Changes
-------
ValidationError (__init__.py)
- Add message (str): human-readable, bounded error message
- Add path (tuple[str|int, ...]): location of the offending value
- Keep data (Any): raw backend object, unchanged for backward compat
- __str__ now returns message; __repr__ is structured

JsonValidationError (json.py)
- __get_most_relevant_jsve: recurse into nested jsonschema context
  errors to surface the deepest leaf failure for oneOf/anyOf checks,
  instead of emitting the generic parent message
- _shorten_message: (1) replace a bloated repr(instance) with a
  head...tail summary for uniqueItems/large-document failures, then
  (2) hard-cap the whole message at 256 chars + ellipsis

XmlValidationError (xml.py)
- _shorten_xml_message: hard-cap lxml _LogEntry.message at 256 chars,
  preventing the full SPDX enumeration set from appearing verbatim
- path populated from _LogEntry.path (XPath location string)

Before vs after (invalid-license-id test files from issue CycloneDX#827)
  JSON str(error): 111,672 chars  ->  257 chars
  XML  str(error):  13,430 chars  ->  257 chars
  error.message:   AttributeError ->  bounded str
  error.path:      AttributeError ->  structured tuple

Tests (test_validation_json.py, test_validation_xml.py)
- Assert error is the correct subtype (JsonValidationError /
  XmlValidationError)
- Assert .message is a str and .path is a tuple
- Assert len(message) <= 257 across every invalid test file for every
  schema version

Closes CycloneDX#827

Signed-off-by: Saquib Saifee <saquibsaifee2@gmail.com>
@saquibsaifee

Copy link
Copy Markdown
Contributor Author

@jkowalleck is this along the lines of what you had in mind for #827?

PR #940 builds directly on the work from #834 and #840 already in main. Here's what it adds:

  • ValidationError now has .message (bounded str) and .path (tuple) alongside the existing .data
  • JSON: recurses into oneOf/anyOf context errors to surface the deepest failure, then caps the message at 256 chars — so the full SPDX licence list no longer leaks through
  • XML: same 256 char cap on lxml's _LogEntry.message

Before/after on the exact test files from the issue (attaching screenshots):

  • JSON str(error): 111,672 → 257 chars
  • XML str(error): 13,430 → 257 chars
  • .message and .path: were AttributeError, now work as expected

Raw data is still there for anyone depending on it. Happy to adjust the message cap value, the path shape, or anything else — just let me know!

Test code and output:

"""AFTER — our branch"""
from cyclonedx.schema import SchemaVersion
from cyclonedx.validation.json import JsonStrictValidator
from cyclonedx.validation.xml import XmlValidator

SEP = '=' * 72

# --------------- JSON ---------------
json_file = 'tests/_data/schemaTestData/1.2/invalid-license-id-1.2.json'
with open(json_file) as fh:
    json_data = fh.read()

json_validator = JsonStrictValidator(SchemaVersion.V1_2)
error = json_validator.validate_str(json_data)

print(SEP)
print('JSON')
print(SEP)
print(f'type(error)       : {type(error).__qualname__}')
print(f'type(error.data)  : {type(error.data).__qualname__}')
print(f'len(str(error))   : {len(str(error)):,} chars')
print(f'error.message     : {error.message!r}')
print(f'error.path        : {error.path!r}')

# --------------- XML ---------------
xml_file = 'tests/_data/schemaTestData/1.1/invalid-license-id-1.1.xml'
with open(xml_file) as fh:
    xml_data = fh.read()

xml_validator = XmlValidator(SchemaVersion.V1_1)
error = xml_validator.validate_str(xml_data)

print()
print(SEP)
print('XML')
print(SEP)
print(f'type(error)       : {type(error).__qualname__}')
print(f'type(error.data)  : {type(error.data).__qualname__}')
print(f'len(str(error))   : {len(str(error)):,} chars')
print(f'error.message     : {error.message!r}')
print(f'error.path        : {error.path!r}')
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validation errors are hard to present safely to the user (missing abstraction)

1 participant