Skip to content

docs: metadata schemas need a top-level "document" wrapper - #4

Merged
cdbartholomew merged 1 commit into
mainfrom
fix/readme-metadata-schema-document-wrapper
Sep 23, 2026
Merged

cdbartholomew merged 1 commit into
mainfrom
fix/readme-metadata-schema-document-wrapper

Conversation

@cdbartholomew

Copy link
Copy Markdown
Contributor

The problem

Both README.md and nodejs-api/README.md show a bare field-to-type map as the schema value:

'schema': {
    'invoice_number': 'string',
    'date': 'string',
    ...
}

That shape can never work. The backend reads a top-level document key off the schema before looking at its contents (JsonMetadataSchema.parseAndValidate) and throws if it is absent:

java.util.concurrent.CompletionException:
java.lang.IllegalArgumentException: Document schema is missing

Anyone copying the published example hits this immediately — which is exactly how it was reported by a customer piloting Iris against a one-page claim form. They tried both a formal JSON Schema and the simplified form from our README, got the identical error from both, and reasonably concluded the SDK was broken.

The SDK is fine. ExtractionOptions.to_extraction_request() serializes correctly (by_alias=True, exclude_none=True); the documentation was wrong.

Verified against production

api.vectorize.io, same request shape the SDK builds, inferSchema=false:

schema value Result
bare formal JSON Schema ❌ Document schema is missing
bare field-to-type map (the old example) ❌ Document schema is missing
{"document": <formal JSON Schema>} ✅ metadata extracted
{"document": <field-to-type map>} ✅ metadata extracted

So the wrapper is the only thing that was missing — both representations work once wrapped.

The exact Python block this PR adds was executed against production before committing, and returned:

{"date":"2026-02-04","invoice_number":"INV-77120","total_amount":1899.5,"vendor_name":"ACME Supplies Ltd."}

Note total_amount comes back as the number 1899.5, not a string — declared types are honored end to end, which is why the examples now use a real JSON Schema rather than the looser form.

Changes

  • Wrap both README examples in document, using real JSON Schema with type and description per property
  • State the failure mode explicitly so the error string is searchable
  • Document infer_metadata_schema / inferMetadataSchema — needs to be False when supplying your own schemas
  • Document the optional sibling sections key for per-chunk metadata
  • Drop the "OpenAPI spec format recommended" wording, which described neither the old example nor the actual requirement

Not included

Only the two published examples are fixed here. Worth considering separately: the API currently accepts a schema with no document key and fails asynchronously mid-extraction, so the customer only learns about it after upload and processing. Validating at POST /extraction and returning a 400 with the reason would turn this from a silent job failure into an immediate, self-explanatory error.

The metadata examples in README.md and nodejs-api/README.md show a bare
field-to-type map as the `schema` value. That shape can never work: the
backend reads a top-level `document` key off the schema before looking at
its contents, and throws

    java.util.concurrent.CompletionException:
    java.lang.IllegalArgumentException: Document schema is missing

if it is absent. Anyone copying the published example hits this
immediately, which is how it was reported.

Verified against the production endpoint (api.vectorize.io), same request
shape the SDK builds, inferSchema=false:

  bare formal JSON Schema          -> FAILS, "Document schema is missing"
  bare field-to-type map (old doc) -> FAILS, "Document schema is missing"
  {"document": formal JSON Schema} -> SUCCEEDS, metadata extracted
  {"document": field-to-type map}  -> SUCCEEDS, metadata extracted

So the wrapper is the only thing that was missing — both schema
representations work once wrapped. The examples now use a real JSON
Schema anyway, because declared types are honored end to end: the
invoice example in this commit returns total_amount as the number
1899.5 rather than a string.

The exact Python block added here was executed against production before
committing, and returned:

    {"date":"2026-02-04","invoice_number":"INV-77120",
     "total_amount":1899.5,"vendor_name":"ACME Supplies Ltd."}

Also drops the "OpenAPI spec format" wording, which described neither the
old example nor the actual requirement, and documents infer_metadata_schema
/ inferMetadataSchema plus the optional `sections` key.
@cdbartholomew
cdbartholomew merged commit 3cd6281 into main Sep 23, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants