Skip to content

Transformer.transform() raises pydantic ValidationError ("tags: extra_forbidden") after submitting the job when tags are set #6090

Description

@Alex-Mesmer

PySDK Version

  • PySDK V2 (2.x)
  • PySDK V3 (3.x)

Describe the bug
Transformer.transform() raises a pydantic.ValidationError for TransformJob whenever the Transformer was created with non-empty tags. The error is raised after the transform job has already been submitted to SageMaker: transform() builds and submits the CreateTransformJob request successfully, then reconstructs a local sagemaker.core.resources.TransformJob from the request dict but that resource model declares no tags field and sets extra="forbid", so the round-trip fails when the request contains tags.

Another major issue is that there are no tags attached to the created transformation job.

Consequences: the transform job is actually created on SageMaker, but the SDK call raises and the caller never gets the job handle. The failure is independent of the tag value/shape, any non-empty tags triggers it, because the tags field itself is rejected on the resource model.

A related issue, the shape of the tags between ModelTrainer and Transformer are not the same.

To reproduce
This is exactly the construction transform() performs:

from sagemaker.core.resources import TransformJob
TransformJob(transform_job_name="x", tags=[{"key": "team", "value": "ml"}])

Real-world trigger:

from sagemaker.core.transformer import Transformer
t = Transformer(model_name="my-model", instance_count=1, instance_type="ml.m5.large",
                output_path="s3://bucket/out/", tags=[{"Key": "team", "Value": "ml"}])
t.transform(data="s3://bucket/in/", content_type="text/csv")   # job submits, then raises

Actual behavior:

pydantic_core._pydantic_core.ValidationError: 1 validation error for TransformJob
tags
  Extra inputs are not permitted [type=extra_forbidden, input_value=[{'key': 'team', 'value': 'ml'}], input_type=list]
    For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden

Expected behavior
Transformer.transform() succeeds with tags set and returns the job handle; tags are applied to the transform job.

Root Cause
In sagemaker-core/src/sagemaker/core/transformer.py, Transformer.transform():

  • L717: transform_request["Tags"] = tags
  • L405: create_transform_job(**request) - job is submitted
  • L411–L412:
transformed = transform_util(serialized_request, "CreateTransformJobRequest")
self.latest_transform_job = TransformJob(**transformed)   # sagemaker.core.resources.TransformJob

transformed includes a tags key, but sagemaker.core.resources.TransformJob has no tags/Tags field and model_config["extra"] == "forbid" → extra_forbidden.

Suggested fix
Any of: (1) add a tags field to sagemaker.core.resources.TransformJob; (2) drop tags from transformed before TransformJob(**transformed) at L412; or (3) build the post-submit local object via TransformJob.get(...) instead of TransformJob(**transformed).

Screenshots or logs
If applicable, add screenshots or logs to help explain your problem.

System information
A description of your system. Please provide:

  • SageMaker Python SDK version: sagemaker==3.15.1, sagemaker-core==2.16.0
  • Framework name (eg. PyTorch) or algorithm (eg. KMeans):
  • Framework version:
  • Python version:
  • CPU or GPU:
  • Custom Docker image (Y/N):

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions