Skip to content

fix(serve): repack source_code for image_uri/ModelTrainer builds - #6112

Open
lucasjia-aws wants to merge 4 commits into
aws:masterfrom
lucasjia-aws:fix/6105
Open

fix(serve): repack source_code for image_uri/ModelTrainer builds#6112
lucasjia-aws wants to merge 4 commits into
aws:masterfrom
lucasjia-aws:fix/6105

Conversation

@lucasjia-aws

Copy link
Copy Markdown
Collaborator

ModelBuilder classified any image_uri build with a model artifact and source_code (no model/inference_spec) as passthrough and dropped the inference code, so build() produced a model.tar.gz without code/ and register() yielded a package that could not serve. This regressed the classic v2 Model(model_data, entry_point, source_dir) repack behavior.

In _build_for_passthrough(), distinguish a pure image-only deployment from an "image + model artifact + source_code" build. For the latter, keep the source code and bridge model_path to s3_model_data_url so the existing _upload_code(repack=True) path repacks the code into the model artifact, producing a self-contained tarball. Pure-image (Nova/BYOC) passthrough behavior is unchanged. ModelTrainer builds are covered too since they are normalized to model_path before validation.

Also pass script_dependencies (a list derived from SourceCode) to repack_model instead of the deprecated self.dependencies auto-detect dict, which repack_model would otherwise iterate as filesystem paths.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

ModelBuilder classified any image_uri build with a model artifact and
source_code (no model/inference_spec) as passthrough and dropped the
inference code, so build() produced a model.tar.gz without code/ and
register() yielded a package that could not serve. This regressed the
classic v2 Model(model_data, entry_point, source_dir) repack behavior.

In _build_for_passthrough(), distinguish a pure image-only deployment
from an "image + model artifact + source_code" build. For the latter,
keep the source code and bridge model_path to s3_model_data_url so the
existing _upload_code(repack=True) path repacks the code into the model
artifact, producing a self-contained tarball. Pure-image (Nova/BYOC)
passthrough behavior is unchanged. ModelTrainer builds are covered too
since they are normalized to model_path before validation.

Also pass script_dependencies (a list derived from SourceCode) to
repack_model instead of the deprecated self.dependencies auto-detect
dict, which repack_model would otherwise iterate as filesystem paths.
@lucasjia-aws

Copy link
Copy Markdown
Collaborator Author

related to issue #6105

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

Claude finished @lucasjia-aws's task in 6m 29s —— View job


Code Review — PR #6112: repack source_code for image_uri/ModelTrainer builds

  • Read PR diff (/tmp/pr.diff)
  • Trace passthrough → _create_model_prepare_container_def(_base)_upload_code(repack=True) paths
  • Analyze correctness, best practices, tests
  • Post review

Overall this is a well-reasoned, well-commented fix. The logic is sound: bridging model_paths3_model_data_url, clearing s3_upload_path, and letting is_repack()/_upload_code(repack=True) produce a self-contained model.tar.gz correctly restores the classic v2 repack behavior, and the model-data resolution (repacked_model_data or s3_upload_path or s3_model_data_url at model_builder.py:2437 / :2588) picks up repacked_model_data as intended. The script_dependencies fix is also correct — _upload_code's non-repack branch already relies on self.script_dependencies (model_builder.py:2280), and self.dependencies is the deprecated auto-detect dict ({"auto": ...}) that repack_model would wrongly iterate as filesystem paths.

A few points worth addressing:

1. No tests added for the new behavior (most important)

All 46 additions are in model_builder.py; there are no test changes. This PR fixes a regression and adds a new decision branch in _build_for_passthrough() plus changes repack dependency handling, yet nothing guards against re-regression. There's already a good home for these in sagemaker-serve/tests/unit/test_model_builder_methods.py (test_build_for_passthrough_*). Please add coverage for:

  • image_uri + s3 model_path + source_code → asserts _upload_code(repack=True) runs and repacked_model_data drives ModelDataUrl (code ends up under code/).
  • model_paths3_model_data_url bridging and s3_upload_path reset to None.
  • pure image-only passthrough (Nova/BYOC) → unchanged (regression guard for the existing behavior at model_builder.py:1920-1929).
  • repack_dependenciesrepack_model is called with the script_dependencies list, not self.dependencies.

2. has_source_code requires both entry_point and source_dir (model_builder.py:1894-1896)

_initialize_script_mode_variables can set entry_point while leaving source_dir=None (model_builder.py:1659-1662) when SourceCode.source_dir is unset but entry_script is a standalone file. In that case has_source_code is False, so this new branch is skipped and the fall-through path at :1926-1927 silently nulls entry_point/source_dir — dropping the inference code, i.e. the exact regression this PR fixes, just for the single-file layout. This does mirror is_repack() (:2246), so it may be an acceptable known limitation, but if single-file entry points are a supported input it's worth handling or at least logging a warning rather than silently dropping code.

3. Minor: inconsistent str() guarding

model_builder.py:1887 uses str(self.model_path).startswith("s3://") while the pre-existing fall-through at :1920 uses self.model_path.startswith(...). Both are guarded by a truthiness check so neither errors, but the defensive str() in the new code suggests model_path might sometimes be non-str — if that's a real concern it applies to :1920 too; if not, the str() is unnecessary. Not blocking.

Nothing here is a correctness blocker for the happy path — the primary ask is test coverage (#1). Points #2 and #3 are for your judgment.

Note: the inline-comment tool wasn't available in this session, so I've consolidated the review here with file:line references instead of inline anchors.

aviruthen
aviruthen previously approved these changes Jul 27, 2026
ModelBuilder classified any image_uri build without an in-memory model
or inference_spec as passthrough, ignoring source_code. _build_for_
passthrough() then cleared source_dir/entry_point, so custom inference
code was silently dropped: no repack occurred, the container pointed at
the raw model artifact, register()/deploy() produced a model with no
serving code. Regressed from v2 Model(model_data, entry_point,
source_dir) + _RepackModelStep; introduced by aws#5969 (commit ebeb54c)
which added the passthrough nulling.

Fix the passthrough classification at its source: do not treat a build
as passthrough when source_code is set (both 1P and non-1P image
branches). Such builds route to the normal model-server path and repack
the code into the artifact.

Also:
- Relax the _build_for_model_server guard to accept s3_model_data_url
  (a prebuilt training artifact) alongside model/MLflow/inference_spec.
- In _upload_code repack, pass script_dependencies (a list) instead of
  the deprecated self.dependencies auto-detect dict, which repack_model
  would otherwise iterate as filesystem paths.
- Update _build_for_model_server unit tests to set s3_model_data_url
  so the missing-parameter cases still validate.
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Claude finished @lucasjia-aws's task in 8m 52s —— View job


Code Review — PR #6112

  • Read PR diff (/tmp/pr.diff)
  • Trace passthrough classification, _build_for_model_server validation, and the repack=True path
  • Analyze correctness, best practices, tests
  • Post review

The current diff takes a different approach than the PR description. The body describes bridging model_path → s3_model_data_url inside _build_for_passthrough(), but this diff doesn't touch that method. Instead it (1) excludes source_code from passthrough classification (model_builder.py:1854, :1865) so a source_code build falls through to the real builders, and (2) relaxes the _build_for_model_server validation to accept s3_model_data_url (model_builder_servers.py:105-114), plus the script_dependencies fix. The mechanics of what's here are sound, but please reconcile the description with the code.

1. Validation + repack key on s3_model_data_url, but ModelTrainer/model_path sets model_path — the scenario the body claims to cover may still fail (most important)

The PR body states "ModelTrainer builds are covered too since they are normalized to model_path before validation." But the new validation checks s3_model_data_url, not model_path:

if (not self.model and not mlflow_path and not self.inference_spec
        and not self.s3_model_data_url):
    raise ValueError(...)

For a ModelTrainer (or TrainingJob), build() sets self.model_path and nulls self.model (model_builder.py:3021-3041) — it does not populate self.s3_model_data_url. I searched the module and found no path that copies model_path → s3_model_data_url (it's only ever read from the user param / serve_settings, or reset to None). So:

  • ModelTrainer + source_code + model_server → reaches _build_for_model_server, all four conditions falsy → still raises ValueError.
  • Even if it passed, the repack itself gates on s3_model_data_url (_upload_code, model_builder.py:2249), so with only model_path set, repack=True would skip repacking and no code/ would land in the tarball.

Net: the fix only actually works when the user passes s3_model_data_url= explicitly. The model_path/ModelTrainer regression named in the body doesn't appear to be covered by this diff. Please confirm with an end-to-end test, or bridge model_path → s3_model_data_url (as the description says) so both inputs work.

2. Behavior change for non-1P image_uri + source_code with no model_server

With source_code now excluded from passthrough, a non-1P image_uri + source_code build with no model_server set falls through to _passthrough = False and then hits the existing guard at model_builder.py:1871-1875 ("Model_server must be set when non-first-party image_uri is set"). Previously this was treated as passthrough (image-only, code silently dropped). Erroring is arguably the correct new behavior, but it is a behavior change from "silently deploys image-only" to "raises" — worth a changelog note and a test so it's intentional, not incidental.

3. Test coverage is only negative-path (still the gap flagged in the prior review)

All six test additions just add mock_builder.s3_model_data_url = None to keep the existing "missing required params" assertions passing. There is no positive coverage for the new behavior:

  • s3_model_data_url set (no model/mlflow/inference_spec) → validation passes and routes to the correct server builder.
  • source_code present → passthrough is not selected (_passthrough = False) — regression guard for model_builder.py:1854/:1865.
  • repack=True calls repack_model with the script_dependencies list rather than self.dependencies (the deprecated auto-detect dict) — guard for model_builder.py:2289/2305.

Please add at least these three; #1 above especially needs an end-to-end assertion that code ends up under code/.

Minor

  • script_dependencies fix (model_builder.py:2287-2289, :2305) is correct — repack_model treats dependencies as a list of filesystem paths (sagemaker-core/.../common_utils.py:567), so passing the {"auto": ...} dict would have been wrong. Good catch.
  • The two passthrough branches (:1848, :1859) are now identical except for the is_1p_image_uri sign; not new here, but they could be collapsed.

Nothing here is a happy-path blocker for the explicit s3_model_data_url case, which is implemented correctly. The main asks are: reconcile the description, verify/close the model_path/ModelTrainer gap (#1), and add positive-path tests (#3).

Note: the inline-comment tool wasn't available in this session, so I've consolidated the review here with file:line references.

nargokul
nargokul previously approved these changes Jul 28, 2026
Restore v2-parity repack behavior for image_uri + model artifact +
source_code builds by repacking inside _build_for_passthrough(), instead
of gating these builds out of passthrough into the model-server path.

Routing such builds to _build_for_model_server exposed pre-existing gaps
in the server builders (e.g. _build_for_djl reads hf_model_config, which
is unset when model is None) and diverged from the path that worked
before serve 1.15.0. Repacking within passthrough matches the 1.12.0
behavior and keeps the fix self-contained.

- _build_validations: drop the source_code passthrough exclusion so these
  builds stay on the passthrough path
- _build_for_passthrough: when source_code and a model artifact are both
  present, bridge model_path to s3_model_data_url and repack into the
  artifact; otherwise keep the pure image-only passthrough behavior
- _build_for_model_server: revert the s3_model_data_url guard relaxation
- tests: revert the now-unneeded servers guard test changes

Verified end-to-end (build -> deploy -> invoke) for both the ticket
scenario (image_uri + model_server=DJL_SERVING + source_code +
s3_model_data_url) and the aws#6105 scenario (no model_server).
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

Code Review — PR #6112

  • Read PR diff (/tmp/pr.diff)
  • Trace passthrough → _create_model_upload_code(repack=True) paths
  • Analyze correctness, tests
  • Post review

View job run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants