Skip to content

MODEL_ARTIFACTS.md

Latest commit

 

History

History
48 lines (37 loc) · 2.01 KB

MODEL_ARTIFACTS.md

File metadata and controls

48 lines (37 loc) · 2.01 KB
Raw

Model artifact storage policy

SCIoT currently contains several generated model artifacts in the source tree. This policy defines the target direction for future cleanup without disrupting existing tests and demos.

Artifact classes

Class Examples Git policy
Source/model tooling model split scripts, loaders, manifests Keep in Git.
Small deterministic fixtures tiny test samples, minimal metadata used by tests Keep in Git under tests/fixtures/ or tests/test_samples/.
Runtime metadata layer_sizes_*.json, valid_offloading_points_*.json Keep only when deterministic and small; otherwise regenerate or fetch.
Large model binaries .h5, .tflite, exported model folders Move to release assets, Git LFS, or external artifact storage.
Research exports historical experiment folders and generated plots Archive outside the runtime tree or externalize.

Preferred storage approach

The preferred long-term model distribution path is:

  1. Keep source code, manifests, and tiny deterministic fixtures in Git.
  2. Publish large model binaries as versioned release assets or external artifacts.
  3. Add a machine-readable model manifest with:
    • model ID and version;
    • compatible client/server protocol version;
    • expected input size;
    • list of required files;
    • SHA-256 checksums;
    • file sizes;
    • download URLs or local cache keys.
  4. Add a fetch/verify command that downloads missing artifacts into a local cache and verifies checksums before runtime use.

Local cache and offline use

The eventual fetch command should support:

  • a default cache outside src/, for example data/models/ or a user cache;
  • an override environment variable for CI and offline labs;
  • a --verify-only mode for CI;
  • a documented offline workflow using pre-populated cache files.

Migration rule

Do not add new large model binaries to Git. If a test needs a model, prefer a tiny deterministic fixture or mark the test as requiring external model artifacts.