Skip to content

Strip image parts for models without image input, like the CommandCode CLI does #5

Description

@yelixir-dev

The CommandCode CLI never sends image parts to a model that has no image input modality. In dist/cli.mjs the model client gates on the registry before building the wire body:

const r = !e.registry.supportsVision({ model: t.model }) && hasImage2({ messages: t.messages })
  ? stripImages({ messages: t.messages })
  : t.messages;

stripImages removes every image part and substitutes placeholder text, keeping only the most recent user message's image slots as numbered placeholders.

The bridge cannot do this today: providerModelsFromCatalog in src/model-catalog.ts only keeps id, name, context_length, and owned_by, so there is no modality information to gate on. Once image parts are actually forwarded upstream (PR #3), a request carrying an image to a text-only model produces a body shape the CLI would never emit, besides likely being rejected upstream.

Work needed:

  1. Carry input modality (or an explicit vision flag) through the model catalog, either from the provider catalog payload if it exposes one, or as a maintained list.
  2. Strip image parts for non-vision models before building the generate body, mirroring the CLI's placeholder substitution.
  3. Cover it with a converter test asserting no image part survives for a text-only model.

Blocked on PR #3 landing first, since image parts do not reach the upstream body before then.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions