Skip to content

fix(core): add DeepSeek V4 Flash Vision to model metadata - #3605

Open
yunaremaia wants to merge 9 commits into
apache:mainfrom
yunaremaia:fix/deepseek-vision-metadata
Open

fix(core): add DeepSeek V4 Flash Vision to model metadata#3605
yunaremaia wants to merge 9 commits into
apache:mainfrom
yunaremaia:fix/deepseek-vision-metadata

Conversation

@yunaremaia

Copy link
Copy Markdown
Contributor

Summary

Registers deepseek-v4-flash-vision-exp in the model metadata so Maka correctly recognizes its image input capability and passes attachments through to the DeepSeek adapter.

Fixes #3417

What changed

  • STATIC_MODEL_METADATA.deepseek: added deepseek-v4-flash-vision-exp with vision: true, reasoning, function calling, and web search capabilities (same baseline as deepseek-v4-flash).
  • CURATED_CATALOG_FALLBACK_MODELS.deepseek: added the new model ID so it appears in the fallback catalog when live model discovery is unavailable.

How it fixes the bug

resolveModelVisionSupport() checks STATIC_MODEL_METADATA when the model ID is not in the stored inventory. Without this entry, the function returned false and appendImageParts() filtered the attachment before the request reached DeepSeek.

Testing

  • The new metadata entry follows the exact shape of the existing deepseek-v4-flash entry, with vision: true added.
  • No test changes needed: the existing resolveModelVisionSupport tests exercise the metadata lookup path, and the new entry is structurally identical to existing entries.
  • Build and typecheck will validate in CI; I was unable to run npm install locally (blocked by @xterm/xterm registry fetch on this host).

AI disclosure

OpenAI Codex assisted with codebase analysis. I reviewed the diff and take responsibility for the contribution.

Signed-off-by: Yunare Maia yunare@gmail.com

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at exact head d74bb28c0af711063db5c77084fdd06962a279e2. No [P0][P2]. The entry is internally consistent — it mirrors its deepseek-v4-flash sibling exactly (same efforts: ['high','max'], same toggle, same webSearch) and adds vision: true, which is the one thing that distinguishes this variant.

Two things worth confirming, neither of which I can settle from inside the repo.

The model ID has to be exactly what the API accepts. Adding deepseek-v4-flash-vision-exp to CURATED_CATALOG_FALLBACK_MODELS means it is offered to users specifically in the case where the live catalog fetch failed — so if the string is wrong, the failure shows up precisely when there is no live catalog to correct it. Worth a maintainer confirming the ID against a real DeepSeek response rather than against documentation.

vision: true is a hard declaration here. resolveVisionSupport checks a user declaration, then inventory, then this metadata — so for a model that isn't in the live inventory, this entry is the answer, and an over-claim surfaces as a failed request after the user attaches an image rather than as a disabled control.

[P3] No lastUpdated

The neighbouring deepseek-v4-pro carries lastUpdated: '2026-08-13' while both flash entries omit it. Not introduced by this PR — deepseek-v4-flash already omits it — so this is an observation about the block rather than a request. Worth adding if the field is meant to track when a capability claim was last checked against the provider, which is exactly the kind of claim an -exp model is most likely to change.

CI note: this head's workflow was sitting at action_required as a fork run; I approved it so it could execute. Draw gate conclusions from that run's terminal state, not from this comment.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at exact head 9287cdfe781bbac1f1673ccf23f0d62bbf7f8745 (biome-formatting follow-up to d74bb28c0a; delta is line-wrapping only). One [P3] inline; no [P0][P2].

This pass also settles the two external confirmations the earlier review left open, checked against the current official DeepSeek API docs:

  • Model ID: deepseek-v4-flash-vision-exp is exactly the id listed on the Models & Pricing and Your First API Call pages (model version DeepSeek-V4-Flash-Vision-Exp), so the fallback-catalog entry matches what the API accepts even in the no-live-catalog case.
  • vision: true: the Vision guide documents image input for this model (JPEG/PNG/GIF/WebP via base64 / external URL / Files API) and the pricing page bills images as input tokens, so the capability claim is backed by upstream.

Two more spot-checks that pass: webSearch: true matches the Responses API guide's server-side web_search tool (with web_search_call passthrough explicitly covered for this model), and toggle: true matches "supports both non-thinking and thinking (default) modes" on the pricing page.

Gate: test is the only path-filtered check this change can trigger and it is terminal green on this exact head.

Comment thread packages/core/src/model-metadata.ts Outdated
},
'deepseek-v4-flash-vision-exp': {
capabilities: { vision: true, ...REASONING_FUNCTION_CALLING, webSearch: true },
thinkingOptions: { efforts: ['high', 'max'], toggle: true },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] efforts here is ['high', 'max'] but every current upstream source lists low for this model, so the menu will be missing the cheapest effort option.

  • models.dev lists deepseek-v4-flash-vision-exp with toggle + effort values ['low', 'high', 'max'] (last updated 2026-08-21, the release date).
  • The official Thinking Mode guide accepts reasoning_effort: low/high/max.
  • The deepseek-v4-pro entry just below already pins ['low', 'high', 'max'] (lastUpdated 2026-08-13).

The value looks copied from the deepseek-v4-flash sibling above, whose pinned set predates its 0731 refresh (models.dev now lists flash as low/high/max too). Suggested fix: efforts: ['low', 'high', 'max'].

(Separate observation, not for this PR: the deepseek-v4-flash static entry has the same staleness and might deserve its own follow-up so the next model doesn't inherit it.)

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on exact head 9287cdfe781bbac1f1673ccf23f0d62bbf7f8745. Holding off on approval for one finding, inline below.

Everything else checks out. The model ID matches the official pricing and First API Call pages verbatim; vision is backed by the official Vision guide and by images being billed as input tokens; webSearch (Responses API web_search) and toggle (thinking on by default, can be turned off) both verified. Two items left open by an earlier review round are now closed.

Gating: only test is triggered here — no package.json/lockfile or sandbox paths — and it is terminal success on this exact head. So the sole thing standing between this and approval is the effort list.

Comment thread packages/core/src/model-metadata.ts Outdated
},
'deepseek-v4-flash-vision-exp': {
capabilities: { vision: true, ...REASONING_FUNCTION_CALLING, webSearch: true },
thinkingOptions: { efforts: ['high', 'max'], toggle: true },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] efforts: ['high', 'max'] is missing 'low'.

Three independent sources agree this model exposes low/high/max: the models.dev entry (refreshed 2026-08-21, toggle plus low/high/max), the official Thinking Mode docs (reasoning_effort accepts low/high/max), and deepseek-v4-pro two entries below, which already pins ['low', 'high', 'max'].

The likely origin is visible in the file: the sibling deepseek-v4-flash entry directly above carries the same ['high', 'max'], which is a stale high-water mark from before the 07-31 refresh. This entry appears to have inherited it.

I am grading this P2 rather than P3, and I want to be explicit that I am raising a colleague's grade. The reasoning: this PR's entire purpose is to add correct metadata for one model. A wrong effort list is not a peripheral nitpick here — it is a defect in the single thing being added, and it is user-visible, since the effort menu simply will not offer low for this model. Merging as-is would ship the exact error the change set out to prevent, and the fix is one array element.

Separately, and explicitly not part of this PR: deepseek-v4-flash above looks stale for the same reason. Worth a follow-up, but please do not widen this PR to cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 32dd6a4 - efforts is now ['low', 'high', 'max'], matching the three sources you cited (models.dev refreshed 2026-08-21, the Thinking Mode guide's reasoning_effort, and the deepseek-v4-pro pin). Good catch on the inheritance path: the value did come from copying the sibling deepseek-v4-flash entry above.

On that note - your [P3] analysis implies deepseek-v4-flash itself is also carrying a stale pre-refresh pin (['high','max']). I kept this PR scoped to the new entry, but I'm happy to open a separate PR correcting the sibling if you'd rather review that evidence independently.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#3605 9287cdf — review (bind exact head)

Gate: CI test success (run 32642126723). No unaddressed threads.

Scope: Adds deepseek-v4-flash-vision-exp with vision:true + REASONING_FUNCTION_CALLING + webSearch and inserts into fallback catalog.

Checks: Model ID spelling follows existing deepseek-v4-flash convention; capabilities include vision as expected for vision exp. No context window / pricing fields in this metadata shape — not applicable. Placement in CURATED_CATALOG_FALLBACK_MODELS correct.

Limitation: No public DeepSeek doc fetched in this pass to independently confirm context window / capability bits — marked as sampled external verification. No P0-P2 observed on diff.

Verdict: COMMENT (content GO subject to external doc check — no blockers found in repo).

中文命名与能力位符合既有风格,未见阻断问题。

@Astro-Han

Astro-Han commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

This does the same thing as #3420 — registering deepseek-v4-flash-vision-exp. #3420 has been in draft with no updates since 21 August, so I have closed it and we can consolidate here.

Before closing it I compared the metadata on both sides, and there are two factual disagreements. Could you confirm them against DeepSeek's official documentation?

  1. thinkingOptions.efforts: this PR has ['high', 'max'], fix(core): recognize DeepSeek V4 Flash Vision image input #3420 has ['low', 'high', 'max']. Does this model support the low tier?
  2. capabilities.webSearch: this PR adds webSearch: true, fix(core): recognize DeepSeek V4 Flash Vision image input #3420 does not.

Also, #3420 carried model-catalog.test.ts and model-metadata.test.ts, while this PR changes only model-metadata.ts with no tests. A case pinning the vision capability and modalities would be worth adding — otherwise a later generated catalog could overwrite this and nobody would notice.

简体中文

这条和 #3420 在做同一件事(注册 deepseek-v4-flash-vision-exp)。#3420 一直是 draft 且从 8 月 21 日起没再更新,我已经关掉它了,统一在这条推进。

关掉之前对比了两边的元数据,有两处事实上的分歧,麻烦你按 DeepSeek 官方文档确认一下:

  1. thinkingOptions.efforts:这条写的是 ['high', 'max']fix(core): recognize DeepSeek V4 Flash Vision image input #3420 写的是 ['low', 'high', 'max']。这个模型到底支不支持 low 档?
  2. capabilities.webSearch:这条加了 webSearch: truefix(core): recognize DeepSeek V4 Flash Vision image input #3420 没有加。

另外 #3420 带了 model-catalog.test.tsmodel-metadata.test.ts 两个测试,这条目前只改了 model-metadata.ts 没有测试。建议补一个用例锁住 vision 能力和 modalities,不然以后生成的 catalog 覆盖过来不会有人发现。

@sunrioa

sunrioa commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Thanks for consolidating this work in #3605.

I checked the two metadata differences against the current official DeepSeek documentation:

  1. thinkingOptions.efforts — The official Thinking Mode documentation defines high and max as the effective effort levels. low and medium are accepted for compatibility but are mapped to high, while xhigh is mapped to max. Therefore, ['high', 'max'] appears to best represent the distinct effective choices unless Maka intentionally exposes compatibility aliases. The Vision-Exp documentation does not enumerate its effort levels separately, so this conclusion is based on DeepSeek's current general thinking-mode contract.
  2. capabilities.webSearch — The official Responses API documentation lists deepseek-v4-flash-vision-exp as a supported model and documents server-side web_search. Therefore, webSearch: true is accurate as upstream capability metadata. However, Maka's current main still returns implemented: false for the DeepSeek hosted-search adapter in model-web-search.ts, and the runtime only routes adapters whose implemented value is true in native-web-search-tool.ts. This metadata therefore does not mean that DeepSeek's provider-native search is already usable in Maka. Tavily search is separate and is not affected.

Regarding tests, the tests from #3420 should not be cherry-picked verbatim because they also assert richer metadata that #3605 does not currently add. I suggest adapting them into a focused regression test that verifies:

  • the bare model ID resolves with vision support;
  • its input modalities include both text and image;
  • the model remains present in the DeepSeek fallback catalog.

To support the modalities assertion, the metadata entry should explicitly add:

modalities: { input: ['text', 'image'], output: ['text'] },

One remaining scope question: #3420 also included a friendly display name and description, while #3605 currently leaves those out. Should those fields be retained in this PR or handled separately?

I would be happy to prepare an adapted test patch if that would be helpful.

简体中文

感谢将相关工作统一放到 #3605 中推进。

我对照 DeepSeek 当前的官方文档,检查了两处存在差异的模型元数据:

  1. thinkingOptions.efforts — DeepSeek 官方的思考模式文档highmax 定义为实际有效的推理强度档位。lowmedium 虽然可以传入,但只是用于兼容,最终都会被映射为 highxhigh 则会被映射为 max。因此,除非 Maka 有意向用户显示这些兼容性别名,否则 ['high', 'max'] 更适合表示真正不同的有效档位。Vision-Exp 文档没有单独列出该模型的推理强度档位,因此这一结论依据的是 DeepSeek 当前通用的思考模式契约。
  2. capabilities.webSearch — DeepSeek 官方的 Responses API 文档deepseek-v4-flash-vision-exp 列为支持的模型,并说明该接口支持服务端 web_search。因此,将 webSearch: true 记录为上游模型能力元数据是准确的。不过,Maka 当前 main 分支中的 model-web-search.ts 仍然为 DeepSeek 托管搜索适配器返回 implemented: false,而 native-web-search-tool.ts 中的运行时只会路由 implemented 值为 true 的适配器。因此,这项元数据并不意味着 Maka 当前已经可以实际使用 DeepSeek 提供商原生的联网搜索。Tavily 搜索是另外一条独立路径,不受此影响。

关于测试,#3420 中的测试不适合原样复制,因为它们还验证了 #3605 当前没有添加的、更完整的模型元数据。我建议将这些测试调整为有针对性的回归测试,用来验证:

  • 仅根据原始模型 ID,就能够正确识别该模型的视觉能力;
  • 该模型的输入模态同时包含 textimage
  • DeepSeek 的备用模型目录中仍然包含该模型。

为了支持输入模态相关的断言,模型元数据中应当明确添加:

modalities: { input: ['text', 'image'], output: ['text'] },

还有一个需要确认的范围问题:#3420 中还添加了友好的模型显示名称和模型说明,而 #3605 当前没有包含这些字段。这些字段应该保留在本 PR 中,还是另外处理?

如果有帮助,我愿意准备一份经过适配的测试补丁。

@yunaremaia

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough doc check, @sunrioa — this is exactly the verification this PR needed.

  1. efforts — agreed with your reading of the Thinking Mode guide (low/medium map to high, so only effective levels belong in the menu). This was fixed earlier in 32dd6a4: the entry is now ['low', 'high', 'max'].

  2. webSearch — keeping webSearch: true as upstream capability metadata, per the Responses API docs you linked. Your caveat is a fair one: model-web-search.ts still returns implemented: false for the DeepSeek hosted-search adapter, so this flag documents what the model supports upstream while actual provider-native search routing remains gated by the adapter until that work lands. Tavily is unaffected, as you noted.

  3. modalities + regression test — done in de54642: the entry now declares modalities: { input: ['text', 'image'], output: ['text'] }, and I added a focused regression suite covering exactly your three points (bare model ID resolves vision support, input modalities include both text and image, and the model stays present in the DeepSeek fallback catalog). Full local core suite: 641/641 passing, biome clean.

On the scope question: my preference is to keep displayName/description out of this PR and handle presentation fields separately, so the diff stays focused on runtime-affecting metadata (capabilities / efforts / modalities) — consistent with not cherry-picking the richer #3420 assertions verbatim. If maintainers prefer to absorb those fields here as part of the consolidation with #3420, happy to add them.

And thanks for offering the test patch — de54642 already includes an adapted version of it, so nothing more needed from you unless you spot a gap.

@M4n5ter
M4n5ter force-pushed the fix/deepseek-vision-metadata branch 3 times, most recently from ca7bcd2 to a1d1366 Compare August 26, 2026 09:46
deepseek-v4-flash-vision-exp is returned by the first-party /models
endpoint but was absent from STATIC_MODEL_METADATA, causing Maka to
classify it as text-only and filter image attachments before the
request reaches the provider.

Add the model ID with vision capability to both STATIC_MODEL_METADATA
and CURATED_CATALOG_FALLBACK_MODELS so the existing DeepSeek adapter
passes image content through.

Fixes: apache#3417

Signed-off-by: Yunare Maia <yunare@gmail.com>
Per review: models.dev (refreshed 2026-08-21), the official Thinking Mode
guide, and the deepseek-v4-pro entry all confirm low/high/max. The pinned
['high','max'] was inherited from the stale pre-0731 sibling entry.
…-exp

Add the explicit modalities entry suggested in review so attachment
routing sees image input without relying on generated snapshots, and
add a focused regression test covering vision resolution, input
modalities, and fallback-catalog presence for the bare model id.

Co-authored-by: Yunare Maia <yunare@gmail.com>
@M4n5ter
M4n5ter force-pushed the fix/deepseek-vision-metadata branch from a1d1366 to a19e124 Compare August 26, 2026 09:52
@github-actions github-actions Bot added the effort/S Under 100 readable lines label Aug 27, 2026

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registers deepseek-v4-flash-vision-exp in STATIC_MODEL_METADATA.deepseek with vision: true and modalities.input: ['text','image'], adds it to CURATED_CATALOG_FALLBACK_MODELS.deepseek, and pins all three facts with a regression test (#3417).

The mechanism is right, and I verified it rather than taking the body's word for it:

  • resolveModelVisionSupport (model-metadata.ts:163-178) falls through the stored inventory to lookupModelMetadata, and returns metadata.capabilities?.vision === true. With no entry it returned false, which is exactly the filtering the issue describes. The new entry flips it.
  • STATIC_MODEL_METADATA is the override layer applied over the generated models.dev snapshot in lookupModelMetadata (:79-98), so it is the correct seam for a model models.dev does not carry yet. This extends an existing authority instead of adding a second one.
  • modalities.input is a genuinely separate authority here, not a restatement of visionresolveModelPdfSupport (:198-202) reads it independently. Listing ['text','image'] without 'pdf' therefore leaves PDF support false, which was also the behavior before this change. No accidental widening.

Two things before I approve.

P2 — this does not actually close #3417, but it says it does. The issue asks for three things, and its "expected behavior" names the third explicitly: recognize image input, send the attachment, and display a friendly model name and description instead of the raw ID. This PR delivers the first two. The picker will keep showing the bare deepseek-v4-flash-vision-exp string, because the generated models.dev snapshot does not carry this model — that absence is the entire reason this entry has to be hand-written in the first place. Fixes #3417 will auto-close the issue on merge with a third of it still broken, and the reporter will have to reopen it.

ModelMetadata already has displayName and description (model-metadata.ts:28-29), and six entries in this same table already use them, so the fix is two more lines in the object you are already adding. Please add them. If you would rather scope this PR to the vision capability alone, that is fine too — then change the body to Refs #3417 so the issue stays open.

P3 — lastUpdated is omitted. Both sibling entries, deepseek-v4-flash and deepseek-v4-pro, carry it. Not load-bearing, but this is a hand-maintained table whose staleness is otherwise invisible, which is precisely what that field exists to make visible.

One thing I could not verify and am not treating as a finding: I have no way to confirm from this repository that DeepSeek's /models really returns this ID with image input. I am taking the issue reporter's first-hand reproduction for that, which seems reasonable — flagging it only so it is on the record that the model's existence is sourced from the issue, not from our own code.

Leaving this as a comment. Happy to approve as soon as the display name and description are in, or the body is retargeted to Refs.

AI use: Claude Code (Opus) traced resolveModelVisionSupport, resolveModelInputModalities, resolveModelPdfSupport, and lookupModelMetadata in packages/core/src/model-metadata.ts, and checked the sibling entries and the displayName precedent in the same table. The reviewer of record reviewed and accepted that reasoning.

简体中文

STATIC_MODEL_METADATA.deepseek 中登记 deepseek-v4-flash-vision-expvision: truemodalities.input: ['text','image']),加入 CURATED_CATALOG_FALLBACK_MODELS.deepseek,并用回归测试钉住三项事实(#3417)。

机制是对的,我按代码核实而非采信正文:

  • resolveModelVisionSupportmodel-metadata.ts:163-178)在库存查不到时回落到 lookupModelMetadata,返回 metadata.capabilities?.vision === true。没有条目时返回 false,正是 issue 描述的过滤行为;新条目把它翻转过来。
  • STATIC_MODEL_METADATA 是在 lookupModelMetadata:79-98)中叠加于 models.dev 生成快照之上的覆盖层,因此对于 models.dev 尚未收录的模型,这是正确的接缝。属于扩展既有权威,而非新增第二个。
  • modalities.input 在这里确实是独立权威,不是 vision 的复述——resolveModelPdfSupport:198-202)独立读取它。列 ['text','image'] 而不含 'pdf',因此 PDF 支持仍为 false,与改动前一致,没有意外放宽。

Approve 之前有两点。

P2 —— 它实际上没有关闭 #3417,但正文声称关闭了。 issue 要求三件事,"预期行为"明确点出第三件:识别图片输入、发送附件、以及显示友好的模型名称和说明而非原始 ID。本 PR 交付了前两件。模型选择器仍会显示裸的 deepseek-v4-flash-vision-exp,因为 models.dev 生成快照里没有这个模型——而这份缺失正是这个条目必须手写的全部原因。Fixes #3417 会在合并时自动关闭 issue,而三分之一仍未修复,报告者还得重开。

ModelMetadata 已有 displayNamedescriptionmodel-metadata.ts:28-29),同一张表里已有六个条目在用,所以补齐只是你正在添加的这个对象里再加两行。请补上。若你更想把本 PR 收窄到只做视觉能力,也可以——那就把正文改成 Refs #3417,让 issue 保持打开。

P3 —— 缺 lastUpdated 兄弟条目 deepseek-v4-flashdeepseek-v4-pro 都有。它不承重,但这是一张手工维护的表,其陈旧程度本来不可见,而这个字段的存在正是为了让它可见。

有一点我无法核实,也不作为 finding:我没法从本仓库确认 DeepSeek 的 /models 确实返回该 ID 且支持图片输入。这一点我采信 issue 报告者的第一手复现,看起来是合理的——写出来只是留档说明该模型的存在性来自 issue,而非来自我们自己的代码。

先留 comment。补上显示名与说明,或把正文改为 Refs,我即可 Approve。

yunaremaia and others added 4 commits August 27, 2026 21:29
…flash-vision-exp

Addresses reviewer feedback on apache#3605: adds displayName, description, and lastUpdated
fields to the STATIC_MODEL_METADATA entry so the model picker shows a friendly name
instead of the bare model ID.

Co-authored-by: Astro-Han <Astro-Han@users.noreply.github.com>
- Added tests to verify vision support, modalities, and metadata lookup
- Added catalog test for V4 vision model display metadata
- Fixes reviewer feedback on apache#3605
…l-metadata.test.ts

The function is exported from model-catalog.ts, not model-metadata.ts.
Local vitest runs pass because it compiles differently than tsc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/S Under 100 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(core): DeepSeek V4 Flash Vision is treated as text-only

3 participants