feat(storage): 遗留文件值回填 — ADR-0104 D3 wave 2 (PR-6) - #3535
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
documentation
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 6 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
backfillFileReferences() converts the pre-reference forms a file/image/avatar/
video/audio field may hold — an inline metadata blob ({url, name, size, …}) or
a bare URL string — into the reference form: an opaque sys_file id, owned by the
record's field.
What it will and will not convert:
- A URL naming this platform's own resolver (…/storage/files/:id) already
identifies a sys_file; the field is rewritten to the bare id, no bytes move.
- A data: URI carries its bytes inline; they are uploaded, a sys_file is
registered, and the field is rewritten to its id.
- An external URL is REPORTED, NEVER CONVERTED. Re-hosting third-party
content is a bandwidth, licensing and privacy decision that is not a
migration's to make. ADR-0104 R7 retires these toward an explicit `url`
field, which under AI authoring is the point: it stops "managed file" and
"external link" being the same declaration.
Dry run by default — nothing is written unless apply is set, and the dry-run
report has the same shape as the applied one so the plan can be reviewed and
diffed against what actually happened. Idempotent — a value already in
reference form is recorded as already_id and left alone, so a partially
completed run is safe to repeat.
The backfill never writes the ref_* columns itself: it rewrites the record, and
the claim hooks observe that write and record ownership. One claiming path, so
there is nothing that can disagree with itself — asserted by a test that fails
if the backfill ever starts touching sys_file directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SHpGw3GBA9aFpfwVArRWfd
The 2026-07-27 addendum sequenced the write cutover before the backfill. That is wrong for the same reason the backfill must precede collection, one step further back: narrowing the accepted stored form while records still hold legacy values would reject any update that rewrites such a field, breaking working apps until the backfill catches up. Backfilling first leaves the cutover nothing legacy to reject. Also states plainly why the last two steps are held to different standards. The cutover breaks loudly and recoverably (a rejected write); enabling collection breaks silently and permanently (deleted bytes). Only the second earns the reconciliation evidence requirement, and neither it nor the migration run should happen without an explicit human decision. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHpGw3GBA9aFpfwVArRWfd
…tracking regex CodeQL js/polynomial-redos: the resolver-URL pattern was anchored only at the tail, so the engine retried from every start position. The values it scans come straight out of tenant records, so a value repeating the matched prefix turns that into a polynomial blowup on attacker-influenced data. Replaced with slicing and splitting — linear in the URL's length whatever it contains, and clearer about what it actually accepts. Covered by a timing regression on an adversarial value plus a table of accepted shapes and near-misses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHpGw3GBA9aFpfwVArRWfd
os-zhuang
marked this pull request as ready for review
July 27, 2026 03:23
This was referenced Jul 27, 2026
This was referenced Jul 27, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
承接 #3527（PR-3 独占所有权）。把 cutover 之前存在字段里的遗留形态,转换成引用形态。
转换什么、不转换什么
file/image/avatar/video/audio字段在 cutover 前可能持有 inline 元数据 blob（{url, name, size, …}）或裸 URL 字符串。backfillFileReferences()把它们转成引用形态——一个由该记录字段拥有的不透明sys_fileid。…/storage/files/:id）sys_file,字段改写为裸 iddata:URIsys_file→ 字段改写为新 idhttps://cdn.example.com/…）为什么外部 URL 不转换:那意味着去抓取第三方内容并悄悄地重新托管它——这是带宽、许可和隐私上的决定,不是一次迁移该替用户做的。ADR-0104 R7 的方向是把它们迁到显式的
url字段,而在 AI 写元数据的语境下这正是重点:它让"受管文件"和"外部链接"不再是同一个声明,AI 就没法把两者搞混。安全性质
apply什么都不写,而且 dry-run 报告和实际执行的报告同构,所以计划可以先审、事后可以对差异。already_id并原样保留,所以跑到一半中断可以安全重跑。ref_*列:它只改写记录,由 PR-3 的 claim hook 观察到这次写入并记录所有权。只有一条认领路径,也就不存在两条路径互相矛盾的可能——有一条测试专门盯着这点,一旦回填开始直接碰sys_file就会失败。测试（11 项）
本地 resolver URL 改写（裸串 + 包在 inline blob 里两种）、
data:URI 上传并正确解出 mime/size/name、外部 URL 既不改也不上传、multiple:true逐元素处理（三种值混在一个数组里）、dry run 零写入、幂等（第二次 converted=0）、无存储服务时data:值保持原样、无 file 字段的对象跳过、分页 500 条 + 截断标记,以及上面那条"绝不写 ref_*"的断言。全量:service-storage 136 ✅ · nul-bytes ✅ · lint ✅
位置
按 ADR 2026-07-27 addendum 的重新排序,回填必须先于开启回收:一本账在被回填并对账证明与现实一致之前,不该被授予对不可逆删除的权力。跑完这个之后应当运行
verifyFileReferences()（#3534）确认两边一致。🤖 Generated with Claude Code
https://claude.ai/code/session_01SHpGw3GBA9aFpfwVArRWfd
Generated by Claude Code