Skip to content

Commit 8b199d7

Browse files
authored
fix(files): serve rendered documents and stream workspace archives (#5995)
* fix(files): serve rendered documents instead of generator source Generated docs (docx/pptx/pdf/xlsx) store their generation source as the primary file; the rendered binary lives in a separate content-addressed artifact store. resolveServableDocBytes is the chokepoint that swaps one for the other, and the serve route, single-file download and ~50 tool routes all go through it. Archive compression and the public v1 download read raw bytes instead, so a generated document arrived as source text under a .docx name and Word reported it as corrupt. Both now resolve through the servable reader. Adds fetchServableWorkspaceFileBuffer beside the raw reader so the record to UserFile mapping lives in one place, preserving storageContext; the raw reader's doc comment now says it returns generation source, so the next call site has to choose deliberately. v1 also has to send the resolved content type rather than the record's source MIME, and returns a retryable 409 rather than a 500 when an artifact is still compiling. docNotReadyMessage centralizes the 409 copy, and isRenderableDocumentName is shared so the read path and its callers agree on which extensions can expand. * perf(files): stream workspace archives instead of buffering them The bulk download route materialized every selected file before writing a byte, so peak memory tracked the size of the selection. Ordinary files are now appended as lazy streams: each opens its storage read only when the archiver reaches it, so one entry is resident at a time rather than the whole archive. Generated documents still resolve to buffers first. They are the only entries whose bytes decide anything, and every status this route returns comes from them — once the first byte is written the status code is committed, so those decisions have to happen before the archive starts. The per-entry allowance and byte budget therefore govern documents only. archiver processes appended entries through a sequential queue; lazystream defers each storage read until that entry's turn, since handing the archiver an open stream per entry would hold more connections than the storage client pools. nodeReadableToWebStream moves out of input-validation.server.ts into a shared util rather than being written twice: Readable.toWeb throws ERR_INVALID_STATE when a consumer cancels while the source is still flowing, which is exactly what a cancelled download does. Trade: a storage read failing mid-archive truncates the response rather than returning 500, since the status is already sent. Documents cannot hit this. The table export route already behaves this way. * improvement(files): surface archive download errors in place Bulk and folder downloads navigated to the API route, so any rejection replaced the Files page with the raw JSON error body. Single-file download already fetched and saved the blob, so the two paths had diverged. That was survivable when the only failures were "too many files" and "too large"; resolving rendered documents adds a 409 for a still-compiling artifact, which is reachable in normal use. Both archive paths now fetch the zip and show the server's message as a toast — the route writes that copy for a person, so it is worth surfacing rather than discarding. Single-file download shows its error too instead of only logging it. * improvement(files): simplify the archive route and stop buffering real uploads Four review passes over the branch converged on the same points. Routing every office extension through the buffered path held an entire selection of genuinely uploaded documents in memory — for a check the resolver settles on the first few magic bytes. Only files stored under a generator-source content type need resolving; a real .docx serves what is stored and now streams like anything else, which is what the change was for. With resolution sequential, the AbortController cancelled nothing (its signal never reached the storage read), the success-path over-limit branch was unreachable, and the cancellation guard in the catch could not fire. A plain loop that returns at the point of failure replaces the outcome record, the sentinel, the fan-out helper at limit 1, and four post-hoc scans. ZIP_MATERIALIZE_CONCURRENCY was dead, and the aggregate over-limit message reported a byte count that was by construction under the limit. lazystream and its types are gone: Readable.from over an async generator defers the open the same way and propagates source errors natively, so the PassThrough relay went with them. The client uses requestRaw with the existing binary contract instead of a hand-built query string and a re-typed error extractor, and both archive call sites share one callback. The render headroom moves beside isRenderableDocumentName, the servable reader stops minting a request id per file, and the v1 download returns a view rather than a second full copy. * improvement(files): keep the lazy entry stream in byte mode Readable.from defaults to object mode; these chunks are bytes headed for an archive, so the mode is now explicit. Also makes the fire-and-forget bulk download call consistent with its sibling. * improvement(files): correct the servable reader's 409 note Batch callers build the response from docNotReadyMessage rather than through docNotReadyResponse, so the doc comment now describes the outcome instead of naming one of the two helpers. * improvement(files): correct two comments that described the wrong behavior The resolution loop's comment claimed at most one rendered document is resident. It is not: every resolved buffer is held until the archive is assembled, bounded by the request's remaining budget. The loop resolves one at a time, it does not release one at a time. RENDERED_DOCUMENT_HEADROOM_BYTES was documented as bounding the expansion beyond the declared size, but it is passed straight through as maxBytes and is an absolute ceiling on the rendered document — renamed to MAX_RENDERED_DOCUMENT_BYTES so the name matches. Download failures also carry a fallback message, so a transport error surfaces something better than a bare "Failed to fetch". * fix(files): put streamed files and rendered documents on one byte budget The budget only counted rendered documents, so streamed entries never reserved their declared bytes. A mixed selection could clear the up-front declared-size gate and still ship close to two full limits: ordinary files up to the cap, plus documents drawing a fresh cap of their own. Streamed entries ship exactly what they declared, so their share is known before anything is read and is now reserved up front; documents draw from what is left. The budget-exhausted rejection also quoted declared sizes, which for a selection of small sources reads as a tiny total exceeding the limit. That case has no knowable byte count — the documents have not been rendered — so it now says what happened without inventing a number. * fix(files): attach the download anchor and handle a finalize rejection A detached anchor's click() works in current browsers, but every other download helper in the app attaches first, and a silent no-op on this path would look exactly like a download that never started. Not worth the ambiguity for two DOM operations. archive.finalize() was fired with void, so a failure after the response had started became an unhandled rejection on top of the stream error event that already fails the response. It is caught and logged now. * fix(files): guard the archive download against concurrent clicks Navigating to the route meant a second click just re-navigated. Fetching the archive instead means each click starts another download that holds the whole zip in tab memory, and the Download button gave no sign anything was happening. The button now reflects the in-flight download alongside the existing move and delete states. The ref guard is there as well as the state because two clicks in the same tick would both pass a state check. * fix(files): resolve every generated-document source type, not four of five The archive keyed off a locally enumerated set of source content types that omitted text/x-pdflibjs — the isolated-vm PDF generator. Those files failed the check and streamed their generator source under a .pdf name, which is the exact corruption this change exists to fix. A canonical five-member set already existed in the file viewer; enumerating a fourth copy was the mistake, not the missing string. The set moves to file-utils beside the other file-type predicates, the viewer imports it, and the route asks isGeneratedDocumentSourceType rather than carrying its own list. A parameterized test pins all five, so adding a generator without extending the set fails. * test(files): validate the produced archive with a real zip reader Every existing test mocks the storage stream and reads the result back with the same JS library that wrote it, which cannot catch the failure this route exists to fix: an archive a real zip reader will not open. This drives 5 MB of non-repeating bytes from an fs stream through archiver, the lazy entry generator and the Node-to-web bridge, then checks the output with the operating system's unzip and compares the extracted bytes. Skips rather than fails where unzip is unavailable. * fix(files): bound the markdown export's embedded-asset bundling The embed list is produced by scanning the document body, so its length and the bytes behind it are whatever the author put there. Every referenced asset was downloaded at once with no concurrency bound, no per-file cap and no total cap, so one request could materialize an unbounded number of unbounded files. Its sibling bulk-download route already had a count cap and a byte cap. Metadata is now resolved first, which bounds the download from declared sizes before a byte is read and moves the authorization check off the download path. Assets are then fetched with bounded concurrency and a per-file cap; a single unreadable or oversized asset drops out of the bundle rather than failing the export, which is what the previous allSettled did. * fix(files): mount rendered documents into the sandbox, not generator source Mounting a generated document handed the sandbox its generator source under a .docx name, so a python-docx or openpyxl script failed on a file that looked fine and the agent debugged code that was correct. Both branches were wrong, not just the buffered one: with cloud storage the mount presigns record.key, which is the raw source object, so swapping the buffered read alone would have fixed only local storage. Source-backed documents now always take the servable path — they are bounded by the render ceiling, so routing them through the web process instead of presigning is affordable. The text/binary decision also keyed off record.type, which for these files is text/x-docxjs, so a resolved binary would have been decoded as UTF-8. It reads the resolved content type now. * chore(deps): regenerate the lockfile against staging's dependency rework Staging reworked the OTel split, dropped dead deps and declared emcn peers, so the lockfile is regenerated on top of that rather than carrying a stale merge. The archive dependency is the only addition; lazystream stays as archiver's transitive dep now that it is no longer declared directly. Regenerated incrementally rather than from scratch: a clean resolve fails the bunfig age gate because minimumReleaseAgeExcludes lists only the darwin and linux-gnu @next/swc binaries, not the musl and windows variants that a full re-resolve also pulls. * fix(files): budget sandbox mounts on rendered bytes, not declared source size A source-backed document declares the size of its generator, not of the document, so the per-file and aggregate mount pre-checks were reading a number unrelated to what was about to be mounted — tiny sources cleared the guard and only afterwards was the running total updated with the real length. Those pre-checks now apply only to files that serve what they declare. A document is capped by the read instead, at the smaller of the per-file limit and the budget left, and a size rejection is reported in the same terms as the other mount limits. Same invariant the archive route already had to learn: declared size is not a bound once bytes can be rendered. * test(files): cover the two surfaces added without tests The markdown export route had no test file at all, and the sandbox mount's source-backed branch was exercised by nothing — both were changed in this PR and one of them shipped a defect a reviewer caught within minutes. Export: the embed-count rejection, the declared-bytes rejection landing before any asset leaves storage, the per-asset download cap, an unreadable asset dropping out rather than failing the export, and an unauthorized asset never being read. Mount: a generated document never presigning its raw key even on cloud storage, its rendered bytes mounting as base64 rather than being utf-8 decoded off the source MIME, and the budget rejecting on rendered length. * fix(files): stop the export caps from rejecting documents that used to work The caps I picked would have failed exports that previously succeeded: a screenshot-heavy document can hold more than 100 embeds, and 100 embeds of unoptimised PNGs can exceed 100 MB. Adding a limit to bound memory is right; setting it below real usage turns a memory risk into a broken feature. The byte ceiling now matches the bulk-download route, so the two export surfaces reject at the same size rather than at two invented ones. The count is only a guard on the metadata lookups that run before the byte check, so it moves well above any hand-authored document instead of sitting where a legitimate one could reach it.
1 parent 6aa3e0e commit 8b199d7

20 files changed

Lines changed: 1374 additions & 199 deletions

File tree

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
/**
2+
* @vitest-environment node
3+
*/
4+
import { createMockRequest } from '@sim/testing'
5+
import JSZip from 'jszip'
6+
import { beforeEach, describe, expect, it, vi } from 'vitest'
7+
8+
const {
9+
mockCheckAuth,
10+
mockGetFileMetadataById,
11+
mockVerifyFileAccess,
12+
mockDownloadFile,
13+
mockExtractEmbeddedImageIds,
14+
} = vi.hoisted(() => ({
15+
mockCheckAuth: vi.fn(),
16+
mockGetFileMetadataById: vi.fn(),
17+
mockVerifyFileAccess: vi.fn(),
18+
mockDownloadFile: vi.fn(),
19+
mockExtractEmbeddedImageIds: vi.fn(),
20+
}))
21+
22+
vi.mock('@/lib/auth/hybrid', () => ({ checkSessionOrInternalAuth: mockCheckAuth }))
23+
vi.mock('@/lib/uploads/server/metadata', () => ({
24+
getFileMetadataById: mockGetFileMetadataById,
25+
}))
26+
vi.mock('@/app/api/files/authorization', () => ({ verifyFileAccess: mockVerifyFileAccess }))
27+
vi.mock('@/lib/uploads/core/storage-service', () => ({ downloadFile: mockDownloadFile }))
28+
vi.mock('@/lib/copilot/tools/server/files/embedded-image-refs', () => ({
29+
extractEmbeddedImageIds: mockExtractEmbeddedImageIds,
30+
}))
31+
vi.mock('@sim/audit', () => ({
32+
recordAudit: vi.fn(),
33+
AuditAction: { FILE_DOWNLOADED: 'file.downloaded' },
34+
AuditResourceType: { FILE: 'file' },
35+
}))
36+
vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() }))
37+
38+
import { GET } from '@/app/api/files/export/[id]/route'
39+
40+
const MB = 1024 * 1024
41+
const DOC_ID = 'doc-1'
42+
const context = { params: Promise.resolve({ id: DOC_ID }) }
43+
44+
function request() {
45+
return createMockRequest('GET', undefined, {}, `http://localhost:3000/api/files/export/${DOC_ID}`)
46+
}
47+
48+
function assetRecord(id: string, size: number) {
49+
return {
50+
id,
51+
key: `workspace/ws-1/${id}`,
52+
originalName: `${id}.png`,
53+
contentType: 'image/png',
54+
context: 'workspace',
55+
size,
56+
workspaceId: 'ws-1',
57+
}
58+
}
59+
60+
describe('markdown export bundling', () => {
61+
beforeEach(() => {
62+
vi.clearAllMocks()
63+
mockCheckAuth.mockResolvedValue({ success: true, userId: 'user-1' })
64+
mockVerifyFileAccess.mockResolvedValue(true)
65+
mockGetFileMetadataById.mockImplementation(async (id: string) =>
66+
id === DOC_ID
67+
? {
68+
id: DOC_ID,
69+
key: 'workspace/ws-1/doc.md',
70+
originalName: 'doc.md',
71+
contentType: 'text/markdown',
72+
context: 'workspace',
73+
size: 1024,
74+
workspaceId: 'ws-1',
75+
}
76+
: assetRecord(id, 1 * MB)
77+
)
78+
mockDownloadFile.mockResolvedValue(Buffer.from('# Doc\n'))
79+
mockExtractEmbeddedImageIds.mockReturnValue([])
80+
})
81+
82+
it('rejects a document embedding more assets than an export may bundle', async () => {
83+
mockExtractEmbeddedImageIds.mockReturnValue(
84+
Array.from({ length: 501 }, (_, index) => `img-${index}`)
85+
)
86+
87+
const response = await GET(request(), context)
88+
89+
expect(response.status).toBe(400)
90+
expect((await response.json()).error).toContain('501')
91+
// Rejected on the embed count alone: nothing was resolved or downloaded.
92+
expect(mockGetFileMetadataById).toHaveBeenCalledTimes(1)
93+
})
94+
95+
it('rejects on declared asset bytes before downloading any of them', async () => {
96+
mockExtractEmbeddedImageIds.mockReturnValue(['a', 'b', 'c'])
97+
mockGetFileMetadataById.mockImplementation(async (id: string) =>
98+
id === DOC_ID
99+
? {
100+
id: DOC_ID,
101+
key: 'workspace/ws-1/doc.md',
102+
originalName: 'doc.md',
103+
contentType: 'text/markdown',
104+
context: 'workspace',
105+
size: 1024,
106+
workspaceId: 'ws-1',
107+
}
108+
: assetRecord(id, 100 * MB)
109+
)
110+
111+
const response = await GET(request(), context)
112+
113+
expect(response.status).toBe(400)
114+
expect((await response.json()).error).toContain('exceeds')
115+
// Only the markdown body was read; the 300 MB of assets never left storage.
116+
expect(mockDownloadFile).toHaveBeenCalledTimes(1)
117+
})
118+
119+
it('caps each asset download rather than trusting its declared size', async () => {
120+
mockExtractEmbeddedImageIds.mockReturnValue(['a'])
121+
122+
await GET(request(), context)
123+
124+
const assetCall = mockDownloadFile.mock.calls.find(
125+
([options]) => options.key === 'workspace/ws-1/a'
126+
)
127+
expect(assetCall?.[0].maxBytes).toBe(25 * MB)
128+
})
129+
130+
it('drops an unreadable asset instead of failing the whole export', async () => {
131+
mockExtractEmbeddedImageIds.mockReturnValue(['good', 'bad'])
132+
mockDownloadFile.mockImplementation(async ({ key }: { key: string }) => {
133+
if (key.endsWith('doc.md')) return Buffer.from('# Doc\n![x](/api/files/view/good)\n')
134+
if (key.endsWith('bad')) throw new Error('storage down')
135+
return Buffer.from('png-bytes')
136+
})
137+
138+
const response = await GET(request(), context)
139+
140+
expect(response.status).toBe(200)
141+
const zip = await JSZip.loadAsync(Buffer.from(await response.arrayBuffer()))
142+
expect(zip.file('assets/good.png')).not.toBeNull()
143+
expect(zip.file('assets/bad.png')).toBeNull()
144+
})
145+
146+
it('skips an asset the caller cannot read', async () => {
147+
mockExtractEmbeddedImageIds.mockReturnValue(['secret'])
148+
mockVerifyFileAccess.mockImplementation(async (key: string) => !key.endsWith('secret'))
149+
150+
const response = await GET(request(), context)
151+
152+
expect(response.status).toBe(200)
153+
// Authorization is settled during metadata resolution, before any asset read.
154+
expect(mockDownloadFile.mock.calls.some(([options]) => options.key.endsWith('secret'))).toBe(
155+
false
156+
)
157+
})
158+
})

apps/sim/app/api/files/export/[id]/route.ts

Lines changed: 77 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,33 @@ import { fileExportContract } from '@/lib/api/contracts/storage-transfer'
99
import { parseRequest } from '@/lib/api/server'
1010
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
1111
import { extractEmbeddedImageIds } from '@/lib/copilot/tools/server/files/embedded-image-refs'
12+
import { MATERIALIZE_CONCURRENCY, mapWithConcurrency } from '@/lib/core/utils/concurrency'
1213
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
1314
import { captureServerEvent } from '@/lib/posthog/server'
1415
import type { StorageContext } from '@/lib/uploads/config'
1516
import { getServeStoragePrefix } from '@/lib/uploads/config'
1617
import { downloadFile } from '@/lib/uploads/core/storage-service'
1718
import { getFileMetadataById } from '@/lib/uploads/server/metadata'
19+
import { formatFileSize } from '@/lib/uploads/utils/file-utils'
1820
import { verifyFileAccess } from '@/app/api/files/authorization'
1921
import { encodeFilenameForHeader } from '@/app/api/files/utils'
2022

2123
const logger = createLogger('FilesExportAPI')
2224

25+
/**
26+
* Bundling caps. The embed list comes from scanning the document body, so its length
27+
* and the bytes behind it are whatever the author put there — without these the export
28+
* would materialize an unbounded number of unbounded assets in one request.
29+
*
30+
* The byte ceilings are the real bound and match the bulk-download route, so the two
31+
* export surfaces reject at the same size. The count is only a guard on the metadata
32+
* lookups that precede the byte check, so it sits far above any hand-authored document
33+
* rather than at a number a screenshot-heavy doc could plausibly reach.
34+
*/
35+
const MAX_EXPORT_ASSETS = 500
36+
const MAX_EXPORT_ASSET_BYTES = 25 * 1024 * 1024
37+
const MAX_EXPORT_TOTAL_BYTES = 250 * 1024 * 1024
38+
2339
const MARKDOWN_MIME_TYPES = new Set(['text/markdown', 'text/x-markdown'])
2440
const MARKDOWN_EXTENSIONS = new Set(['md', 'markdown'])
2541

@@ -136,34 +152,73 @@ export const GET = withRouteHandler(
136152
})
137153
}
138154

139-
const fetchResults = await Promise.allSettled(
140-
imageIds.map(async (imageId) => {
141-
const imgRecord = await getFileMetadataById(imageId)
142-
if (!imgRecord) return null
143-
const imgHasAccess = await verifyFileAccess(imgRecord.key, userId)
144-
if (!imgHasAccess) return null
145-
const imgBuffer = await downloadFile({
146-
key: imgRecord.key,
147-
context: imgRecord.context as StorageContext,
148-
})
149-
return { imageId, originalName: imgRecord.originalName, buffer: imgBuffer }
155+
if (imageIds.length > MAX_EXPORT_ASSETS) {
156+
return NextResponse.json(
157+
{
158+
error: `This document embeds ${imageIds.length} files, more than the ${MAX_EXPORT_ASSETS} an export can bundle.`,
159+
},
160+
{ status: 400 }
161+
)
162+
}
163+
164+
// Metadata first: declared sizes bound the download before a byte is read, and the
165+
// authorization check costs nothing to run here.
166+
const assetTargets = (
167+
await mapWithConcurrency(imageIds, MATERIALIZE_CONCURRENCY, async (imageId) => {
168+
try {
169+
const imgRecord = await getFileMetadataById(imageId)
170+
if (!imgRecord) return null
171+
if (!(await verifyFileAccess(imgRecord.key, userId))) return null
172+
return { imageId, record: imgRecord }
173+
} catch (error) {
174+
logger.warn('Failed to resolve asset for export', {
175+
imageId,
176+
error: toError(error).message,
177+
})
178+
return null
179+
}
150180
})
181+
).filter((target): target is NonNullable<typeof target> => target !== null)
182+
183+
const declaredAssetBytes = assetTargets.reduce((sum, target) => sum + target.record.size, 0)
184+
if (declaredAssetBytes > MAX_EXPORT_TOTAL_BYTES) {
185+
return NextResponse.json(
186+
{
187+
error: `Embedded files total ${formatFileSize(declaredAssetBytes)}, which exceeds the ${formatFileSize(MAX_EXPORT_TOTAL_BYTES)} export limit.`,
188+
},
189+
{ status: 400 }
190+
)
191+
}
192+
193+
const fetched = await mapWithConcurrency(
194+
assetTargets,
195+
MATERIALIZE_CONCURRENCY,
196+
async ({ imageId, record: imgRecord }) => {
197+
try {
198+
const buffer = await downloadFile({
199+
key: imgRecord.key,
200+
context: imgRecord.context as StorageContext,
201+
maxBytes: MAX_EXPORT_ASSET_BYTES,
202+
})
203+
return { imageId, originalName: imgRecord.originalName, buffer }
204+
} catch (error) {
205+
// A single unreadable or oversized asset drops out of the bundle rather than
206+
// failing the whole export; the markdown keeps its original link.
207+
logger.warn('Failed to fetch asset for export', {
208+
imageId,
209+
error: toError(error).message,
210+
})
211+
return null
212+
}
213+
}
151214
)
152215

153216
const assetMap = new Map<string, { filename: string; buffer: Buffer }>()
154217
const usedFilenames = new Set<string>()
155218

156-
for (let i = 0; i < fetchResults.length; i++) {
157-
const result = fetchResults[i]
158-
if (result.status === 'rejected') {
159-
logger.warn('Failed to fetch asset for export', {
160-
imageId: imageIds[i],
161-
error: toError(result.reason).message,
162-
})
163-
continue
164-
}
165-
if (!result.value) continue
166-
const { imageId, originalName, buffer } = result.value
219+
for (const result of fetched) {
220+
if (!result) continue
221+
const { imageId, originalName, buffer } = result
167222
const preferred = safeFilename(originalName)
168223
const filename = deduplicatedFilename(preferred, usedFilenames, imageId)
169224
usedFilenames.add(filename)

apps/sim/app/api/tools/file/manage/route.ts

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import { checkInternalAuth } from '@/lib/auth/hybrid'
1111
import { splitWorkspaceFilePath } from '@/lib/copilot/tools/server/files/workspace-file'
1212
import { acquireLock, releaseLock } from '@/lib/core/config/redis'
1313
import { generateRequestId } from '@/lib/core/utils/request'
14+
import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits'
1415
import { ensureAbsoluteUrl } from '@/lib/core/utils/urls'
1516
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
1617
import { isSupportedFileType, parseBuffer } from '@/lib/file-parsers'
@@ -685,7 +686,11 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
685686
const denied = await assertToolFileAccess(userFile.key, userId, requestId, logger)
686687
if (denied) return denied
687688

688-
const buffer = await downloadFileFromStorage(userFile, requestId, logger, {
689+
// Generated docs store their generation source, not the rendered binary, so
690+
// the archive must carry the servable bytes instead of the raw source text.
691+
// A still-compiling artifact throws, and the handler's catch turns that into
692+
// the shared 409 via `docNotReadyResponse`.
693+
const { buffer } = await downloadServableFileFromStorage(userFile, requestId, logger, {
689694
maxBytes: MAX_COMPRESS_FILE_BYTES,
690695
})
691696
totalBytes += buffer.length
@@ -864,6 +869,11 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
864869
}
865870
const notReady = docNotReadyResponse(error)
866871
if (notReady) return notReady
872+
// A file over its per-file cap is a size rejection, not a fault. Rendered
873+
// documents can cross it even when the stored source was well under.
874+
if (isPayloadSizeLimitError(error)) {
875+
return NextResponse.json({ success: false, error: error.message }, { status: 413 })
876+
}
867877
if (error instanceof ShareValidationError) {
868878
return NextResponse.json({ success: false, error: error.message }, { status: 400 })
869879
}

apps/sim/app/api/v1/files/[fileId]/route.ts

Lines changed: 31 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,11 @@ import { parseRequest } from '@/lib/api/server'
66
import { generateRequestId } from '@/lib/core/utils/request'
77
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
88
import { captureServerEvent } from '@/lib/posthog/server'
9-
import { fetchWorkspaceFileBuffer, getWorkspaceFile } from '@/lib/uploads/contexts/workspace'
9+
import {
10+
fetchServableWorkspaceFileBuffer,
11+
getWorkspaceFile,
12+
} from '@/lib/uploads/contexts/workspace'
13+
import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/servable-file-response'
1014
import { performDeleteWorkspaceFileItems } from '@/lib/workspace-files/orchestration'
1115
import {
1216
checkRateLimit,
@@ -48,7 +52,9 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo
4852
return NextResponse.json({ error: 'File not found' }, { status: 404 })
4953
}
5054

51-
const buffer = await fetchWorkspaceFileBuffer(fileRecord)
55+
// Generated docs store their generation source; serve the rendered artifact.
56+
// Its content type is the rendered one, not the source MIME on the record.
57+
const { buffer, contentType } = await fetchServableWorkspaceFileBuffer(fileRecord)
5258

5359
recordAudit({
5460
workspaceId,
@@ -73,21 +79,30 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo
7379
{ groups: { workspace: workspaceId } }
7480
)
7581

76-
return new Response(new Uint8Array(buffer), {
77-
status: 200,
78-
headers: {
79-
'Content-Type': fileRecord.type || 'application/octet-stream',
80-
'Content-Disposition': `attachment; filename="${fileRecord.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(fileRecord.name)}`,
81-
'Content-Length': String(buffer.length),
82-
'X-File-Id': fileRecord.id,
83-
'X-File-Name': encodeURIComponent(fileRecord.name),
84-
'X-Uploaded-At':
85-
fileRecord.uploadedAt instanceof Date
86-
? fileRecord.uploadedAt.toISOString()
87-
: String(fileRecord.uploadedAt),
88-
},
89-
})
82+
// View, not copy — a second full copy would double peak memory for a large file.
83+
return new Response(
84+
new Uint8Array(buffer.buffer as ArrayBuffer, buffer.byteOffset, buffer.byteLength),
85+
{
86+
status: 200,
87+
headers: {
88+
'Content-Type': contentType || fileRecord.type || 'application/octet-stream',
89+
'Content-Disposition': `attachment; filename="${fileRecord.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(fileRecord.name)}`,
90+
'Content-Length': String(buffer.length),
91+
'X-File-Id': fileRecord.id,
92+
'X-File-Name': encodeURIComponent(fileRecord.name),
93+
'X-Uploaded-At':
94+
fileRecord.uploadedAt instanceof Date
95+
? fileRecord.uploadedAt.toISOString()
96+
: String(fileRecord.uploadedAt),
97+
},
98+
}
99+
)
90100
} catch (error) {
101+
// A generated doc whose artifact is still compiling is retryable, not a fault:
102+
// without this the caller sees a 500 and has no reason to try again.
103+
if (isDocNotReadyError(error)) {
104+
return NextResponse.json({ error: docNotReadyMessage() }, { status: 409 })
105+
}
91106
logger.error(`[${requestId}] Error downloading file:`, error)
92107
return NextResponse.json({ error: 'Failed to download file' }, { status: 500 })
93108
}

0 commit comments

Comments
 (0)