Compress oversized images client-side in AI chat #1456
+103
−33
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| import { MAX_IMAGE_BYTES_PER_FILE } from "@stackframe/stack-shared/dist/ai/image-limits"; | ||
|
|
||
| /** | ||
| * Maximum pixel dimension (width or height) for compressed output. | ||
| * 2048px is plenty for AI chat while keeping file sizes manageable. | ||
| */ | ||
| const MAX_DIMENSION = 2048; | ||
|
|
||
| /** | ||
| * Target compressed size: well under the hard 3 MB server limit so the | ||
| * base-64 encoded payload (≈ 33 % larger) still fits comfortably. | ||
| */ | ||
| const COMPRESS_TARGET_BYTES = Math.floor(MAX_IMAGE_BYTES_PER_FILE / 2); | ||
|
|
||
| function canvasToBlob( | ||
| canvas: HTMLCanvasElement, | ||
| type: string, | ||
| quality: number, | ||
| ): Promise<Blob> { | ||
| return new Promise((resolve, reject) => { | ||
| canvas.toBlob( | ||
| (blob) => { | ||
| if (blob == null) { | ||
| reject(new Error("Canvas toBlob returned null")); | ||
| return; | ||
| } | ||
| resolve(blob); | ||
| }, | ||
| type, | ||
| quality, | ||
| ); | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * Compresses an image `File` on the client so it stays well under the | ||
| * server-side size limit. Returns the original file unchanged when it is | ||
| * already small enough. | ||
| * | ||
| * Strategy: | ||
| * 1. Down-scale to at most `MAX_DIMENSION` px on the longest side. | ||
| * 2. Encode as JPEG with decreasing quality until the result fits. | ||
| * 3. If quality alone isn't enough, halve the dimensions and retry. | ||
| */ | ||
| export async function compressImageFile(file: File): Promise<File> { | ||
| if (file.size <= MAX_IMAGE_BYTES_PER_FILE) { | ||
| return file; | ||
| } | ||
|
|
||
| const bitmap = await createImageBitmap(file); | ||
| const dimensionScale = Math.min( | ||
| 1, | ||
| MAX_DIMENSION / Math.max(bitmap.width, bitmap.height), | ||
| ); | ||
| const baseWidth = Math.round(bitmap.width * dimensionScale); | ||
| const baseHeight = Math.round(bitmap.height * dimensionScale); | ||
|
|
||
| const canvas = document.createElement("canvas"); | ||
| const ctx = canvas.getContext("2d"); | ||
| if (ctx == null) { | ||
| bitmap.close(); | ||
| throw new Error("Failed to get canvas 2d context for image compression"); | ||
| } | ||
|
|
||
| // Try progressively smaller sizes until the output fits. | ||
| for ( | ||
| let sizeScale = 1; | ||
| sizeScale >= 0.25; | ||
| sizeScale = Math.round((sizeScale * 0.5) * 100) / 100 | ||
| ) { | ||
| const w = Math.max(1, Math.round(baseWidth * sizeScale)); | ||
| const h = Math.max(1, Math.round(baseHeight * sizeScale)); | ||
| canvas.width = w; | ||
| canvas.height = h; | ||
| ctx.drawImage(bitmap, 0, 0, w, h); | ||
|
Comment on lines
+58
to
+75
Contributor
There was a problem hiding this comment.
Consider filling the canvas with white before drawing (a more neutral default for most transparent images): Prompt To Fix With AIThis is a comment left during a code review.
Path: apps/dashboard/src/components/assistant-ui/compress-image.ts
Line: 58-75
Comment:
**Transparent PNG alpha channel is silently replaced with black**
`canvas.getContext("2d")` draws with a default opaque black background, so any image with transparency (PNG, WebP) that exceeds 3 MB and triggers compression will have its transparent areas filled with solid black in the resulting JPEG. A user who uploads a large transparent logo or screenshot with a transparent background would see the preview and the AI would receive an image with a black background instead — no warning is shown.
Consider filling the canvas with white before drawing (a more neutral default for most transparent images): `ctx.fillStyle = "#ffffff"; ctx.fillRect(0, 0, w, h);` before the `ctx.drawImage` call.
How can I resolve this? If you propose a fix, please make it concise. |
||
|
|
||
| for (let quality = 0.85; quality >= 0.15; quality -= 0.1) { | ||
| const blob = await canvasToBlob(canvas, "image/jpeg", quality); | ||
| if (blob.size <= COMPRESS_TARGET_BYTES) { | ||
| bitmap.close(); | ||
| return new File([blob], file.name, { | ||
| type: "image/jpeg", | ||
| lastModified: file.lastModified, | ||
| }); | ||
|
Comment on lines
+81
to
+84
Contributor
There was a problem hiding this comment.
The compressed file preserves the original name (e.g., Prompt To Fix With AIThis is a comment left during a code review.
Path: apps/dashboard/src/components/assistant-ui/compress-image.ts
Line: 81-84
Comment:
**File name extension disagrees with the JPEG MIME type**
The compressed file preserves the original name (e.g., `photo.png`) while `type` is set to `"image/jpeg"`. The `name` field is surfaced in the composer UI and in the `PendingAttachment`, so a user will see a `.png` filename attached even though the payload is JPEG. The mismatch is unlikely to break the AI call, but it can be confusing and could trip any future server-side validation that checks the extension. Renaming to append `.jpg` (or strip the old extension and add `.jpg`) when compression is applied would keep things consistent.
How can I resolve this? If you propose a fix, please make it concise. |
||
| } | ||
| } | ||
| } | ||
|
|
||
| // Fallback: lowest quality at the smallest attempted dimension. | ||
| const blob = await canvasToBlob(canvas, "image/jpeg", 0.1); | ||
| bitmap.close(); | ||
| return new File([blob], file.name, { | ||
| type: "image/jpeg", | ||
| lastModified: file.lastModified, | ||
| }); | ||
| } | ||
|
Comment on lines
+50
to
+96
Contributor
There was a problem hiding this comment. Ensure
Suggested fix export async function compressImageFile(file: File): Promise<File> {
if (file.size <= MAX_IMAGE_BYTES_PER_FILE) {
return file;
}
- const bitmap = await createImageBitmap(file);
+ const bitmap = await createImageBitmap(file);
const dimensionScale = Math.min(
1,
MAX_DIMENSION / Math.max(bitmap.width, bitmap.height),
);
const baseWidth = Math.round(bitmap.width * dimensionScale);
const baseHeight = Math.round(bitmap.height * dimensionScale);
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
if (ctx == null) {
- bitmap.close();
throw new Error("Failed to get canvas 2d context for image compression");
}
- // Try progressively smaller sizes until the output fits.
- for (
- let sizeScale = 1;
- sizeScale >= 0.25;
- sizeScale = Math.round((sizeScale * 0.5) * 100) / 100
- ) {
- const w = Math.max(1, Math.round(baseWidth * sizeScale));
- const h = Math.max(1, Math.round(baseHeight * sizeScale));
- canvas.width = w;
- canvas.height = h;
- ctx.drawImage(bitmap, 0, 0, w, h);
-
- for (let quality = 0.85; quality >= 0.15; quality -= 0.1) {
- const blob = await canvasToBlob(canvas, "image/jpeg", quality);
- if (blob.size <= COMPRESS_TARGET_BYTES) {
- bitmap.close();
- return new File([blob], file.name, {
- type: "image/jpeg",
- lastModified: file.lastModified,
- });
- }
- }
- }
-
- // Fallback: lowest quality at the smallest attempted dimension.
- const blob = await canvasToBlob(canvas, "image/jpeg", 0.1);
- bitmap.close();
- return new File([blob], file.name, {
- type: "image/jpeg",
- lastModified: file.lastModified,
- });
+ try {
+ // Try progressively smaller sizes until the output fits.
+ for (
+ let sizeScale = 1;
+ sizeScale >= 0.25;
+ sizeScale = Math.round((sizeScale * 0.5) * 100) / 100
+ ) {
+ const w = Math.max(1, Math.round(baseWidth * sizeScale));
+ const h = Math.max(1, Math.round(baseHeight * sizeScale));
+ canvas.width = w;
+ canvas.height = h;
+ ctx.drawImage(bitmap, 0, 0, w, h);
+
+ for (let quality = 0.85; quality >= 0.15; quality -= 0.1) {
+ const blob = await canvasToBlob(canvas, "image/jpeg", quality);
+ if (blob.size <= COMPRESS_TARGET_BYTES) {
+ return new File([blob], file.name, {
+ type: "image/jpeg",
+ lastModified: file.lastModified,
+ });
+ }
+ }
+ }
+
+ // Fallback: lowest quality at the smallest attempted dimension.
+ const blob = await canvasToBlob(canvas, "image/jpeg", 0.1);
+ return new File([blob], file.name, {
+ type: "image/jpeg",
+ lastModified: file.lastModified,
+ });
+ } finally {
+ bitmap.close();
+ }
}🤖 Prompt for AI Agents |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ImageBitmap resource not released if canvasToBlob() throws an error during compression loop