Skip to content

feat: download native ChatGPT files into workspaces#88

Open
jplew wants to merge 10 commits into
Waishnav:mainfrom
jplew:feat/minimal-chatgpt-artifact-handoff
Open

feat: download native ChatGPT files into workspaces#88
jplew wants to merge 10 commits into
Waishnav:mainfrom
jplew:feat/minimal-chatgpt-artifact-handoff

Conversation

@jplew

@jplew jplew commented Jul 20, 2026

Copy link
Copy Markdown

Summary

Replace the artifact-service lifecycle with one one-shot handoff:

open_workspace
download_artifact({ file, workspaceId }) -> { path }

DevSpace validates the native MCP-host file reference, streams it directly into the selected workspace under .devspace/incoming/, chooses a normalized collision-free filename, and returns only the workspace-relative path.

Tool contract

  • Exactly one artifact-specific tool: download_artifact.
  • Inputs are only the native file value and an existing workspaceId.
  • The descriptor retains openai/fileParams: ["file"].
  • Output is only { path }.
  • Destination selection, conflict policy, expected hashes, artifact IDs, host paths, TTLs, pinning, stat/delete/copy, and upload lifecycle are not model-facing.

Implementation

  • Streams under DEVSPACE_ARTIFACT_MAX_FILE_BYTES while computing SHA-256.
  • Publishes into .devspace/incoming/ with deterministic collision suffixes.
  • Uses descriptor-anchored directory creation and publication so pathname replacement cannot redirect writes outside the workspace.
  • Chmods, fsyncs, and verifies through the still-open partial file descriptor; no path-based chmod or hash occurs after publication.
  • Performs bounded cleanup of owned stale partials only.
  • Preserves the trusted OpenAI native-file adapter, regional host restrictions, redirect validation, and log redaction.

Removed lifecycle

  • No persistent artifact object store or completed-object database.
  • No upload/chunk protocol, artifact IDs, quota ledger, TTL, pinning, background cleanup timer, or artifact root.
  • Fresh databases do not create artifact tables.

Migration behavior

Existing legacy artifact tables and bytes are intentionally left untouched. The new implementation does not read, mutate, or opportunistically delete prior user data; cleanup can be performed manually after review.

Verification

  • npm test
  • npm run typecheck
  • npm run build
  • git diff --check

No runtime deployment or service restart was performed as part of this source-only handoff.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds opt-in native artifact staging with secure local storage, OpenAI file adapters, workspace materialization, MCP integration, configuration and database support, cleanup, redacted logging, tests, and documentation.

Changes

Native artifact staging

Layer / File(s) Summary
Configuration and persistence contracts
.env.example, src/config.ts, src/user-config.ts, src/db/*, src/config.test.ts, src/oauth-store.test.ts
Adds artifact enablement, storage limits, TTL settings, persisted configuration fields, SQLite tables, migration version 4, and validation coverage.
Secure artifact storage lifecycle
src/artifacts.ts, src/artifacts.test.ts
Implements resumable chunk uploads, hashing, quotas, content-addressed objects, ownership, deletion, expiration cleanup, health reporting, and filesystem containment checks.
Incoming artifact adapter validation
src/incoming-artifacts.ts, src/incoming-artifacts.test.ts
Adds adapter selection, OpenAI file-reference validation, restricted HTTPS downloads, redirect checks, normalized filenames, stream validation, and redaction tests.
Secure workspace materialization
src/artifact-workspace.ts, src/artifact-workspace.test.ts
Adds integrity-verified artifact copying into contained workspace paths with error, rename, and replace conflict modes.
MCP tools and server lifecycle
src/artifact-tools.ts, src/server.ts, package.json
Registers materialize_artifact, streams incoming files into storage, adds structured responses and logging, wires cleanup and shutdown, and includes the new tests.
Configuration and security documentation
README.md, docs/artifact-exchange.md, docs/configuration.md, docs/security.md
Documents the staging contract, supported connector fields, storage lifecycle, workspace copying, security boundaries, logging exclusions, and configuration entry points.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant MCPServer
  participant IncomingArtifactAdapterRegistry
  participant ArtifactStore
  participant Workspace
  Client->>MCPServer: materialize_artifact(file, workspaceId, destination, onConflict)
  MCPServer->>IncomingArtifactAdapterRegistry: validate and open file reference
  IncomingArtifactAdapterRegistry-->>MCPServer: readable artifact stream
  MCPServer->>ArtifactStore: upload and commit artifact
  ArtifactStore-->>MCPServer: verified artifact metadata
  MCPServer->>Workspace: copy artifact with conflict policy
  Workspace-->>MCPServer: workspace path and integrity metadata
  MCPServer-->>Client: structured materialization result
Loading

Possibly related PRs

  • Waishnav/devspace#78: Updates the same database migration and schema machinery with overlapping version 4 migration infrastructure.

Poem

A bunny carries bytes through a burrow so bright,
Hashes guard each parcel from morning to night.
Safe tools stage files, then tidy their stay,
While workspaces receive only paths the right way.
Hop, hop—secure artifacts are on their way!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 1.05% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main feature area: native ChatGPT file staging into private storage, even though the PR also adds workspace materialization.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/db/schema.ts (1)

97-98: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Align Drizzle index definitions with the migration's DESC ordering.

migrateArtifactExchange creates artifacts_workspace_idx/artifacts_client_idx on last_used_at desc and artifact_uploads_client_idx on created_at desc, but these schema definitions omit the descending order. Since the migration is the source of truth, this drift is harmless today, but it misrepresents the applied indexes and would be lost if migrations are ever regenerated from the schema. Add .desc() to match.

♻️ Proposed alignment
-    index("artifacts_workspace_idx").on(table.workspaceId, table.lastUsedAt),
-    index("artifacts_client_idx").on(table.clientId, table.lastUsedAt),
+    index("artifacts_workspace_idx").on(table.workspaceId, table.lastUsedAt.desc()),
+    index("artifacts_client_idx").on(table.clientId, table.lastUsedAt.desc()),
-    index("artifact_uploads_client_idx").on(table.clientId, table.createdAt),
+    index("artifact_uploads_client_idx").on(table.clientId, table.createdAt.desc()),

Also applies to: 124-124

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/db/schema.ts` around lines 97 - 98, Update the index definitions for
artifacts_workspace_idx and artifacts_client_idx to apply descending order to
lastUsedAt, and update artifact_uploads_client_idx similarly for createdAt.
Match the DESC ordering used by migrateArtifactExchange so the schema accurately
represents the migration-defined indexes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.env.example:
- Line 32: Update the commented DEVSPACE_ARTIFACT_ROOT example in .env.example
to use the generic ~/.local/share/devspace/artifacts path, matching the
documented default and removing the personal username.

In `@src/config.ts`:
- Around line 258-262: Update the artifactDefaultTtlHours configuration parsing
to enforce MAX_TTL_HOURS as its upper bound, while preserving the existing
positive-integer validation and default behavior. Use the existing MAX_TTL_HOURS
symbol rather than introducing a duplicate limit, so uploads using the
configured default remain valid.

---

Nitpick comments:
In `@src/db/schema.ts`:
- Around line 97-98: Update the index definitions for artifacts_workspace_idx
and artifacts_client_idx to apply descending order to lastUsedAt, and update
artifact_uploads_client_idx similarly for createdAt. Match the DESC ordering
used by migrateArtifactExchange so the schema accurately represents the
migration-defined indexes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d8458deb-e1ad-48b0-a198-c2656551e358

📥 Commits

Reviewing files that changed from the base of the PR and between 80423b5 and a162350.

📒 Files selected for processing (18)
  • .env.example
  • README.md
  • docs/artifact-exchange.md
  • docs/configuration.md
  • docs/security.md
  • package.json
  • src/artifact-tools.ts
  • src/artifacts.test.ts
  • src/artifacts.ts
  • src/config.test.ts
  • src/config.ts
  • src/db/migrations.ts
  • src/db/schema.ts
  • src/incoming-artifacts.test.ts
  • src/incoming-artifacts.ts
  • src/oauth-store.test.ts
  • src/server.ts
  • src/user-config.ts

Comment thread .env.example Outdated
Comment thread src/config.ts Outdated
@jplew
jplew force-pushed the feat/minimal-chatgpt-artifact-handoff branch from a162350 to bb1d3e3 Compare July 20, 2026 04:55
@jplew

jplew commented Jul 20, 2026

Copy link
Copy Markdown
Author

Addressed the initial review findings in a single amended commit:

  • replaced the personal artifact-root example with $HOME/...
  • centralized and enforced the maximum configured artifact TTL, with an upper-bound regression test
  • aligned Drizzle index definitions with the migration's descending order using desc(...)

Re-ran npm test, npm run typecheck, npm run build, and git diff --check upstream/main...HEAD; all passed.

@jplew
jplew marked this pull request as draft July 20, 2026 08:06
@jplew
jplew marked this pull request as ready for review July 20, 2026 11:22

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/artifact-workspace.test.ts (1)

1-59: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add negative-path coverage for this security-sensitive module.

Only the "error" mode happy path is tested. Given copyArtifactToWorkspace is responsible for path-containment, symlink rejection, and multiple conflict-resolution modes, consider adding cases for: destination outside the workspace root, symlinked destination/parent, pre-existing destination under onConflict: "error", and "rename"/"replace" behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/artifact-workspace.test.ts` around lines 1 - 59, Extend the test coverage
around copyArtifactToWorkspace with negative and conflict-mode cases: reject
destinations outside workspaceRoot, reject symlinked destinations and parents,
and preserve the existing destination when onConflict is "error". Add assertions
that "rename" creates a distinct destination and "replace" overwrites the
existing file, including returned path/renamed metadata where applicable.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@package.json`:
- Line 31: Update the “test” script in package.json to include
src/local-agent-targets.test.ts in the chained tsx test commands, preserving the
existing ordering and execution behavior of the other suites.

In `@src/artifact-workspace.ts`:
- Around line 48-104: Update the failure cleanup in the artifact materialization
flow around copyVerifiedArtifactToTemp and the surrounding try/catch so that,
once the artifact has been materialized, failures from chmod,
assertContainedWorkspaceRegularFile, or post-move integrity verification also
remove finalPath. Preserve cleanup for the temporary path, avoid deleting
unrelated pre-existing destinations, and ensure replace mode does not leave
failed content behind; consider deferring its destructive rename until
verification can succeed if needed.

---

Nitpick comments:
In `@src/artifact-workspace.test.ts`:
- Around line 1-59: Extend the test coverage around copyArtifactToWorkspace with
negative and conflict-mode cases: reject destinations outside workspaceRoot,
reject symlinked destinations and parents, and preserve the existing destination
when onConflict is "error". Add assertions that "rename" creates a distinct
destination and "replace" overwrites the existing file, including returned
path/renamed metadata where applicable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b152289b-b5d8-488a-a3b6-f0b9a9a3e97d

📥 Commits

Reviewing files that changed from the base of the PR and between bb1d3e3 and 172b71b.

📒 Files selected for processing (10)
  • docs/artifact-exchange.md
  • docs/configuration.md
  • docs/security.md
  • package.json
  • src/artifact-tools.ts
  • src/artifact-workspace.test.ts
  • src/artifact-workspace.ts
  • src/incoming-artifacts.test.ts
  • src/incoming-artifacts.ts
  • src/server.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • docs/security.md
  • docs/configuration.md
  • src/server.ts
  • src/incoming-artifacts.test.ts
  • src/incoming-artifacts.ts

Comment thread package.json Outdated
Comment thread src/artifact-workspace.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/artifact-workspace.test.ts (1)

9-145: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Solid coverage matching the copyArtifactToWorkspace contract. Error/rename/replace conflict handling, path escape, and symlink-based unsafe parent/destination cases all correctly assert the documented error codes (workspace_destination_exists, workspace_path_escape, workspace_parent_unsafe, workspace_destination_unsafe).

Two edge cases from the contract remain untested: workspace_destination_invalid (destination equal to workspaceRoot) and workspace_rename_exhausted (all rename attempts collide). Not blocking, but would round out coverage of the security boundary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/artifact-workspace.test.ts` around lines 9 - 145, Extend the
copyArtifactToWorkspace test coverage with a destination equal to workspaceRoot
and assert workspace_destination_invalid. Add a rename-conflict case that
pre-creates every candidate rename path until the configured limit and assert
workspace_rename_exhausted, while preserving the existing artifact and cleanup
setup.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/artifact-workspace.test.ts`:
- Around line 9-145: Extend the copyArtifactToWorkspace test coverage with a
destination equal to workspaceRoot and assert workspace_destination_invalid. Add
a rename-conflict case that pre-creates every candidate rename path until the
configured limit and assert workspace_rename_exhausted, while preserving the
existing artifact and cleanup setup.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: df2aa18b-b755-4e9a-8dab-d05d74534527

📥 Commits

Reviewing files that changed from the base of the PR and between 172b71b and 5edc5a4.

📒 Files selected for processing (3)
  • package.json
  • src/artifact-workspace.test.ts
  • src/artifact-workspace.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/artifact-workspace.ts

@Waishnav

Copy link
Copy Markdown
Owner

@jplew Really cool feature and PR

I'ven't gone through whole changes yet but these are my first impressions

  1. why so many tool call expose to model? isn't there should be just single tool call like download_artifact? like life-cycle around around artifact files are even needed at all, if those files can be moved/delete/renamed using other tools like bash or exec_command
  2. can't we handle this on tooling level only like by default generated artifact will be auto-downloaded inside ~/.local/share/devspace/artifact/ i.e no need to expose this to model

philosophy of devspace usually is to keep things very minimally expose to model as much as possible and handle at tooling level if it's possible

will test it by tomorrow but yeah pretty cool idea which i never thought could be possible (it seems like chatgpt apps sdk are pretty powerful)

@jplew

jplew commented Jul 21, 2026

Copy link
Copy Markdown
Author

philosophy of devspace usually is to keep things very minimally expose to model as much as possible and handle at tooling level if it's possible

Thanks for reviewing, all good points. I can definitely abstract away internals and consolidate everything into a single call. Will post a follow-up soon.

@jplew

jplew commented Jul 21, 2026

Copy link
Copy Markdown
Author

done, condensed everything into a single call.
image

@Waishnav Waishnav left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for consolidating this into one model-facing call — that direction matches DevSpace well. I ran the full test suite, typecheck, build, and git diff --check; they pass.

Before merging, I think this should be trimmed to a one-shot handoff rather than a persistent artifact service:

download_artifact({ file, workspaceId }) -> { path }

DevSpace can choose a collision-free path under .devspace/incoming/; existing filesystem tools can move/rename/delete it. This would let us remove model-facing destination, onConflict, and expectedSha256, plus the internal upload/chunk/TTL/pinning/quota/database lifecycle.

There are also four filesystem-safety blockers in the current implementation:

  1. ensureSecureDirectory() creates/chmods before proving containment, so an intermediate symlink can cause changes outside the artifact root.
  2. Workspace materialization performs path-based chmod/hash/cleanup after rename, leaving a TOCTOU window where the destination can be swapped.
  3. Artifact deletion removes the DB row before successfully removing the object; failures leave permanent untracked bytes and incorrect quota/cleanup state.
  4. Startup unconditionally chmods an existing configured artifact root to 0700, which can silently break shared or service-managed directories.

Please simplify the implementation and address these before merge. The native-file adapter, streaming validation, size enforcement, safe filename handling, and tests are all valuable and can remain in the smaller design.

@jplew jplew changed the title feat: stage native ChatGPT files into private storage feat: download native ChatGPT files into workspaces Jul 21, 2026
@jplew

jplew commented Jul 21, 2026

Copy link
Copy Markdown
Author

@Waishnav I made the requested change. Test looks like this so it appears to be working:
image

Last message in attached convo is an easy move.

@Waishnav

Copy link
Copy Markdown
Owner

@jplew heyo can you retest real quick, i've added back the path param and removed that .devspace/incoming/ as default download folder i.e letting model decide where it wants to download it, just want to make sure everything is all right with my recent changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants