feat(ai): stream AI suggestions over HTTP - #664
Conversation
2265cba to
6bf913f
Compare
2ee7bd0 to
497be8b
Compare
497be8b to
9ff73a0
Compare
9ff73a0 to
8f8ee4a
Compare
8f8ee4a to
bd3667c
Compare
| }); | ||
| } | ||
|
|
||
| const res: any = new Writable({ |
There was a problem hiding this comment.
probably could be typed, since you assign it right away
There was a problem hiding this comment.
Added FakeResponse for this.
| const response = result.toUIMessageStreamResponse(); | ||
|
|
||
| res.status(response.status); | ||
| response.headers.forEach((value, key) => res.setHeader(key, value)); | ||
|
|
||
| if (!response.body) { | ||
| res.end(); | ||
|
|
||
| return; | ||
| } | ||
|
|
||
| Readable.fromWeb(response.body as NodeReadableStream<Uint8Array>).pipe(res); | ||
| } catch (error) { | ||
| next(error); | ||
| } | ||
| }); |
There was a problem hiding this comment.
we can just use
result.pipeUIMessageStreamToResponse(res);with extra options or just
result.pipeTextStreamToResponse(res);since we don't need any metadata toolcalls etc
There was a problem hiding this comment.
Good suggestion. And since I don't see any possibilities that ask-ai will need text/event-stream format, I'm using pipeTextStreamToResponse here.
@FeironoX5 If you're still working on stream receiving in hawk.garage, could you agree/disagree with this?
An Ask AI suggestion takes tens of seconds to generate, and the GraphQL resolver can only return it once the model has finished. Suggestions are now also available as a stream over a plain Express route, guarded by the same workspace membership check as the resolver and rejecting requests missing the project, event or repetition id before reaching the model.
bd3667c to
97fa6c2
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## fix/ai-prompt-injection #664 +/- ##
==========================================================
Coverage ? 47.98%
==========================================================
Files ? 59
Lines ? 2684
Branches ? 569
==========================================================
Hits ? 1288
Misses ? 1318
Partials ? 78 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
createFakeResponse assigned every Express-shaped method it needed right after construction, so it could be typed as that shape from the start instead of any - reviewer feedback on #664. Also adds writeHead, which the AI SDK's response-piping helpers call directly, bypassing Express's status()/setHeader() convenience methods.
routes.ts landed in integrations/vercel-ai/ in the original commit, even though it only calls askAiService and never touches the transport - the same domain-code-in-an-adapter-directory problem services/ai.ts itself had before it moved into askAi/. Wire its imports to the new location and expose it through the askAi barrel, alongside AskAiService. Also switches result.toUIMessageStreamResponse() + manual Response-to-Express bridging for result.pipeTextStreamToResponse(res) - reviewer feedback on #664. The model call is tool-less by design (see VercelAIApi's docstring), so there's no tool-call/reasoning metadata to carry, and plain text drops the SSE envelope this otherwise never needed. Drops the now-unused ReadableStream/Response ESLint globals that only existed for the old SSE-based test fixture.
There was a problem hiding this comment.
Pull request overview
Adds an HTTP streaming endpoint for AI suggestions and wires it into the API, alongside extending the Vercel AI integration with a streaming call and updating tests/utilities to support streaming responses.
Changes:
- Introduces
GET /integration/ai/streamExpress route and app wiring for AI suggestion streaming. - Extends the Vercel AI integration with a
stream()method and adds service-levelstreamSuggestion(). - Adds/updates Jest tests and introduces a reusable Express request/response test helper that can capture streamed bodies/headers.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/services/askAiRoutes.test.ts | New tests covering auth/validation/error cases and streaming response behavior for /integration/ai/stream. |
| test/services/askAi.test.ts | Adds service-level coverage for streamSuggestion() behavior. |
| test/integrations/vercel-ai.test.ts | Adds integration-level coverage for vercelAIApi.stream() forwarding to streamText. |
| test/integrations/github-routes.test.ts | Refactors tests to reuse the new makeExpressRequest helper. |
| test/helpers/expressRequest.ts | New helper to drive Express apps without a socket and capture streamed responses/headers. |
| src/services/types.ts | Exports Event type for reuse by services. |
| src/services/askAi/service.ts | Adds streamSuggestion() and refactors event lookup into getEventOrThrow(). |
| src/services/askAi/routes.ts | New Express router for AI streaming endpoint and authorization checks. |
| src/services/askAi/index.ts | Exports appendAiAssistantRoutes for app integration. |
| src/integrations/vercel-ai/index.ts | Adds stream() wrapper around streamText and centralizes provider gateway options. |
| src/index.ts | Registers AI assistant routes on the main Express app. |
| src/directives/requireUserInWorkspace.ts | Exports checkUserInWorkspaceByProjectId for use from Express routes. |
| package.json | Bumps package version. |
Suppressed comments (2)
src/services/askAi/routes.ts:87
- The inner
catchconverts anystreamSuggestionerror into a 404 and returnserror.messageto the caller. That will misreport transport/DB failures as "not found" and can leak internal error details (e.g. events factory errors that include ids). Only map the known not-found case to 404; rethrow unexpected errors so the outer handler cannext(error)and return a 5xx.
try {
result = await askAiService.streamSuggestion(eventsFactory, eventId, originalEventId);
} catch (error) {
res.status(404).json({ error: error instanceof Error ? error.message : 'Event not found' });
src/services/askAi/routes.ts:91
- This route calls
result.pipeTextStreamToResponse(res), but the PR description says the AI SDK'stoUIMessageStreamResponse()(a Fetch APIResponse) is adapted onto the Express response. As written, there is no adaptation and the call isn’t type-checked (becauseresultis implicitlyany), so a wrong method name or incompatible stream type would only fail at runtime. Consider explicitly usingtoUIMessageStreamResponse()and piping its status/headers/body into Express.
result.pipeTextStreamToResponse(res);
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov flagged the 3-arg (statusCode, statusMessage, headers) form as uncovered - nothing in this codebase calls writeHead with a status message, only (statusCode, headers). Narrowing to the one shape actually used instead of adding a test just to exercise dead code.
Codecov flagged routes.ts's patch coverage - the gaps predate this branch (they were already unexercised in the original commit), but this PR is what ships them, so closing them here rather than filing it as someone else's problem. Covers: the non-Error fallback message in both catch blocks, a missing request context, an unexpected synchronous throw reaching Express's error handling, and appendAiAssistantRoutes itself (tests only exercised createAiStreamRouter mounted by hand). routes.ts is now at 100% statement/branch/line coverage.
Copilot review on #664: projectId comes from req.query, which Express parses as string[] for a repeated key (?projectId=a&projectId=b). The route cast it straight to string and forwarded it to checkUserInWorkspaceByProjectId/getEventsFactory, both expecting a single id - eventId and originalEventId already had the typeof guard this was missing. authorizeProjectAccess now validates and returns the narrowed id instead of the caller re-casting it. makeExpressRequest's query param takes string | string[] now, to let tests simulate a repeated key.
Copilot review on #664: getEventOrThrow only handled a falsy return from getEventRepetition, but it can also throw - EventsFactory throws "Cant find event repetition for repetitionId: ..." on an unmatched id, echoing the raw id back, and an invalid id format throws a raw BSON error. Both reached the HTTP route's catch block unfiltered. Catches and normalizes to the same generic message as the missing-event case.
The stream route mapped any error from streamSuggestion to a 404 "Event not found", including failures unrelated to the event lookup (e.g. a stream construction error). Only the exact "Event not found" error is now reported as 404; anything else is forwarded to Express's error handling. Flagged by Copilot while reviewing #668, against code this PR added.
GET /integration/ai/streamExpress route.Responsereturned by the AI SDK'stoUIMessageStreamResponse()is adapted onto the Express response.ReadableStream/Responseas ESLint globals (.eslintrc.js) - valid Node 18+ runtime globals that predate ESLint's bundlednodeenv.