Support custom base URLs for self-hosted AI providers (vLLM, self-hosted Whisper, etc.)

[jask87]

Cap's self-hosting docs currently support AI features only via fixed cloud providers — DEEPGRAM_API_KEY for transcription, and OPENAI_API_KEY / GROQ_API_KEY / ANTHROPIC_API_KEY for summaries. For teams self-hosting Cap specifically for data sovereignty, having AI calls still go out to third-party clouds undercuts the point.
Request: Add optional base-URL overrides so these features can point at self-hosted infrastructure instead:

OPENAI_BASE_URL (or similar) so the summary/title/chapter generation can point at an OpenAI-compatible endpoint — e.g. vLLM, which exposes a drop-in-compatible /v1/chat/completions API. This should be a small change since the OpenAI SDK already supports a baseURL param.
A similar override for transcription — either a DEEPGRAM_BASE_URL (Deepgram supports self-hosted deployments too), or more flexibly, support for any OpenAI-compatible /v1/audio/transcriptions endpoint, which self-hosted Whisper servers (including vLLM's own transcription API) already expose.

This would let self-hosters keep all AI processing entirely on their own infrastructure, matching the same self-hosting story that already applies to storage (S3-compatible) and the media server.

[linear]

CAP-755

[tembo]

You're right that this should be a relatively small change for the OpenAI-compatible endpoints. Here's what I found in the codebase:

For AI summaries/title/chapter generation:

The OpenAI calls are made via raw fetch() in apps/web/workflows/generate-ai.ts (see callOpenAi function around line 432-449). It currently hardcodes https://api.openai.com/v1/chat/completions. The Groq client in apps/web/lib/groq-client.ts uses the groq-sdk which also supports a baseURL constructor option.

There's also apps/web/lib/messenger/agent.ts which makes direct OpenAI calls for the chat feature (line 592-620).

To implement OPENAI_BASE_URL, you'd need to:

1. Add the env var to packages/env/server.ts (where all other AI keys are defined, around line 89-92)
2. Update callOpenAi() in generate-ai.ts to use it
3. Update the messenger agent's OpenAI call similarly

For transcription:

This is more involved. Transcription uses the Deepgram SDK (@deepgram/sdk) in apps/web/workflows/transcribe.ts (line 342-355). The SDK's createClient() doesn't expose a simple baseURL override like OpenAI's does.

Two paths forward:

• DEEPGRAM_BASE_URL: Would need changes to the Deepgram client initialization. Deepgram's self-hosted deployments use the same API format.
• OpenAI-compatible transcription endpoint: Would require a more significant refactor to support an alternative transcription provider alongside Deepgram. The transcription flow extracts audio to MP3 first (extractAudio step), then sends it to Deepgram.

For reference, the transcription options are configured in getDeepgramTranscriptionOptions() (line 304-325) and use the nova-3 model with smart formatting and utterance detection.