diff --git a/.env.example b/.env.example index 2b0f784e..921b21a3 100644 --- a/.env.example +++ b/.env.example @@ -23,7 +23,7 @@ PUBLIC_URL= # Obsidian Sync credentials # Generate OBSIDIAN_AUTH_TOKEN once with: -# docker run --rm -it --entrypoint get-token \ +# docker run --rm -it --entrypoint get-sync-token \ # ghcr.io/aliasunder/vault-cortex:remote OBSIDIAN_AUTH_TOKEN= VAULT_NAME= diff --git a/AGENTS.md b/AGENTS.md index 878b74f5..17ab7084 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -52,7 +52,7 @@ obsidian-headless/ # Lockfile-pinned obsidian-headless for D package-lock.json # sha512 integrity hashes (supply-chain security) rootfs/ # Container filesystem overlay (remote target) etc/s6-overlay/ # init chain + svc-obsidian-sync + svc-vault-mcp - usr/local/bin/get-token # interactive Obsidian Sync token helper + usr/local/bin/get-sync-token # interactive Obsidian Sync token helper (manual flow) docker-compose.yml # Lightsail: single vault-cortex:remote service docker-compose.local.yml # Contributor dev: builds from source .env.example # template for Lightsail .env @@ -85,6 +85,7 @@ cli/ # npx vault-cortex CLI (published as vaul scaffold.ts # File generation (.env) docker.ts # Container management (docker run, health-check wait) upgrade.ts # Upgrade command (pull + re-create + health check) + get-sync-token.ts # Get-sync-token subcommand (Sync token auto-capture via volume mount) env.ts # Environment file handling (.env generation) token.ts # Secure token generation (openssl rand) vault.ts # Vault path validation diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index c4816d94..f3dc1015 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -731,29 +731,29 @@ Vault Cortex runs anywhere Docker does — the reference deployment uses Lightsa ## Key Decisions -| Decision | Rationale | -| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Lightsail over ECS | $12–24 vs ~$50+. Single-user server. | -| API Gateway over Caddy | Free HTTPS URL without a custom domain, SST native, and a Lambda authorizer for path-aware auth (OAuth endpoints pass through, `/mcp` validates). Tradeoff: 10-minute idle timeout on HTTP connections can cause `Connection closed` on first call after idle. | -| Obsidian Sync over git-based sync | Bidirectional real-time sync to all devices, automatic conflict resolution, no manual push/pull. Tradeoff: dependency on Obsidian's proprietary cloud service. | -| Single image over a separate sync container | The `:remote` target absorbs the [obsidian-headless-sync-docker fork](https://github.com/aliasunder/obsidian-headless-sync-docker)'s s6 scaffolding (`rootfs/`, including the `get-token` helper) and installs `obsidian-headless` from a sha512-pinned lockfile via `npm ci` — the fork's Alpine base can't host `onnxruntime-node` (musl), so building `FROM` it was never an option. One image means one repo, one CI, one version, and no Compose requirement for users (`docker run`/Podman/nerdctl all work). s6-overlay's static binaries run on glibc, and `DEVICE_NAME`-aware initial registration carries over from the fork. In the `:remote` target the two processes have shared fate through `/vault` — the MCP server without sync serves a stale vault; sync without the server serves nothing — so a single supervised container is the semantically honest packaging, not a convenience bundle. The `local` target has no sync process and stays single-process under tini. | -| OAuth 2.1 + static token | OAuth 2.1 (PKCE) for browser-capable clients — automatic token refresh, no secret in config after consent. Static bearer token for CLI tools and scripts where a browser flow isn't practical. Both validated at two independent layers (Lambda + Express) using the same HMAC key. | -| Custom JWT over JWT libraries | 50-line HS256 implementation vs 200KB+ library bundle. Lambda authorizer stays tiny. Constant-time comparison prevents timing attacks. Acceptable for a single-algorithm use case. | -| JWT over opaque tokens | Verifiable at Lambda edge without shared state. HS256 with MCP_AUTH_TOKEN. | -| 60-day sliding refresh | Active clients never re-auth; leaked tokens bounded. Standard OAuth practice. | -| Auto-snapshot (`addOn`) | Native Lightsail primitive over hand-rolled cron + S3. Daily, 7-day retention, captures full boot disk including SSH-installed state. | -| Pulumi `protect` + `retainOnDelete` | IaC seatbelt over `replaceOnChanges` gymnastics. Intentional replaces require explicit unprotect — the friction is the feature. | -| Debian slim over Alpine | `onnxruntime-node` (bundled by `@huggingface/transformers` for local embeddings) requires glibc. Alpine uses musl — no musl build exists. Hard architectural constraint, not a preference. | -| SQLite FTS5 | Zero services, embedded, personal scale. | -| sqlite-vec over pgvector/Pinecone | Vectors live alongside FTS5 in the same SQLite database — loaded as an extension into the same connection (`sqliteVec.load(db)`), not a separate datastore or service. No network hop, no second process, no API key. Keeps the "zero services, embedded, personal scale" principle established by FTS5. | -| chokidar | Node-native, same process as SQLite. Embedding hook for vector index updates. | -| Streamable HTTP | Current MCP spec (2025-11-25). SSE is deprecated. | -| 405 on `GET /mcp` (no standalone stream) | The server never sends server-initiated messages, so the optional GET-opened SSE stream would only ever sit idle until an upstream proxy timeout kills it — surfacing as gateway 5xx noise in monitoring. The Streamable HTTP spec explicitly allows servers that don't offer the stream to reject the GET with 405 (`Allow: POST, DELETE`). Clients fall back cleanly; POST responses still stream per request. | -| GHCR over ECR | GITHUB_TOKEN auth, no AWS IAM for images. | -| Express 5 over Fastify/Hono | Ecosystem maturity, middleware compatibility. Express 5's native async error handling eliminated wrapper boilerplate. MCP SDK reference implementation uses Express. | -| Atomic writes + per-file mutex | MCP handlers are concurrent — two tools could write the same file. Write-to-tmp-then-rename prevents partial writes; `link()` no-clobber (`atomicWriteFileExclusive`) closes the TOCTOU race on moves. Per-file mutex prevents conflicting operations: fail-fast for intent-based writes (patch/replace), serializing for read-inside-lock writes (memory append). Multi-file locking (`withExclusiveMultiFileLock`) covers moves, which must read and write the moved note plus every backlink source as one unit. (→ [Data Integrity](#data-integrity)) | -| Factory over class | Functional style. Closure holds db ref, no `this`. | -| `type` over `interface` | Uniform syntax — `type` handles unions, intersections, tuples, mapped types, and object shapes; `interface` only handles objects, so you'd need both anyway. No accidental declaration merging (interfaces with the same name silently merge — a library augmentation feature that's a footgun in application code). Negligible performance difference in practice. | -| Hybrid search over LightRAG | 30% of natural-language queries fail on FTS-only (vocabulary mismatch), but vector-only loses precision on exact terms and technical jargon where keyword matching excels. Hybrid keeps both strengths. LightRAG requires a ≥32B LLM for entity extraction — far too heavy for a VPS — and the vault's wikilinks already encode a hand-authored knowledge graph. [qmd](https://github.com/tobi/qmd) demonstrated how lightweight hybrid search can be: FTS5 + sqlite-vec + RRF in a single SQLite file, all application-layer code. vault-cortex applies the same patterns with lighter ONNX models ([bge-small-en-v1.5](https://huggingface.co/Xenova/bge-small-en-v1.5) 33M/~25MB vs [qmd](https://github.com/tobi/qmd)'s ~2GB GGUF stack). Opt-out via `EMBEDDING_ENABLED=false` — no model download, no vector tables — and graceful FTS-only fallback when vectors aren't available. | -| RRF fusion (k=60) | Merges FTS keyword and vector similarity ranked lists by rank position, not score — BM25 scores and cosine distances are on incomparable scales, so any score-based combination would need normalization. Top-rank bonuses (+0.05 rank 1, +0.02 ranks 2–3) reward results that either system placed highly. Validated at 8/9 on the vocabulary-mismatch evaluation, ~8ms added latency. Inspired by [qmd](https://github.com/tobi/qmd). | -| Position-aware blending over full reranker | RRF alone scored 8/9 — it bridged vocabulary gaps but couldn't resolve intent-heavy queries ("how I feel about AI tools") where both keyword and vector signals miss. A cross-encoder reranker fills that gap by scoring each (query, document) pair jointly, but pure reranker sort also scored 8/9 — it fixed the intent queries but over-prioritized topical relevance, demoting structurally correct results (TASKS.md) on task-oriented queries. Position-aware blending combines both: top RRF hits are protected (75% retrieval weight for ranks 1–3) while the reranker rescues demoted results at lower ranks (60% reranker weight for ranks 11+). The only approach that scored 9/9, 0 regressions, ~200ms added latency. Opt-out via `RERANK_MODE=none`. | +| Decision | Rationale | +| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Lightsail over ECS | $12–24 vs ~$50+. Single-user server. | +| API Gateway over Caddy | Free HTTPS URL without a custom domain, SST native, and a Lambda authorizer for path-aware auth (OAuth endpoints pass through, `/mcp` validates). Tradeoff: 10-minute idle timeout on HTTP connections can cause `Connection closed` on first call after idle. | +| Obsidian Sync over git-based sync | Bidirectional real-time sync to all devices, automatic conflict resolution, no manual push/pull. Tradeoff: dependency on Obsidian's proprietary cloud service. | +| Single image over a separate sync container | The `:remote` target absorbs the [obsidian-headless-sync-docker fork](https://github.com/aliasunder/obsidian-headless-sync-docker)'s s6 scaffolding (`rootfs/`, including the interactive Sync-token helper, since rewritten and renamed `get-sync-token`) and installs `obsidian-headless` from a sha512-pinned lockfile via `npm ci` — the fork's Alpine base can't host `onnxruntime-node` (musl), so building `FROM` it was never an option. One image means one repo, one CI, one version, and no Compose requirement for users (`docker run`/Podman/nerdctl all work). s6-overlay's static binaries run on glibc, and `DEVICE_NAME`-aware initial registration carries over from the fork. In the `:remote` target the two processes have shared fate through `/vault` — the MCP server without sync serves a stale vault; sync without the server serves nothing — so a single supervised container is the semantically honest packaging, not a convenience bundle. The `local` target has no sync process and stays single-process under tini. | +| OAuth 2.1 + static token | OAuth 2.1 (PKCE) for browser-capable clients — automatic token refresh, no secret in config after consent. Static bearer token for CLI tools and scripts where a browser flow isn't practical. Both validated at two independent layers (Lambda + Express) using the same HMAC key. | +| Custom JWT over JWT libraries | 50-line HS256 implementation vs 200KB+ library bundle. Lambda authorizer stays tiny. Constant-time comparison prevents timing attacks. Acceptable for a single-algorithm use case. | +| JWT over opaque tokens | Verifiable at Lambda edge without shared state. HS256 with MCP_AUTH_TOKEN. | +| 60-day sliding refresh | Active clients never re-auth; leaked tokens bounded. Standard OAuth practice. | +| Auto-snapshot (`addOn`) | Native Lightsail primitive over hand-rolled cron + S3. Daily, 7-day retention, captures full boot disk including SSH-installed state. | +| Pulumi `protect` + `retainOnDelete` | IaC seatbelt over `replaceOnChanges` gymnastics. Intentional replaces require explicit unprotect — the friction is the feature. | +| Debian slim over Alpine | `onnxruntime-node` (bundled by `@huggingface/transformers` for local embeddings) requires glibc. Alpine uses musl — no musl build exists. Hard architectural constraint, not a preference. | +| SQLite FTS5 | Zero services, embedded, personal scale. | +| sqlite-vec over pgvector/Pinecone | Vectors live alongside FTS5 in the same SQLite database — loaded as an extension into the same connection (`sqliteVec.load(db)`), not a separate datastore or service. No network hop, no second process, no API key. Keeps the "zero services, embedded, personal scale" principle established by FTS5. | +| chokidar | Node-native, same process as SQLite. Embedding hook for vector index updates. | +| Streamable HTTP | Current MCP spec (2025-11-25). SSE is deprecated. | +| 405 on `GET /mcp` (no standalone stream) | The server never sends server-initiated messages, so the optional GET-opened SSE stream would only ever sit idle until an upstream proxy timeout kills it — surfacing as gateway 5xx noise in monitoring. The Streamable HTTP spec explicitly allows servers that don't offer the stream to reject the GET with 405 (`Allow: POST, DELETE`). Clients fall back cleanly; POST responses still stream per request. | +| GHCR over ECR | GITHUB_TOKEN auth, no AWS IAM for images. | +| Express 5 over Fastify/Hono | Ecosystem maturity, middleware compatibility. Express 5's native async error handling eliminated wrapper boilerplate. MCP SDK reference implementation uses Express. | +| Atomic writes + per-file mutex | MCP handlers are concurrent — two tools could write the same file. Write-to-tmp-then-rename prevents partial writes; `link()` no-clobber (`atomicWriteFileExclusive`) closes the TOCTOU race on moves. Per-file mutex prevents conflicting operations: fail-fast for intent-based writes (patch/replace), serializing for read-inside-lock writes (memory append). Multi-file locking (`withExclusiveMultiFileLock`) covers moves, which must read and write the moved note plus every backlink source as one unit. (→ [Data Integrity](#data-integrity)) | +| Factory over class | Functional style. Closure holds db ref, no `this`. | +| `type` over `interface` | Uniform syntax — `type` handles unions, intersections, tuples, mapped types, and object shapes; `interface` only handles objects, so you'd need both anyway. No accidental declaration merging (interfaces with the same name silently merge — a library augmentation feature that's a footgun in application code). Negligible performance difference in practice. | +| Hybrid search over LightRAG | 30% of natural-language queries fail on FTS-only (vocabulary mismatch), but vector-only loses precision on exact terms and technical jargon where keyword matching excels. Hybrid keeps both strengths. LightRAG requires a ≥32B LLM for entity extraction — far too heavy for a VPS — and the vault's wikilinks already encode a hand-authored knowledge graph. [qmd](https://github.com/tobi/qmd) demonstrated how lightweight hybrid search can be: FTS5 + sqlite-vec + RRF in a single SQLite file, all application-layer code. vault-cortex applies the same patterns with lighter ONNX models ([bge-small-en-v1.5](https://huggingface.co/Xenova/bge-small-en-v1.5) 33M/~25MB vs [qmd](https://github.com/tobi/qmd)'s ~2GB GGUF stack). Opt-out via `EMBEDDING_ENABLED=false` — no model download, no vector tables — and graceful FTS-only fallback when vectors aren't available. | +| RRF fusion (k=60) | Merges FTS keyword and vector similarity ranked lists by rank position, not score — BM25 scores and cosine distances are on incomparable scales, so any score-based combination would need normalization. Top-rank bonuses (+0.05 rank 1, +0.02 ranks 2–3) reward results that either system placed highly. Validated at 8/9 on the vocabulary-mismatch evaluation, ~8ms added latency. Inspired by [qmd](https://github.com/tobi/qmd). | +| Position-aware blending over full reranker | RRF alone scored 8/9 — it bridged vocabulary gaps but couldn't resolve intent-heavy queries ("how I feel about AI tools") where both keyword and vector signals miss. A cross-encoder reranker fills that gap by scoring each (query, document) pair jointly, but pure reranker sort also scored 8/9 — it fixed the intent queries but over-prioritized topical relevance, demoting structurally correct results (TASKS.md) on task-oriented queries. Position-aware blending combines both: top RRF hits are protected (75% retrieval weight for ranks 1–3) while the reranker rescues demoted results at lower ranks (60% reranker weight for ranks 11+). The only approach that scored 9/9, 0 regressions, ~200ms added latency. Opt-out via `RERANK_MODE=none`. | diff --git a/DEPLOY.md b/DEPLOY.md index 41225fd4..9497c909 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -62,7 +62,13 @@ Then open `~/.config/vault-cortex/.env` and fill in the remaining values: The [`.env.example`](./.env.example) file also includes optional configuration for the embedding pipeline (`EMBEDDING_ENABLED`), the reranker (`RERANK_MODE`), the memory system (`MEMORY_ENABLED`, `MEMORY_DIR`, `PROTECTED_PATHS`, `ORPHAN_EXCLUDE_FOLDERS`), timezone (`TZ`), and OAuth metadata (`SERVICE_DOCUMENTATION_URL`). All have sensible defaults — see the [Configuration](./README.md#configuration) section in the README. ```bash -docker run --rm -it --entrypoint get-token ghcr.io/aliasunder/vault-cortex:remote +npx vault-cortex get-sync-token +``` + +Or run the Docker image directly: + +```bash +docker run --rm -it --entrypoint get-sync-token ghcr.io/aliasunder/vault-cortex:remote ``` **4. Authenticate to GHCR** (once per machine): @@ -280,7 +286,7 @@ To find your stage: `cat .sst/stage` (after your first deploy). | `GHCR_TOKEN` | Personal access token (classic) with `write:packages` + `read:packages`. Used by `docker login` both at build-push and on-instance pull. Persists across runs; rotate when stale. | | `DOCKERHUB_TOKEN` | Optional. Docker Hub access token with `Read & Write` repository permissions. Used by deploy (image push) and dockerhub-description (DOCKERHUB.md sync). Only needed when `DOCKERHUB_USERNAME` is set. | | `MCP_AUTH_TOKEN` | Same value as the SST secret of the same name. Written into the instance `.env` for the Express auth layer. | -| `OBSIDIAN_AUTH_TOKEN` | Output of `docker run --rm -it --entrypoint get-token ghcr.io/aliasunder/vault-cortex:remote`. | +| `OBSIDIAN_AUTH_TOKEN` | Output of `npx vault-cortex get-sync-token` — see [One-time setup](#one-time-setup). | | `VAULT_PASSWORD` | Optional. Only set if your vault uses end-to-end encryption. Empty value is fine and ships through to `.env` as `VAULT_PASSWORD=`. | | `SSH_PUBKEY` | Public key contents of your `~/.ssh/vault-cortex.pub` (literal, single line). Same key local dev and CI use — see [Prerequisites](#prerequisites). | | `SSH_PRIVATE_KEY` | Private half (`~/.ssh/vault-cortex`, full multi-line block including BEGIN/END markers). Loaded by `webfactory/ssh-agent` for SCP/SSH to the instance. | diff --git a/Dockerfile b/Dockerfile index 1e545de0..6637a7f0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -74,7 +74,7 @@ EXPOSE 8000 # --------------------------------------------------------------------------- # remote: s6-overlay supervises obsidian-sync + vault-mcp in one container. # Absorbed from aliasunder/obsidian-headless-sync-docker (rootfs/, including -# the get-token helper), adapted from Alpine to Debian. +# the interactive token helper, since rewritten as get-sync-token), adapted from Alpine to Debian. # --------------------------------------------------------------------------- FROM base AS remote @@ -140,9 +140,9 @@ RUN userdel -r node \ && chown -R obsidian:obsidian /vault /home/obsidian # s6 service definitions (init chain + svc-obsidian-sync + svc-vault-mcp) -# and the interactive get-token helper (rootfs/usr/local/bin/get-token). +# and the interactive get-sync-token helper (rootfs/usr/local/bin/get-sync-token). COPY rootfs/ / -RUN chmod +x /usr/local/bin/get-token \ +RUN chmod +x /usr/local/bin/get-sync-token \ && chmod +x /etc/s6-overlay/scripts/* \ && chmod +x /etc/s6-overlay/s6-rc.d/svc-obsidian-sync/run \ /etc/s6-overlay/s6-rc.d/svc-vault-mcp/run diff --git a/cli/README.md b/cli/README.md index 29c00f70..d046fe49 100644 --- a/cli/README.md +++ b/cli/README.md @@ -28,6 +28,26 @@ container; this CLI scaffolds the config so you don't have to. Existing files are never overwritten without asking. +## Get Sync Token + +Generate an Obsidian Sync auth token without leaving the CLI: + +```bash +npx vault-cortex get-sync-token +``` + +The command opens the Obsidian login inside Docker. Once you've signed +in, it captures your token and prints it — nothing to dig out of the +login output. Use `--dir` to write the token straight into an existing +`.env` instead: + +```bash +npx vault-cortex get-sync-token --dir ./vault-cortex +``` + +During `init --mode remote`, this flow is offered automatically when Docker +is available. + ## Upgrade Pull the latest image, re-create the container, and verify health: diff --git a/cli/src/__tests__/docker.test.ts b/cli/src/__tests__/docker.test.ts index c4cbef15..c2f6beba 100644 --- a/cli/src/__tests__/docker.test.ts +++ b/cli/src/__tests__/docker.test.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from "vitest" import { buildDockerRunArgs, + buildObsidianLoginArgs, CONTAINER_NAME, LOCAL_IMAGE, pollHealth, @@ -187,6 +188,71 @@ describe("buildDockerRunArgs", () => { }) }) +describe("buildObsidianLoginArgs", () => { + it("produces the correct args on macOS (no --user flag)", () => { + const args = buildObsidianLoginArgs({ + configMountPath: "/tmp/vault-cortex-sync-token-abc", + platform: "darwin", + uid: 501, + gid: 20, + }) + + expect(args).toEqual([ + "run", + "--rm", + "-it", + "--entrypoint", + "ob", + "-v", + "/tmp/vault-cortex-sync-token-abc:/home/obsidian/.config", + REMOTE_IMAGE, + "login", + ]) + }) + + it("includes --user uid:gid on Linux", () => { + const args = buildObsidianLoginArgs({ + configMountPath: "/tmp/vault-cortex-sync-token-abc", + platform: "linux", + uid: 1000, + gid: 1000, + }) + + expect(args).toEqual([ + "run", + "--rm", + "-it", + "--entrypoint", + "ob", + "-v", + "/tmp/vault-cortex-sync-token-abc:/home/obsidian/.config", + "--user", + "1000:1000", + REMOTE_IMAGE, + "login", + ]) + }) + + it("omits --user on Linux when uid/gid are not provided", () => { + const args = buildObsidianLoginArgs({ + configMountPath: "/tmp/test", + platform: "linux", + }) + + expect(args).toEqual([ + "run", + "--rm", + "-it", + "--entrypoint", + "ob", + "-v", + "/tmp/test:/home/obsidian/.config", + REMOTE_IMAGE, + "login", + ]) + }) +}) + describe("pollHealth", () => { it("returns true as soon as the endpoint responds ok", async () => { const fetchStub = async (): Promise => okResponse diff --git a/cli/src/__tests__/env.test.ts b/cli/src/__tests__/env.test.ts index 8f30678a..44dbab09 100644 --- a/cli/src/__tests__/env.test.ts +++ b/cli/src/__tests__/env.test.ts @@ -1,6 +1,5 @@ import { describe, expect, it } from "vitest" -import { REMOTE_IMAGE } from "../docker.js" import { buildLocalEnv, buildRemoteEnv } from "../env.js" describe("buildLocalEnv", () => { @@ -81,9 +80,7 @@ describe("buildRemoteEnv", () => { expect(env).toMatch(/^OBSIDIAN_AUTH_TOKEN=$/m) expect(env).toContain("FILL THIS IN") - // The guidance must name the actual get-token image, not just the - // subcommand — "get-token" alone would pass with a stale image name. - expect(env).toContain(`get-token \\\n# ${REMOTE_IMAGE}`) + expect(env).toContain("npx vault-cortex get-sync-token") }) it("states defaulted sync settings as uncommented lines", () => { diff --git a/cli/src/__tests__/get-sync-token.test.ts b/cli/src/__tests__/get-sync-token.test.ts new file mode 100644 index 00000000..d54ee8fd --- /dev/null +++ b/cli/src/__tests__/get-sync-token.test.ts @@ -0,0 +1,356 @@ +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + writeFileSync, +} from "node:fs" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { describe, expect, it } from "vitest" + +import { captureObsidianToken, runGetSyncToken } from "../get-sync-token.js" +import type { DockerRunner } from "../docker.js" +import type { Prompts } from "../prompts.js" + +/** + * Destination sentence passed to captureObsidianToken in direct-call tests — + * production callers supply their own flow-specific sentence (stored in + * .env / printed / written to a path). + */ +const TOKEN_DESTINATION_MESSAGE = "The token is captured automatically." + +const createSilentPrompts = () => { + const errors: string[] = [] + const warnings: string[] = [] + const logs: string[] = [] + const prints: string[] = [] + let introMessage = "" + let outroMessage = "" + + const prompts: Prompts = { + intro: (message) => { + introMessage = message ?? "" + }, + outro: (message) => { + outroMessage = message ?? "" + }, + note: () => {}, + print: (message) => { + prints.push(message) + }, + log: (message) => { + logs.push(message) + }, + warn: (message) => { + warnings.push(message) + }, + error: (message) => { + errors.push(message) + }, + select: async () => "", + text: async () => "", + password: async () => "", + confirm: async () => false, + spinner: () => ({ start: () => {}, stop: () => {} }), + } + + return { prompts, errors, warnings, logs, prints, introMessage, outroMessage } +} + +/** + * Creates a DockerRunner whose runObsidianLogin writes a fake + * auth_token file into the config mount path, simulating the + * containerized login writing the token file. + */ +const dockerWithToken = (token: string): DockerRunner => ({ + isDaemonRunning: () => true, + dockerRun: () => false, + pullImage: () => false, + stopAndRemoveContainer: () => false, + runObsidianLogin: (configMountPath) => { + const tokenDir = join(configMountPath, "obsidian-headless") + mkdirSync(tokenDir, { recursive: true }) + writeFileSync(join(tokenDir, "auth_token"), token) + return true + }, +}) + +const dockerFailsLogin: DockerRunner = { + isDaemonRunning: () => true, + dockerRun: () => false, + pullImage: () => false, + stopAndRemoveContainer: () => false, + runObsidianLogin: () => false, +} + +const dockerDown: DockerRunner = { + isDaemonRunning: () => false, + dockerRun: () => false, + pullImage: () => false, + stopAndRemoveContainer: () => false, + runObsidianLogin: () => false, +} + +describe("captureObsidianToken", () => { + it("returns the token when the login writes the auth_token file", () => { + const { prompts } = createSilentPrompts() + + const token = captureObsidianToken( + { + docker: dockerWithToken("abc123-sync-token"), + prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBe("abc123-sync-token") + }) + + it("trims whitespace from the token file", () => { + const { prompts } = createSilentPrompts() + + const token = captureObsidianToken( + { + docker: dockerWithToken(" token-with-whitespace \n"), + prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBe("token-with-whitespace") + }) + + it("returns undefined when docker run fails", () => { + const silent = createSilentPrompts() + + const token = captureObsidianToken( + { + docker: dockerFailsLogin, + prompts: silent.prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBeUndefined() + expect(silent.warnings[0]).toBe( + "The Obsidian login did not complete — you can run it later with:\n" + + " npx vault-cortex get-sync-token", + ) + }) + + it("returns undefined and warns when the token file is empty", () => { + const silent = createSilentPrompts() + + const token = captureObsidianToken( + { + docker: dockerWithToken(""), + prompts: silent.prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBeUndefined() + expect(silent.warnings[0]).toBe( + "The Obsidian login finished, but no token was captured — the " + + "token file was missing, empty, or unreadable. You can retry with:\n" + + " npx vault-cortex get-sync-token", + ) + }) + + it("returns undefined and warns when the login succeeds but writes no token file", () => { + const silent = createSilentPrompts() + const dockerSucceedsButNoFile: DockerRunner = { + ...dockerDown, + isDaemonRunning: () => true, + runObsidianLogin: () => true, + } + + const token = captureObsidianToken( + { + docker: dockerSucceedsButNoFile, + prompts: silent.prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBeUndefined() + expect(silent.warnings[0]).toBe( + "The Obsidian login finished, but no token was captured — the " + + "token file was missing, empty, or unreadable. You can retry with:\n" + + " npx vault-cortex get-sync-token", + ) + }) + + it("treats a docker runner throw as a failed run and returns undefined", () => { + const silent = createSilentPrompts() + const dockerThrows: DockerRunner = { + ...dockerDown, + isDaemonRunning: () => true, + runObsidianLogin: () => { + throw new Error("spawn docker ENOENT") + }, + } + + const token = captureObsidianToken( + { + docker: dockerThrows, + prompts: silent.prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(token).toBeUndefined() + expect(silent.warnings).toEqual([ + "Docker run failed — spawn docker ENOENT", + "The Obsidian login did not complete — you can run it later with:\n" + + " npx vault-cortex get-sync-token", + ]) + }) + + it("cleans up the temp directory even on failure", () => { + const { prompts } = createSilentPrompts() + const tempDirs: string[] = [] + const dockerTracker: DockerRunner = { + ...dockerDown, + isDaemonRunning: () => true, + runObsidianLogin: (configMountPath) => { + tempDirs.push(configMountPath) + return false + }, + } + + captureObsidianToken( + { docker: dockerTracker, prompts }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(tempDirs).toHaveLength(1) + expect(existsSync(tempDirs[0])).toBe(false) + }) + + it("logs the handoff message before running docker", () => { + const silent = createSilentPrompts() + + captureObsidianToken( + { + docker: dockerWithToken("token"), + prompts: silent.prompts, + }, + TOKEN_DESTINATION_MESSAGE, + ) + + expect(silent.logs[0]).toBe( + "Handing the terminal to the Obsidian login — it will ask for your " + + "account email, password, and MFA code. The token is captured automatically.", + ) + }) +}) + +describe("runGetSyncToken subcommand", () => { + it("prints the token to stdout when --dir is not set", async () => { + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + {}, + { prompts: silent.prompts, docker: dockerWithToken("my-sync-token") }, + ) + + expect(exitCode).toBe(0) + expect(silent.logs[0]).toBe( + "Handing the terminal to the Obsidian login — it will ask for your " + + "account email, password, and MFA code. The token is captured " + + "automatically and printed at the end.", + ) + expect(silent.logs).toContain("Your OBSIDIAN_AUTH_TOKEN:") + expect(silent.prints).toEqual(["\n my-sync-token\n"]) + }) + + it("exits 1 when the docker daemon is not running", async () => { + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + {}, + { prompts: silent.prompts, docker: dockerDown }, + ) + + expect(exitCode).toBe(1) + expect(silent.errors[0]).toBe( + "Container runtime not running — start Docker Desktop, Colima,\n" + + "OrbStack, or another Docker-compatible runtime and try again.", + ) + }) + + it("exits 1 when token capture fails", async () => { + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + {}, + { prompts: silent.prompts, docker: dockerFailsLogin }, + ) + + expect(exitCode).toBe(1) + expect(silent.errors[0]).toBe("Could not capture the auth token.") + }) + + it("writes the token to .env when --dir is set", async () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-sync-token-")) + writeFileSync( + join(targetDir, ".env"), + "MCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=old-token\nVAULT_NAME=MyVault\n", + ) + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + { dir: targetDir }, + { prompts: silent.prompts, docker: dockerWithToken("new-sync-token") }, + ) + + expect(exitCode).toBe(0) + expect(silent.logs[0]).toBe( + "Handing the terminal to the Obsidian login — it will ask for your " + + "account email, password, and MFA code. The token is captured " + + `automatically and written to ${join(targetDir, ".env")}.`, + ) + expect(readFileSync(join(targetDir, ".env"), "utf8")).toBe( + "MCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=new-sync-token\nVAULT_NAME=MyVault\n", + ) + expect(silent.logs).toContain(`Token written to ${join(targetDir, ".env")}`) + }) + + it("exits 1 when --dir .env has no OBSIDIAN_AUTH_TOKEN line", async () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-sync-token-")) + writeFileSync(join(targetDir, ".env"), "MCP_AUTH_TOKEN=abc\n") + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + { dir: targetDir }, + { prompts: silent.prompts, docker: dockerWithToken("new-sync-token") }, + ) + + expect(exitCode).toBe(1) + expect(silent.errors[0]).toBe( + `Could not patch ${join(targetDir, ".env")} — the file is missing ` + + "or has no OBSIDIAN_AUTH_TOKEN line. Run init first.", + ) + }) + + it("exits 1 when --dir .env does not exist", async () => { + const targetDir = join( + mkdtempSync(join(tmpdir(), "vault-cli-sync-token-")), + "nonexistent", + ) + const silent = createSilentPrompts() + + const exitCode = await runGetSyncToken( + { dir: targetDir }, + { prompts: silent.prompts, docker: dockerWithToken("new-sync-token") }, + ) + + expect(exitCode).toBe(1) + expect(silent.errors[0]).toBe( + `Could not patch ${join(targetDir, ".env")} — the file is missing ` + + "or has no OBSIDIAN_AUTH_TOKEN line. Run init first.", + ) + }) +}) diff --git a/cli/src/__tests__/init.test.ts b/cli/src/__tests__/init.test.ts index 0e4da32d..a9f4dc91 100644 --- a/cli/src/__tests__/init.test.ts +++ b/cli/src/__tests__/init.test.ts @@ -91,7 +91,7 @@ const dockerUnavailable: DockerRunner = { dockerRun: () => false, pullImage: () => false, stopAndRemoveContainer: () => false, - runGetToken: () => false, + runObsidianLogin: () => false, } const fetchNever: typeof fetch = async () => { @@ -540,13 +540,13 @@ describe("runInit interactive local flow", () => { }) describe("runInit remote flow", () => { - it("asks the remote sequence and writes .env", async () => { + it("asks the remote sequence with auto-capture declined and writes .env", async () => { const targetDir = makeTargetDir() const scripted = createScriptedPrompts([ "https://vault.example.com/", // public URL (trailing slash trimmed) "MyVault", // vault name - false, // don't run get-token now (offered because daemon is running) - "sync-token-xyz", // obsidian sync token + false, // don't generate the token now (declined auto-capture) + "sync-token-xyz", // paste fallback — obsidian sync token false, // no end-to-end encryption false, // don't start the server ]) @@ -568,7 +568,7 @@ describe("runInit remote flow", () => { expect(scripted.asked).toEqual([ "Public base URL clients will use to reach this server (no /mcp — it's added for you):", "Exact name of your Obsidian vault (case-sensitive):", - "Run the get-token command now?", + "Generate the token now?", "Paste the Obsidian Sync token (leave blank to fill in .env later):", "Does your vault use end-to-end encryption?", "Start the server now?", @@ -581,12 +581,58 @@ describe("runInit remote flow", () => { expect(scripted.prints[0]).toContain("Optional settings (timezone, memory") }) - it("skips the compose-up offer when the sync token was left blank", async () => { + it("skips paste prompt when auto-capture succeeds", async () => { + const targetDir = makeTargetDir() + const scripted = createScriptedPrompts([ + "https://vault.example.com", + "MyVault", + true, // generate the token now + false, // no end-to-end encryption + false, // don't start the server + ]) + const dockerWithCapture: DockerRunner = { + ...dockerUnavailable, + isDaemonRunning: () => true, + runObsidianLogin: (configMountPath) => { + const tokenDir = join(configMountPath, "obsidian-headless") + mkdirSync(tokenDir, { recursive: true }) + writeFileSync(join(tokenDir, "auth_token"), "captured-token") + return true + }, + } + + const exitCode = await runInit( + { mode: "remote", dir: targetDir }, + { + prompts: scripted.prompts, + docker: dockerWithCapture, + fetchFn: fetchNever, + }, + ) + + expect(exitCode).toBe(0) + expect(scripted.logs).toContain( + "Handing the terminal to the Obsidian login — it will ask for your " + + "account email, password, and MFA code. The token is captured " + + "automatically and stored in your .env — nothing to copy.", + ) + expect(scripted.asked).not.toContain( + "Paste the Obsidian Sync token (leave blank to fill in .env later):", + ) + const envContent = readFileSync(join(targetDir, ".env"), "utf8") + // Exact line match — a substring check would also pass for a commented + // or prefixed entry (e.g. "# OBSIDIAN_AUTH_TOKEN=captured-token"). + expect(envContent.split("\n")).toContain( + "OBSIDIAN_AUTH_TOKEN=captured-token", + ) + }) + + it("skips the docker-run offer when the sync token was left blank", async () => { const targetDir = makeTargetDir() const scripted = createScriptedPrompts([ "http://203.0.113.10:8000", "MyVault", - "", // blank token — fill in later + "", // blank token — fill in later (Docker unavailable, no capture offer) false, // no encryption ]) @@ -660,7 +706,7 @@ describe("runInit with a kept existing .env", () => { dockerRun: () => true, pullImage: () => true, stopAndRemoveContainer: () => true, - runGetToken: () => false, + runObsidianLogin: () => false, } const fetchedUrls: string[] = [] const fetchRecorder: typeof fetch = async (url) => { @@ -713,26 +759,25 @@ describe("runInit remote encryption password", () => { const scripted = createScriptedPrompts([ "https://vault.example.com", "MyVault", - "sync-token-xyz", + false, // decline auto-capture + "sync-token-xyz", // paste fallback true, // vault uses end-to-end encryption "hunter2", // password (masked prompt) false, // don't start the server ]) - const dockerComposeReady: DockerRunner = { + const dockerReady: DockerRunner = { isDaemonRunning: () => true, dockerRun: () => false, pullImage: () => false, stopAndRemoveContainer: () => false, - runGetToken: () => false, + runObsidianLogin: () => false, } - // get-token confirm slots in after VAULT_NAME when Docker is usable. - scripted.remaining.splice(2, 0, false) const exitCode = await runInit( { mode: "remote", dir: targetDir }, { prompts: scripted.prompts, - docker: dockerComposeReady, + docker: dockerReady, fetchFn: fetchNever, }, ) @@ -745,61 +790,29 @@ describe("runInit remote encryption password", () => { }) }) -describe("runInit get-token paste prompt wording", () => { - it('says "printed above" only when get-token ran to completion', async () => { - const targetDir = makeTargetDir() - const scripted = createScriptedPrompts([ - "https://vault.example.com", - "MyVault", - true, // run get-token now - "sync-token-xyz", - false, // no encryption - false, // don't start the server - ]) - const dockerWithWorkingGetToken: DockerRunner = { - isDaemonRunning: () => true, - dockerRun: () => false, - pullImage: () => false, - stopAndRemoveContainer: () => false, - runGetToken: () => true, - } - - await runInit( - { mode: "remote", dir: targetDir }, - { - prompts: scripted.prompts, - docker: dockerWithWorkingGetToken, - fetchFn: fetchNever, - }, - ) - - expect(scripted.asked).toContain( - "Paste the Obsidian Sync token printed above (leave blank to fill in .env later):", - ) - }) - - it('omits "printed above" when get-token failed', async () => { +describe("runInit sync-token auto-capture fallback", () => { + it("falls back to paste prompt when auto-capture fails", async () => { const targetDir = makeTargetDir() const scripted = createScriptedPrompts([ "https://vault.example.com", "MyVault", - true, // try to run get-token - "", // blank token — fill in later + true, // try to generate the token + "", // paste fallback — blank token, fill in later false, // no encryption ]) - const dockerWithFailingGetToken: DockerRunner = { + const dockerWithFailingCapture: DockerRunner = { isDaemonRunning: () => true, dockerRun: () => false, pullImage: () => false, stopAndRemoveContainer: () => false, - runGetToken: () => false, + runObsidianLogin: () => false, } await runInit( { mode: "remote", dir: targetDir }, { prompts: scripted.prompts, - docker: dockerWithFailingGetToken, + docker: dockerWithFailingCapture, fetchFn: fetchNever, }, ) @@ -807,6 +820,8 @@ describe("runInit get-token paste prompt wording", () => { expect(scripted.asked).toContain( "Paste the Obsidian Sync token (leave blank to fill in .env later):", ) - expect(scripted.warnings[0]).toContain("get-token did not complete") + expect(scripted.warnings[0]).toContain( + "The Obsidian login did not complete", + ) }) }) diff --git a/cli/src/__tests__/program.test.ts b/cli/src/__tests__/program.test.ts index 8db3d1a6..3527ea6d 100644 --- a/cli/src/__tests__/program.test.ts +++ b/cli/src/__tests__/program.test.ts @@ -1,12 +1,14 @@ import { describe, expect, it } from "vitest" import { buildProgram } from "../program.js" +import type { GetSyncTokenFlags } from "../get-sync-token.js" import type { InitFlags } from "../init.js" import type { UpgradeFlags } from "../upgrade.js" const buildCapturingProgram = () => { const initCalls: InitFlags[] = [] const upgradeCalls: UpgradeFlags[] = [] + const getSyncTokenCalls: GetSyncTokenFlags[] = [] const program = buildProgram({ version: "0.0.0-test", runInit: async (flags) => { @@ -17,12 +19,16 @@ const buildCapturingProgram = () => { upgradeCalls.push(flags) return 0 }, + runGetSyncToken: async (flags) => { + getSyncTokenCalls.push(flags) + return 0 + }, }) for (const command of [program, ...program.commands]) { command.exitOverride() command.configureOutput({ writeOut: () => {}, writeErr: () => {} }) } - return { program, initCalls, upgradeCalls } + return { program, initCalls, upgradeCalls, getSyncTokenCalls } } describe("buildProgram init", () => { @@ -93,3 +99,23 @@ describe("buildProgram upgrade", () => { expect(upgradeCalls).toEqual([{}]) }) }) + +describe("buildProgram get-sync-token", () => { + it("passes --dir through to runGetSyncToken", async () => { + const { program, getSyncTokenCalls } = buildCapturingProgram() + + await program.parseAsync(["get-sync-token", "--dir", "/opt/vault-cortex"], { + from: "user", + }) + + expect(getSyncTokenCalls).toEqual([{ dir: "/opt/vault-cortex" }]) + }) + + it("invokes get-sync-token with no flags when none are given", async () => { + const { program, getSyncTokenCalls } = buildCapturingProgram() + + await program.parseAsync(["get-sync-token"], { from: "user" }) + + expect(getSyncTokenCalls).toEqual([{}]) + }) +}) diff --git a/cli/src/__tests__/scaffold.test.ts b/cli/src/__tests__/scaffold.test.ts index a8fb7d27..6e727500 100644 --- a/cli/src/__tests__/scaffold.test.ts +++ b/cli/src/__tests__/scaffold.test.ts @@ -12,6 +12,7 @@ import { describe, expect, it } from "vitest" import { buildFilesToWrite, detectMode, + patchEnvObsidianToken, readEnvPort, readEnvVaultPath, writeFiles, @@ -270,3 +271,83 @@ describe("writeFiles permissions", () => { expect(fileMode).toBe(0o600) }) }) + +describe("patchEnvObsidianToken", () => { + it("replaces an existing token value", () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-patch-")) + const envPath = join(targetDir, ".env") + writeFileSync( + envPath, + "MCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=old-token\nVAULT_NAME=MyVault\n", + ) + + const result = patchEnvObsidianToken(envPath, "new-token") + + expect(result).toBe(true) + expect(readFileSync(envPath, "utf8")).toBe( + "MCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=new-token\nVAULT_NAME=MyVault\n", + ) + }) + + it("replaces an empty token value", () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-patch-")) + const envPath = join(targetDir, ".env") + writeFileSync(envPath, "OBSIDIAN_AUTH_TOKEN=\n") + + const result = patchEnvObsidianToken(envPath, "filled-in") + + expect(result).toBe(true) + expect(readFileSync(envPath, "utf8")).toBe( + "OBSIDIAN_AUTH_TOKEN=filled-in\n", + ) + }) + + it("returns false when the file does not exist", () => { + const result = patchEnvObsidianToken( + join(tmpdir(), "vault-cli-no-such-file", ".env"), + "token", + ) + + expect(result).toBe(false) + }) + + it("returns false when the file has no OBSIDIAN_AUTH_TOKEN line", () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-patch-")) + const envPath = join(targetDir, ".env") + writeFileSync(envPath, "MCP_AUTH_TOKEN=abc\nVAULT_PATH=/vault\n") + + const result = patchEnvObsidianToken(envPath, "token") + + expect(result).toBe(false) + expect(readFileSync(envPath, "utf8")).toBe( + "MCP_AUTH_TOKEN=abc\nVAULT_PATH=/vault\n", + ) + }) + + it("writes tokens containing $ patterns literally (no regex interpolation)", () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-patch-")) + const envPath = join(targetDir, ".env") + writeFileSync(envPath, "OBSIDIAN_AUTH_TOKEN=old-token\n") + + const result = patchEnvObsidianToken(envPath, "abc$&def$$1$'end") + + expect(result).toBe(true) + expect(readFileSync(envPath, "utf8")).toBe( + "OBSIDIAN_AUTH_TOKEN=abc$&def$$1$'end\n", + ) + }) + + it("preserves surrounding content when patching", () => { + const targetDir = mkdtempSync(join(tmpdir(), "vault-cli-patch-")) + const envPath = join(targetDir, ".env") + const original = + "# Comment\nMCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=old\nVAULT_NAME=Test\n# Footer\n" + writeFileSync(envPath, original) + + patchEnvObsidianToken(envPath, "new") + + expect(readFileSync(envPath, "utf8")).toBe( + "# Comment\nMCP_AUTH_TOKEN=abc\nOBSIDIAN_AUTH_TOKEN=new\nVAULT_NAME=Test\n# Footer\n", + ) + }) +}) diff --git a/cli/src/__tests__/upgrade.test.ts b/cli/src/__tests__/upgrade.test.ts index d2610065..9a9d38e4 100644 --- a/cli/src/__tests__/upgrade.test.ts +++ b/cli/src/__tests__/upgrade.test.ts @@ -46,7 +46,7 @@ const dockerReady: DockerRunner = { dockerRun: () => true, pullImage: () => true, stopAndRemoveContainer: () => true, - runGetToken: () => false, + runObsidianLogin: () => false, } const dockerDown: DockerRunner = { @@ -54,7 +54,7 @@ const dockerDown: DockerRunner = { dockerRun: () => false, pullImage: () => false, stopAndRemoveContainer: () => false, - runGetToken: () => false, + runObsidianLogin: () => false, } const fetchOk: typeof fetch = async () => ({ ok: true }) as Response diff --git a/cli/src/docker.ts b/cli/src/docker.ts index 4baa924e..1ac641b3 100644 --- a/cli/src/docker.ts +++ b/cli/src/docker.ts @@ -23,8 +23,54 @@ export type DockerRunner = { pullImage: (image: string) => boolean /** Stops and removes the vault-cortex container (idempotent). */ stopAndRemoveContainer: () => boolean - /** Runs the vault-cortex get-token flow with inherited stdio. */ - runGetToken: () => boolean + /** Runs the Obsidian login with a volume mount for token auto-capture. */ + runObsidianLogin: (configMountPath: string) => boolean +} + +export type ObsidianLoginArgParams = { + configMountPath: string + /** Defaults to process.platform. */ + platform?: NodeJS.Platform + /** Host UID for --user flag on Linux. */ + uid?: number + /** Host GID for --user flag on Linux. */ + gid?: number +} + +/** + * Builds the `docker run` args for the Obsidian login with a volume mount + * that captures the auth token file. Runs `ob login` directly instead of + * the image's get-sync-token script: the script's additions are locating and + * printing the token, and the mount makes both unnecessary — the CLI reads + * the token file itself, and not echoing a credential keeps it out of + * terminal scrollback. Pure function for testability. + * + * On Linux, includes `--user uid:gid` when uid/gid are provided — Node + * exposes process.getuid/getgid on every POSIX platform, so in practice the + * flag is always set there — keeping the token file host-user-owned. macOS + * Docker Desktop translates UIDs automatically, so no flag is needed. + */ +export const buildObsidianLoginArgs = ( + params: ObsidianLoginArgParams, +): string[] => { + const { configMountPath, platform = process.platform, uid, gid } = params + + const args = [ + "run", + "--rm", + "-it", + "--entrypoint", + "ob", + "-v", + `${configMountPath}:/home/obsidian/.config`, + ] + + if (platform === "linux" && uid !== undefined && gid !== undefined) { + args.push("--user", `${uid}:${gid}`) + } + + args.push(REMOTE_IMAGE, "login") + return args } /** @@ -105,10 +151,14 @@ export const createDockerRunner = (): DockerRunner => ({ spawnSync("docker", ["pull", image], { stdio: "inherit" }).status === 0, stopAndRemoveContainer: () => spawnSync("docker", ["rm", "-f", CONTAINER_NAME]).status === 0, - runGetToken: () => + runObsidianLogin: (configMountPath) => spawnSync( "docker", - ["run", "--rm", "-it", "--entrypoint", "get-token", REMOTE_IMAGE], + buildObsidianLoginArgs({ + configMountPath, + uid: process.getuid?.(), + gid: process.getgid?.(), + }), { stdio: "inherit" }, ).status === 0, }) diff --git a/cli/src/env.ts b/cli/src/env.ts index d9d67113..02896943 100644 --- a/cli/src/env.ts +++ b/cli/src/env.ts @@ -1,5 +1,3 @@ -import { REMOTE_IMAGE } from "./docker.js" - export type LocalEnvAnswers = { mcpAuthToken: string vaultPath: string @@ -210,8 +208,7 @@ VAULT_PASSWORD=${answers.vaultPassword}` answers.obsidianAuthToken === "" ? `# Obsidian Sync auth token — FILL THIS IN before starting the server. # Generate once with: -# docker run --rm -it --entrypoint get-token \\ -# ${REMOTE_IMAGE}` +# npx vault-cortex get-sync-token` : `# Obsidian Sync auth token.` return `# vault-cortex — remote quickstart (Obsidian Sync) diff --git a/cli/src/get-sync-token.ts b/cli/src/get-sync-token.ts new file mode 100644 index 00000000..2e1d00aa --- /dev/null +++ b/cli/src/get-sync-token.ts @@ -0,0 +1,199 @@ +import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs" +import { tmpdir } from "node:os" +import { join, resolve } from "node:path" + +import type { DockerRunner } from "./docker.js" +import type { Prompts } from "./prompts.js" +import { patchEnvObsidianToken } from "./scaffold.js" +import { expandTilde } from "./vault.js" + +export type GetSyncTokenFlags = { + dir?: string +} + +export type GetSyncTokenDeps = { + prompts: Prompts + docker: DockerRunner +} + +/** Message from an unknown throw — Error instances keep their message. */ +const describeError = (error: unknown): string => + error instanceof Error ? error.message : String(error) + +/** + * Creates the temp dir the container's config mount writes into. + * Returns undefined (after warning) when creation fails. + */ +const makeTempMountDir = (prompts: Prompts): string | undefined => { + try { + return mkdtempSync(join(tmpdir(), "vault-cortex-sync-token-")) + } catch (error) { + prompts.warn( + `Could not create a temp directory for token capture — ${describeError(error)}`, + ) + return undefined + } +} + +/** + * Runs the interactive Obsidian login container. A throw from the Docker + * runner is reported and treated the same as a non-zero exit. + */ +const runLoginContainer = ( + configMountPath: string, + deps: GetSyncTokenDeps, +): boolean => { + const { docker, prompts } = deps + try { + return docker.runObsidianLogin(configMountPath) + } catch (error) { + prompts.warn(`Docker run failed — ${describeError(error)}`) + return false + } +} + +/** + * Reads the captured token file from the config mount. Returns undefined + * when the file is missing, empty, or unreadable — the caller treats all + * three as "no token captured". + */ +const readCapturedTokenFile = (configMountPath: string): string | undefined => { + const tokenPath = join(configMountPath, "obsidian-headless", "auth_token") + try { + if (!existsSync(tokenPath)) return undefined + const token = readFileSync(tokenPath, "utf8").trim() + return token || undefined + } catch { + return undefined + } +} + +/** + * Best-effort removal of the temp mount dir. Failing to remove it (e.g. + * root-owned files left by the container) must not turn a successful + * capture into a failure, so it warns instead of throwing. + */ +const removeTempMountDir = ( + configMountPath: string, + prompts: Prompts, +): void => { + try { + rmSync(configMountPath, { recursive: true, force: true }) + } catch (error) { + prompts.warn( + `Could not remove temp directory ${configMountPath} — ${describeError(error)}`, + ) + } +} + +/** + * Runs the Obsidian login (`ob login`) inside a Docker container with a + * volume mount that captures the auth token file. The interactive login + * (email, password, MFA) shows in the terminal, but the resulting token is + * read from the mounted config dir — never printed, so it stays out of + * terminal scrollback. + * + * tokenDestinationMessage finishes the handoff message by telling the user + * where the captured token ends up — the destination differs per flow + * (init stores it in the generated .env; the subcommand prints it, or + * writes it to an existing .env with --dir). + * + * Returns the token string on success, undefined on any failure — each + * fallible operation is wrapped individually by the helpers above, so no + * catch-all is needed here. The bare try/finally only scopes the temp dir + * (acquire → release); it has no catch and swallows nothing. + */ +export const captureObsidianToken = ( + deps: GetSyncTokenDeps, + tokenDestinationMessage: string, +): string | undefined => { + const { prompts } = deps + const configMountPath = makeTempMountDir(prompts) + if (!configMountPath) return undefined + + try { + prompts.log( + "Handing the terminal to the Obsidian login — it will ask for your " + + `account email, password, and MFA code. ${tokenDestinationMessage}`, + ) + const loginSucceeded = runLoginContainer(configMountPath, deps) + if (!loginSucceeded) { + prompts.warn( + "The Obsidian login did not complete — you can run it later with:\n" + + " npx vault-cortex get-sync-token", + ) + return undefined + } + const token = readCapturedTokenFile(configMountPath) + if (!token) { + prompts.warn( + "The Obsidian login finished, but no token was captured — the " + + "token file was missing, empty, or unreadable. You can retry with:\n" + + " npx vault-cortex get-sync-token", + ) + return undefined + } + return token + } finally { + removeTempMountDir(configMountPath, prompts) + } +} + +/** + * Subcommand entry: generate an Obsidian Sync token via Docker. + * Without --dir, prints the token to stdout. + * With --dir, writes it directly to `/.env`. + */ +export const runGetSyncToken = async ( + flags: GetSyncTokenFlags, + deps: GetSyncTokenDeps, +): Promise => { + const { prompts, docker } = deps + + if (!docker.isDaemonRunning()) { + prompts.error( + "Container runtime not running — start Docker Desktop, Colima,\n" + + "OrbStack, or another Docker-compatible runtime and try again.", + ) + return 1 + } + + prompts.intro("vault-cortex get-sync-token") + + // Resolve the destination up front so the login handoff message can tell + // the user where the token will end up. + const envFilePath = flags.dir + ? join(resolve(expandTilde(flags.dir)), ".env") + : undefined + const tokenDestinationMessage = envFilePath + ? `The token is captured automatically and written to ${envFilePath}.` + : "The token is captured automatically and printed at the end." + + const token = captureObsidianToken( + { docker, prompts }, + tokenDestinationMessage, + ) + if (!token) { + prompts.error("Could not capture the auth token.") + return 1 + } + + if (!envFilePath) { + prompts.log("Your OBSIDIAN_AUTH_TOKEN:") + prompts.print(`\n ${token}\n`) + prompts.outro("Done.") + return 0 + } + + const patched = patchEnvObsidianToken(envFilePath, token) + if (!patched) { + prompts.error( + `Could not patch ${envFilePath} — the file is missing or has no ` + + "OBSIDIAN_AUTH_TOKEN line. Run init first.", + ) + return 1 + } + prompts.log(`Token written to ${envFilePath}`) + prompts.outro("Done.") + return 0 +} diff --git a/cli/src/init.ts b/cli/src/init.ts index 6328694e..06d52a86 100644 --- a/cli/src/init.ts +++ b/cli/src/init.ts @@ -1,11 +1,12 @@ import { join, resolve } from "node:path" import { buildLocalEnv, buildRemoteEnv } from "./env.js" +import { captureObsidianToken } from "./get-sync-token.js" import { buildLocalConnectMessage, buildRemoteConnectMessage, } from "./messages.js" -import { REMOTE_IMAGE, pollHealth, type DockerRunner } from "./docker.js" +import { pollHealth, type DockerRunner } from "./docker.js" import { buildFilesToWrite, readEnvPort, @@ -55,33 +56,21 @@ const askMode = async (prompts: Prompts): Promise => { return isMode(selected) ? selected : "local" } -const GET_TOKEN_COMMAND = `docker run --rm -it --entrypoint get-token \\ - ${REMOTE_IMAGE}` - /** - * Offers to run the vault-cortex image's get-token flow in this terminal. - * Returns true only when it ran to completion (and so printed a token the - * user can scroll up to). The handoff log exists because the clack UI gives - * way to raw docker output — image pull, then the tool's own login prompts. + * Offers to auto-capture the Obsidian Sync token via a Docker volume mount. + * Returns the captured token string, or undefined when the user declines or + * the capture fails (the caller falls back to a paste prompt). */ -const offerGetTokenRun = async ( +const offerSyncTokenCapture = async ( prompts: Prompts, docker: DockerRunner, -): Promise => { - const runNow = await prompts.confirm("Run the get-token command now?", true) - if (!runNow) return false - prompts.log( - "Handing the terminal to get-token — it will ask for your Obsidian " + - "account login and print a token at the end.", +): Promise => { + const runNow = await prompts.confirm("Generate the token now?", true) + if (!runNow) return undefined + return captureObsidianToken( + { docker, prompts }, + "The token is captured automatically and stored in your .env — nothing to copy.", ) - const tokenGenerated = docker.runGetToken() - if (!tokenGenerated) { - prompts.warn( - "get-token did not complete — you can run it later and edit .env.", - ) - return false - } - return true } /** @@ -344,7 +333,7 @@ const runLocalInit = async ( } // Remote flow (VPS + Obsidian Sync): resolve target dir → PUBLIC_URL → -// VAULT_NAME → Obsidian Sync token (optionally running get-token via +// VAULT_NAME → Obsidian Sync token (optionally running the Obsidian login via // Docker) → optional E2E vault password → generate token → write .env → // optionally start → print connect instructions. Always interactive — // the sync-token step can't be defaulted. @@ -369,23 +358,22 @@ const runRemoteInit = async ( const publicUrl = await askPublicUrl(prompts) const vaultName = await askVaultName(prompts) - // The Obsidian Sync token comes from an interactive docker run (the - // get-token entrypoint logs into Obsidian). We print the command, offer to - // run it when Docker is usable, then ask the user to paste the result — - // get-token writes to the terminal, so it can't be captured automatically. - // A blank answer is allowed: the .env is written with an empty - // OBSIDIAN_AUTH_TOKEN and a fill-this-in comment. - prompts.note(GET_TOKEN_COMMAND, "Obsidian Sync token — generate once with") - const getTokenRan = docker.isDaemonRunning() - ? await offerGetTokenRun(prompts, docker) - : false - // "printed above" is only true when get-token actually ran to completion. - const pastePrompt = getTokenRan - ? "Paste the Obsidian Sync token printed above (leave blank to fill in .env later):" - : "Paste the Obsidian Sync token (leave blank to fill in .env later):" - const obsidianAuthToken = ( - await prompts.text(pastePrompt, { defaultValue: "" }) - ).trim() + // Auto-capture the Obsidian Sync token via a Docker volume mount when + // the daemon is reachable. Falls back to a paste prompt when capture + // fails or the user declines. + const capturedToken = docker.isDaemonRunning() + ? await offerSyncTokenCapture(prompts, docker) + : undefined + // Masked prompt: the sync token is a credential and must not echo into + // the terminal or scrollback. An empty submission still means "fill in + // .env later" — clack's password prompt accepts blank input. + const obsidianAuthToken = + capturedToken ?? + ( + await prompts.password( + "Paste the Obsidian Sync token (leave blank to fill in .env later):", + ) + ).trim() const usesEncryption = await prompts.confirm( "Does your vault use end-to-end encryption?", diff --git a/cli/src/main.ts b/cli/src/main.ts index ee3dadb5..acc2b53a 100644 --- a/cli/src/main.ts +++ b/cli/src/main.ts @@ -1,4 +1,5 @@ import { createDockerRunner } from "./docker.js" +import { runGetSyncToken } from "./get-sync-token.js" import { runInit } from "./init.js" import { buildProgram } from "./program.js" import { createPrompts } from "./prompts.js" @@ -19,6 +20,11 @@ export const run = async (version: string): Promise => { docker: createDockerRunner(), fetchFn: fetch, }), + runGetSyncToken: (flags) => + runGetSyncToken(flags, { + prompts: createPrompts(), + docker: createDockerRunner(), + }), }) await program.parseAsync() } diff --git a/cli/src/program.ts b/cli/src/program.ts index 61b03131..0ffba15a 100644 --- a/cli/src/program.ts +++ b/cli/src/program.ts @@ -1,5 +1,6 @@ import { Command } from "commander" +import type { GetSyncTokenFlags } from "./get-sync-token.js" import type { InitFlags } from "./init.js" import type { UpgradeFlags } from "./upgrade.js" @@ -7,6 +8,7 @@ export type ProgramOptions = { version: string runInit: (flags: InitFlags) => Promise runUpgrade: (flags: UpgradeFlags) => Promise + runGetSyncToken: (flags: GetSyncTokenFlags) => Promise } export const buildProgram = (options: ProgramOptions): Command => { @@ -52,6 +54,19 @@ export const buildProgram = (options: ProgramOptions): Command => { process.exitCode = await options.runUpgrade(flags) }) + program + .command("get-sync-token") + .description( + "Generate an Obsidian Sync auth token via Docker and print it or write it to .env", + ) + .option( + "--dir ", + "directory containing .env to update with the token", + ) + .action(async (flags: GetSyncTokenFlags) => { + process.exitCode = await options.runGetSyncToken(flags) + }) + program.action(() => { program.help() }) diff --git a/cli/src/scaffold.ts b/cli/src/scaffold.ts index 74a8fa2e..80fb5373 100644 --- a/cli/src/scaffold.ts +++ b/cli/src/scaffold.ts @@ -85,6 +85,30 @@ export const detectMode = (envFilePath: string): Mode | undefined => { return OBSIDIAN_AUTH_TOKEN_LINE.test(content) ? "remote" : "local" } +/** + * Patches the OBSIDIAN_AUTH_TOKEN value in an existing .env file. + * Returns true when the patch succeeded, false when the file is missing + * or has no active OBSIDIAN_AUTH_TOKEN line (e.g. a local-mode .env). + */ +export const patchEnvObsidianToken = ( + envFilePath: string, + token: string, +): boolean => { + if (!existsSync(envFilePath)) return false + const content = readFileSync(envFilePath, "utf8") + /** Matches the full OBSIDIAN_AUTH_TOKEN line for replacement. */ + const fullTokenLine = /^OBSIDIAN_AUTH_TOKEN=.*$/m + if (!fullTokenLine.test(content)) return false + // Function replacement avoids $ pattern interpretation ($&, $', etc.) + // that String.prototype.replace applies to string replacements. + const patched = content.replace( + fullTokenLine, + () => `OBSIDIAN_AUTH_TOKEN=${token}`, + ) + writeFileSync(envFilePath, patched) + return true +} + /** * Writes the files into targetDir (created if missing). Existing files * are never overwritten silently: identical content is skipped, and differing diff --git a/deploy/remote/.env.example b/deploy/remote/.env.example index 15cc0e5a..008afb43 100644 --- a/deploy/remote/.env.example +++ b/deploy/remote/.env.example @@ -13,7 +13,7 @@ MCP_AUTH_TOKEN= PUBLIC_URL= # Obsidian Sync auth token. Generate once with: -# docker run --rm -it --entrypoint get-token \ +# docker run --rm -it --entrypoint get-sync-token \ # ghcr.io/aliasunder/vault-cortex:remote OBSIDIAN_AUTH_TOKEN= diff --git a/deploy/remote/README.md b/deploy/remote/README.md index c9f7ee1f..fe1e3ad4 100644 --- a/deploy/remote/README.md +++ b/deploy/remote/README.md @@ -60,8 +60,17 @@ Or clone the repo and `cd deploy/remote`. **3. Generate your Obsidian Sync auth token** (one-time): +If you have Node.js >= 20.12 on this machine, the CLI runs the login and +captures the token for you: + +```bash +npx vault-cortex get-sync-token +``` + +Otherwise, run the Docker image directly: + ```bash -docker run --rm -it --entrypoint get-token \ +docker run --rm -it --entrypoint get-sync-token \ ghcr.io/aliasunder/vault-cortex:remote ``` diff --git a/rootfs/etc/s6-overlay/scripts/init-check-auth b/rootfs/etc/s6-overlay/scripts/init-check-auth index 7ac882fc..a3d762df 100644 --- a/rootfs/etc/s6-overlay/scripts/init-check-auth +++ b/rootfs/etc/s6-overlay/scripts/init-check-auth @@ -1,11 +1,18 @@ #!/command/with-contenv sh -# Validate that OBSIDIAN_AUTH_TOKEN is set before proceeding. -set -e +# Init-chain gate: refuse to start when no Obsidian Sync auth token is +# configured — everything downstream (login, sync-setup) would fail with a +# far less obvious error. +set -eu -if [ -z "$OBSIDIAN_AUTH_TOKEN" ]; then - echo "[obsidian-sync] ERROR: OBSIDIAN_AUTH_TOKEN is not set." >&2 - echo "[obsidian-sync] Run: docker run --rm -it --entrypoint get-token " >&2 - exit 1 +if [ -n "${OBSIDIAN_AUTH_TOKEN:-}" ]; then + echo "[obsidian-sync] Auth token present." + exit 0 fi -echo "[obsidian-sync] Auth token present." +{ + echo "[obsidian-sync] ERROR: OBSIDIAN_AUTH_TOKEN is empty or unset." + echo "[obsidian-sync] Generate one with:" + echo "[obsidian-sync] docker run --rm -it --entrypoint get-sync-token " + echo "[obsidian-sync] then add it to your .env and restart the container." +} >&2 +exit 1 diff --git a/rootfs/etc/s6-overlay/scripts/init-obsidian-login b/rootfs/etc/s6-overlay/scripts/init-obsidian-login index 0b0ccc05..b12483c7 100644 --- a/rootfs/etc/s6-overlay/scripts/init-obsidian-login +++ b/rootfs/etc/s6-overlay/scripts/init-obsidian-login @@ -1,11 +1,14 @@ #!/command/with-contenv sh -# Authenticate with Obsidian using the provided auth token. -set -e +# Init-chain step: log obsidian-headless in to the Obsidian account, using +# the OBSIDIAN_AUTH_TOKEN provided via the environment (validated by +# init-check-auth, which runs before this step). Runs as the obsidian user +# so the stored credentials are owned by the service account. +set -eu -echo "[obsidian-sync] Logging in to Obsidian..." -if ! s6-setuidgid obsidian ob login; then - echo "[obsidian-sync] ERROR: ob login failed." >&2 +echo "[obsidian-sync] Authenticating with Obsidian..." +s6-setuidgid obsidian ob login || { + echo "[obsidian-sync] ERROR: login was rejected — the auth token may be" >&2 + echo "[obsidian-sync] stale. Generate a fresh one with get-sync-token." >&2 exit 1 -fi - -echo "[obsidian-sync] Login successful." +} +echo "[obsidian-sync] Authenticated." diff --git a/rootfs/usr/local/bin/get-sync-token b/rootfs/usr/local/bin/get-sync-token new file mode 100755 index 00000000..59dffa53 --- /dev/null +++ b/rootfs/usr/local/bin/get-sync-token @@ -0,0 +1,46 @@ +#!/bin/sh +# get-sync-token — interactive helper for the manual (no-CLI) flow: signs in +# to the user's Obsidian account and prints the Sync auth token for .env. +# +# Run with the s6 services bypassed: +# docker run --rm -it --entrypoint get-sync-token +# +# The npx CLI (`npx vault-cortex get-sync-token`) captures the token file +# through a volume mount instead and never runs this script. + +set -eu + +# The Dockerfile sets HOME=/home/obsidian; default it so the script also +# works outside the image (e.g. during development). +export HOME="${HOME:-/home/obsidian}" + +printf '%s\n' \ + "" \ + "Obsidian Sync token setup" \ + "Sign in with your Obsidian account — email, password, and MFA code if enabled." \ + "" + +ob login + +# obsidian-headless writes the token under XDG config; fall back to a +# filesystem search in case a future version relocates it. +token_file="${HOME}/.config/obsidian-headless/auth_token" +if [ ! -s "$token_file" ]; then + token_file="$(find "$HOME" -name auth_token -path '*obsidian-headless*' 2>/dev/null | head -n 1)" +fi + +# -s also rejects an empty path (the find found nothing) and an empty file. +if [ ! -s "${token_file}" ]; then + printf '%s\n' \ + "The login finished but no auth token file was found." \ + "Search for it manually with:" \ + " find ~ -name auth_token 2>/dev/null" >&2 + exit 1 +fi + +printf '%s\n' \ + "" \ + "Login complete. Copy this line into your .env:" \ + "" \ + " OBSIDIAN_AUTH_TOKEN=$(cat "$token_file")" \ + "" diff --git a/rootfs/usr/local/bin/get-token b/rootfs/usr/local/bin/get-token deleted file mode 100644 index a1fc5319..00000000 --- a/rootfs/usr/local/bin/get-token +++ /dev/null @@ -1,62 +0,0 @@ -#!/bin/sh -# Interactive helper: logs in to Obsidian and prints the resulting auth token. -# -# Usage (override entrypoint so s6-overlay services are not started): -# docker run --rm -it --entrypoint get-token - -set -e - -# HOME is set by the Dockerfile ENV; this fallback ensures the script works -# when run outside the container (e.g. during development). -export HOME="${HOME:-/home/obsidian}" - -echo "" -echo "=== obsidian-headless: Get Auth Token ===" -echo "" -echo "Log in to your Obsidian account." -echo "You will be prompted for your email, password, and MFA code (if enabled)." -echo "" - -ob login - -echo "" -echo "===========================================" - -# Locate the stored token file (fall back to a find if path differs) -TOKEN="" -CANDIDATES=" - ${HOME}/.config/obsidian-headless/auth_token - ${HOME}/.local/share/obsidian-headless/auth_token - ${HOME}/.obsidian-headless/auth_token -" - -for candidate in $CANDIDATES; do - if [ -f "$candidate" ]; then - TOKEN=$(cat "$candidate") - break - fi -done - -if [ -z "$TOKEN" ]; then - TOKEN_FILE=$(find "$HOME" -path "*obsidian-headless*" -name "auth_token" 2>/dev/null | head -1) - if [ -n "$TOKEN_FILE" ]; then - TOKEN=$(cat "$TOKEN_FILE") - fi -fi - -if [ -z "$TOKEN" ]; then - echo "Could not locate the auth token file automatically." >&2 - echo "Search manually with:" >&2 - echo " find ~ -path '*obsidian-headless*' 2>/dev/null" >&2 - exit 1 -fi - -echo "" -echo "Your OBSIDIAN_AUTH_TOKEN:" -echo "" -echo " $TOKEN" -echo "" -echo "Add this to your .env file:" -echo " OBSIDIAN_AUTH_TOKEN=$TOKEN" -echo "" -echo "==========================================="