Skip to content

feat(009): context-aware reply options (Smart Replies)#6

Open
alch3mistdev wants to merge 1 commit into
mainfrom
claude/context-prompt-options-qkc13u
Open

feat(009): context-aware reply options (Smart Replies)#6
alch3mistdev wants to merge 1 commit into
mainfrom
claude/context-prompt-options-qkc13u

Conversation

@alch3mistdev

Copy link
Copy Markdown
Owner

Summary

Adds Smart Replies (spec 009-context-prompt-options): a third on-device skill alongside STT and cleanup so mid-session follow-ups can be picked instead of dictated. When opted in, opening the Bark menu reads the focused app's latest message (Accessibility) and offers tappable replies. Picking one types it in — Bark never presses Return (auto-submit is deliberately out of scope).

Three reply tiers, matching the request:

  1. Yes/NoQuestionClassifier detects polar questions → exactly Yes/No (instant, no model).
  2. Most-likely repliesMLXBranchSuggester asks the on-device Qwen3-4B for the likely replies (bounded/parsed, hard deadline, falls back to quick replies).
  3. Custom — none fit → dictate normally with the hotkey.

Design

Follows the deterministic-first / LLM-optional pattern (ADR-003); no changes to the DictationStateMachine (parallel flow, like re-insert).

  • BarkCore (pure, unit-tested): ContextProvider + BranchSuggester protocols, ConversationContext / BranchOption, QuestionClassifier, BasicBranchSuggester, and an injection-safe BranchPromptTemplate (fenced context, output parse/bound).
  • BarkEngines: AccessibilityContextReader — best-effort AX read of the focused window (bounded depth/nodes/chars, AX messaging timeout; documented residual).
  • BarkCleanupMLX: shared MLXModelHost so cleanup + suggestions reuse one Qwen3-4B download/container; MLXBranchSuggester; lean no-op stubs.
  • Bark: controller orchestration, menu Smart Replies section, Settings toggle, Privacy note.

Privacy / safety

  • Opt-in, off by default (smartRepliesEnabled) — the only feature that reads other apps' on-screen text (least privilege).
  • On-device only — context goes only to the on-device model; nothing transmitted; never persisted (held only while the menu is open).
  • Read context is fenced as untrusted data with a "data, not instructions" guardrail; picks reuse the existing sanitize / focus re-verify / secure-field guards; Return is never synthesized.
  • Documented in docs/SECURITY.md.

⚠️ Verification status (please run on macOS)

This was developed in a Linux container with no Swift toolchain and no macOS-26 SDK, so per the constitution's "Evidence or it didn't happen" principle these are NOT yet claimed and are left unchecked in tasks.md:

  • T017swift build + swift test green; MLX target compiles (cp Package-mlx.swift Package.swift && swift build).
  • T018 — adversarial review (Codex + ef-adversary) on the diff.

Tests are written (Tests/BarkCoreTests/BranchSuggestionTests.swift, Tests/BarkAppTests/SmartRepliesTests.swift) but unproven until compiled. Likely first-compile watch points: @Observable tracking of the private branchContext via the hasBranchContext/canSuggestWithLLM computed props, and whether AccessibilityContextReader's NSWorkspace.frontmostApplication call should be @MainActor (like FocusProbe.currentTarget()).

Spec Kit docs

specs/009-context-prompt-options/{spec,plan,tasks}.md.

🤖 Generated with Claude Code


Generated by Claude Code

Add a third on-device skill alongside STT and cleanup so mid-session
follow-ups can be picked instead of dictated. When opted in, Bark reads
the focused app's latest message (Accessibility) and offers tappable
replies: deterministic Yes/No or quick replies instantly, or the most
likely replies from the on-device LLM. Picking one types it in — Bark
never presses Return (auto-submit deferred). None fit? Dictate a custom
reply with the normal hotkey.

Follows the deterministic-first / LLM-optional pattern (ADR-003):
- BarkCore: ContextProvider + BranchSuggester protocols, ConversationContext
  / BranchOption, QuestionClassifier, BasicBranchSuggester, and an
  injection-safe BranchPromptTemplate (fenced context, output parse/bound).
  All pure and unit-tested.
- BarkEngines: AccessibilityContextReader (best-effort AX read, bounded,
  documented residual).
- BarkCleanupMLX: shared MLXModelHost so cleanup + suggestions reuse one
  Qwen3-4B download/container; MLXBranchSuggester; lean stubs.
- Bark: controller orchestration (parallel flow, no state-machine changes),
  menu Smart Replies section, Settings toggle (off by default), Privacy note.

Opt-in and on-device only: nothing is transmitted or persisted; picks reuse
the existing sanitize / focus re-verify / secure-field guards.

Spec Kit docs in specs/009-context-prompt-options/. Build/test verification
(T017) and adversarial review (T018) are pending on macOS — this Linux
environment has no Swift toolchain or macOS-26 SDK, so they are NOT claimed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V1pCom6Cw6UN84HZ5JtsE5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants