feat(009): context-aware reply options (Smart Replies)#6
Open
alch3mistdev wants to merge 1 commit into
Open
Conversation
Add a third on-device skill alongside STT and cleanup so mid-session follow-ups can be picked instead of dictated. When opted in, Bark reads the focused app's latest message (Accessibility) and offers tappable replies: deterministic Yes/No or quick replies instantly, or the most likely replies from the on-device LLM. Picking one types it in — Bark never presses Return (auto-submit deferred). None fit? Dictate a custom reply with the normal hotkey. Follows the deterministic-first / LLM-optional pattern (ADR-003): - BarkCore: ContextProvider + BranchSuggester protocols, ConversationContext / BranchOption, QuestionClassifier, BasicBranchSuggester, and an injection-safe BranchPromptTemplate (fenced context, output parse/bound). All pure and unit-tested. - BarkEngines: AccessibilityContextReader (best-effort AX read, bounded, documented residual). - BarkCleanupMLX: shared MLXModelHost so cleanup + suggestions reuse one Qwen3-4B download/container; MLXBranchSuggester; lean stubs. - Bark: controller orchestration (parallel flow, no state-machine changes), menu Smart Replies section, Settings toggle (off by default), Privacy note. Opt-in and on-device only: nothing is transmitted or persisted; picks reuse the existing sanitize / focus re-verify / secure-field guards. Spec Kit docs in specs/009-context-prompt-options/. Build/test verification (T017) and adversarial review (T018) are pending on macOS — this Linux environment has no Swift toolchain or macOS-26 SDK, so they are NOT claimed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1pCom6Cw6UN84HZ5JtsE5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Smart Replies (spec
009-context-prompt-options): a third on-device skill alongside STT and cleanup so mid-session follow-ups can be picked instead of dictated. When opted in, opening the Bark menu reads the focused app's latest message (Accessibility) and offers tappable replies. Picking one types it in — Bark never presses Return (auto-submit is deliberately out of scope).Three reply tiers, matching the request:
QuestionClassifierdetects polar questions → exactlyYes/No(instant, no model).MLXBranchSuggesterasks the on-device Qwen3-4B for the likely replies (bounded/parsed, hard deadline, falls back to quick replies).Design
Follows the deterministic-first / LLM-optional pattern (ADR-003); no changes to the
DictationStateMachine(parallel flow, like re-insert).ContextProvider+BranchSuggesterprotocols,ConversationContext/BranchOption,QuestionClassifier,BasicBranchSuggester, and an injection-safeBranchPromptTemplate(fenced context, output parse/bound).AccessibilityContextReader— best-effort AX read of the focused window (bounded depth/nodes/chars, AX messaging timeout; documented residual).MLXModelHostso cleanup + suggestions reuse one Qwen3-4B download/container;MLXBranchSuggester; lean no-op stubs.Privacy / safety
smartRepliesEnabled) — the only feature that reads other apps' on-screen text (least privilege).docs/SECURITY.md.This was developed in a Linux container with no Swift toolchain and no macOS-26 SDK, so per the constitution's "Evidence or it didn't happen" principle these are NOT yet claimed and are left unchecked in
tasks.md:swift build+swift testgreen; MLX target compiles (cp Package-mlx.swift Package.swift && swift build).Tests are written (
Tests/BarkCoreTests/BranchSuggestionTests.swift,Tests/BarkAppTests/SmartRepliesTests.swift) but unproven until compiled. Likely first-compile watch points:@Observabletracking of the privatebranchContextvia thehasBranchContext/canSuggestWithLLMcomputed props, and whetherAccessibilityContextReader'sNSWorkspace.frontmostApplicationcall should be@MainActor(likeFocusProbe.currentTarget()).Spec Kit docs
specs/009-context-prompt-options/{spec,plan,tasks}.md.🤖 Generated with Claude Code
Generated by Claude Code