Add Performance settings pane with opt-in LLM latency tracking

[jkrauska]

Summary

Adds a new "Performance" Settings pane with an off-by-default toggle that records the timestamp, model name, and latency_ms of every LLM generation into a 100-entry ring buffer (persisted as JSON in UserDefaults). When tracking is on, both engines and the AI→llama locale fallback are captured at the router boundary, so the same gate controls all paths.

This is purely informational — the captured data is shown only inside the Performance pane and is intended to help users (and contributors) eyeball variations in per-model latency, so they can compare how different llama models or Apple Intelligence perform on their own machine. Nothing in the autocomplete pipeline reads these metrics back; they don't influence routing, ranking, or any user-facing behavior.

Validation

xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' \
  build -derivedDataPath build/DerivedData CODE_SIGNING_ALLOWED=NO
# ** BUILD SUCCEEDED **

xcodebuild -project Cotabby.xcodeproj -scheme Cotabby -destination 'platform=macOS' \
  build-for-testing -derivedDataPath build/DerivedData CODE_SIGNING_ALLOWED=NO
# ** TEST BUILD SUCCEEDED **

UI checked manually: opened Settings → Performance, flipped the toggle on, triggered suggestions, watched entries appear newest-first with monospaced timestamp/duration columns and an enabled Clear button. Toggling off short-circuits recording at the router; existing entries remain until cleared. Empty state copy switches between "tracking is off" and "no requests yet" based on the toggle.

swiftlint lint --quiet not run locally (binary not installed on this machine); CI will gate.

Linked issues

None.

Risk / rollout notes

• Defaults: cotabbyPerformanceTrackingEnabled defaults to false and the ring buffer is empty on first launch; existing users see no behavior change until they opt in.
• New UserDefaults keys (both off/empty by default):
  • cotabbyPerformanceTrackingEnabled: Bool
  • cotabbyPerformanceMetricEntries: Data (JSON-encoded [PerformanceMetricEntry], capped at 100)
• Hot path cost: when the toggle is off, the router does a single Bool read and returns — no allocation, no UserDefaults write.
• Composition change: SuggestionEngineRouter now takes performanceMetricsStore and a llamaModelNameProvider closure; SettingsCoordinator / SettingsContainerView thread a shared PerformanceMetricsStore. LlamaRuntimeManager gains a read-only currentModelFilename accessor so the recorder can label entries with the actual GGUF filename.
• pbxproj: regenerated via xcodegen generate for the two new files (PerformanceMetricsStore.swift, PerformancePaneView.swift). Auto-discovery handled them; no manual project.yml edits.

Greptile Summary

Adds an opt-in "Performance" Settings pane that records the timestamp, model name, and latency_ms of every LLM generation into a 100-entry ring buffer persisted via UserDefaults. The toggle defaults to false, so no existing user sees any behavior change.

• PerformanceMetricsStore is a new @MainActor ObservableObject that owns the ring buffer; SuggestionEngineRouter (already @MainActor) calls recordPerformanceMetric after every successful engine response — covering the direct Apple Intelligence path, the direct llama path, and the locale-fallback llama path.
• PerformancePaneView renders a newest-first table of captured entries with inline empty-state copy that distinguishes "tracking is off" from "no entries yet", plus a Clear button that removes the UserDefaults blob.
• SuggestionSettingsModel gains a isPerformanceTrackingEnabled flag (defaults false) wired to UserDefaults; SettingsCoordinator, SettingsContainerView, and CotabbyAppEnvironment are updated to thread the shared store through composition.

Confidence Score: 5/5

Safe to merge — the recording gate defaults off, the hot path is a single bool read, and actor isolation is handled correctly throughout.

The feature is entirely additive and off by default. The router is correctly @mainactor, so all calls into PerformanceMetricsStore are properly isolated. The ring buffer cap keeps UserDefaults writes bounded and infrequent. No existing behaviour is changed for users who never visit the Performance pane.

No files require special attention; the only note is a cosmetic usability gap in PerformancePaneView's timestamp formatter.

Important Files Changed

Filename	Overview
Cotabby/Models/PerformanceMetricsStore.swift	New @mainactor ring-buffer store; clear guard, load/persist logic, and cap enforcement look correct. Previous review comment about silent persist failure has been addressed with an error log.
Cotabby/Services/Runtime/SuggestionEngineRouter.swift	Router is @mainactor so all PerformanceMetricsStore and llamaModelNameProvider calls are correctly isolated; recording covers all three success paths (direct Apple Intelligence, direct llama, AI to llama fallback).
Cotabby/UI/Settings/Panes/PerformancePaneView.swift	UI logic is clean; timestampFormatter omits dateStyle, so entries loaded from a previous session will show only a time-of-day with no indication of which day — minor usability gap for a persisted history.
Cotabby/Models/SuggestionSettingsModel.swift	Adds isPerformanceTrackingEnabled following the exact pattern of other boolean flags; default false, idempotent setter, persist/load wired correctly.
Cotabby/App/Core/CotabbyAppEnvironment.swift	Wires PerformanceMetricsStore and the weak-runtimeManager provider closure into the router and settings coordinator; composition is straightforward.
CotabbyTests/PromptPolicyTests.swift	Existing router tests updated to satisfy new init signature; builds cleanly. Toggle-gate branch coverage was flagged in a previous review.
Cotabby/UI/Settings/SettingsCategory.swift	Adds .performance case with correct label and SF Symbol; SettingsAttentionEvaluator updated to return nil for the new case.

Sequence Diagram

sequenceDiagram
    participant SC as SuggestionCoordinator
    participant SER as SuggestionEngineRouter (@MainActor)
    participant FM as FoundationModelEngine
    participant LE as LlamaEngine
    participant PMS as PerformanceMetricsStore
    participant UD as UserDefaults

    SC->>SER: generateSuggestion(request)
    alt "selectedEngine == .appleIntelligence"
        SER->>FM: generateSuggestion(request)
        FM-->>SER: SuggestionResult (latency)
        SER->>SER: recordPerformanceMetric(Apple Intelligence, latency)
        SER->>PMS: record(modelName:latencyMs:) [if tracking enabled]
        PMS->>UD: persist JSON blob
    else unsupportedLanguageOrLocale fallback
        SER->>FM: generateSuggestion(request)
        FM-->>SER: throws unsupportedLanguageOrLocale
        SER->>LE: generateSuggestion(request)
        LE-->>SER: SuggestionResult (latency)
        SER->>SER: recordPerformanceMetric(llamaModelName, latency)
        SER->>PMS: record(modelName:latencyMs:) [if tracking enabled]
        PMS->>UD: persist JSON blob
    else "selectedEngine == .llamaOpenSource"
        SER->>LE: generateSuggestion(request)
        LE-->>SER: SuggestionResult (latency)
        SER->>SER: recordPerformanceMetric(llamaModelName, latency)
        SER->>PMS: record(modelName:latencyMs:) [if tracking enabled]
        PMS->>UD: persist JSON blob
    end
    SER-->>SC: SuggestionResult

Loading

_{Reviews (3): Last reviewed commit: "Regenerate Cotabby.xcodeproj to match pr..." | Re-trigger Greptile}

[FuJacob]

Thanks for this!! this is definitely gonna help a lot