Salvage stale completions on type-ahead (behind a flag)#435
Conversation
When a prediction finishes generating after the user has kept typing, the result was generated against an older prefix and the coordinator drops it on the generation-number guard. This adds a salvage path: trim the characters typed during the race off the front of the continuation and render the remainder instead of showing nothing. The behavior is gated behind a hidden, default-off setting (cotabbyStaleCompletionSalvageEnabled). The salvage opportunity is logged as "stale-salvageable" even when the flag is off, so the rescue rate is measurable from the logs before the path is enabled. - StaleCompletionReconciler: pure trim plus suffix/prefix overlap rules - apply() now receives the request context and attempts salvage at the stale-drop point, reusing the existing process/selection/suffix guards - Settings flag plumbed through the snapshot and its publisher - Unit tests for the pure reconciler; project regenerated via XcodeGen
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| let overlap = longestSuffixPrefixOverlap(suffix: typed, prefix: continuation) | ||
| if overlap >= minimumOverlap { | ||
| return finalize( | ||
| String(continuation.dropFirst(overlap)), | ||
| typedSinceRequest: typed, | ||
| confidence: .overlap | ||
| ) | ||
| } |
There was a problem hiding this comment.
finalize rejects the trimmed tail (e.g. the tail is whitespace-only because the user's typed suffix covers the entire non-whitespace portion of the continuation), reconcile returns nil without trying any shorter overlaps that passed elementsEqual. In practice this is rare — it requires the typed text's trailing segment to align exactly with all non-whitespace content in the continuation — but it means a valid shorter salvage is silently skipped. Consider returning the best valid (non-nil) finalize result rather than only trying the longest overlap.
| let overlap = longestSuffixPrefixOverlap(suffix: typed, prefix: continuation) | |
| if overlap >= minimumOverlap { | |
| return finalize( | |
| String(continuation.dropFirst(overlap)), | |
| typedSinceRequest: typed, | |
| confidence: .overlap | |
| ) | |
| } | |
| let overlap = longestSuffixPrefixOverlap(suffix: typed, prefix: continuation) | |
| if overlap >= minimumOverlap { | |
| if let reconciled = finalize( | |
| String(continuation.dropFirst(overlap)), | |
| typedSinceRequest: typed, | |
| confidence: .overlap | |
| ) { | |
| return reconciled | |
| } | |
| // Longest overlap produced an empty/whitespace tail; fall through to nil. | |
| } |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Resolves the SuggestionSettingsModel conflict against main (keeping the emoji customization settings) and removes the isStaleCompletionSalvageEnabled flag entirely, so salvaged stale completions are always presented (no longer gated).
| guard liveContext.generation == result.generation else { | ||
|
|
||
| latestRawModelOutput = SuggestionDebugLogger.debugPreview(result.rawText) | ||
|
|
||
| if let salvage = salvagedStaleCompletion( | ||
| result: result, | ||
| requestContext: requestContext, | ||
| liveContext: liveContext | ||
| ) { | ||
| presentSalvagedCompletion(salvage, result: result, liveContext: liveContext, workID: workID) | ||
| return | ||
| } |
There was a problem hiding this comment.
The PR description explicitly states the salvage path is "gated behind a hidden, default-off setting" (cotabbyStaleCompletionSalvageEnabled), that with the flag off the only behavior change is an extra stale-salvageable log line, and that the setting is read into the settings snapshot at launch. Neither the flag nor the stale-salvageable log stage appear anywhere in the codebase: SuggestionSettingsModel, SuggestionSettingsSnapshot, and SuggestionSubsystemContracts.swift all have zero references to it.
As written, whenever salvagedStaleCompletion returns a non-nil value the salvaged completion is presented unconditionally, making the salvage path active for every user from the first install of this build. The staged roll-out and instrumentation period described in the risk notes never happen.
1 participant
Summary
Inline autocomplete has an unavoidable race: a request is built from the prefix at request time, the model decodes for some milliseconds, and a fast typist can append characters before the result lands. Today the coordinator's generation-number guard drops any result whose prefix changed during decode, so those keystrokes produce no suggestion at all (the
stale-droppath inapply).This adds a salvage path. When a stale result comes back, it trims the characters typed during the race off the front of the continuation and renders the remainder instead of nothing. A request built from "thanks for meet" whose model returned "ing with me today" still shows " with me today" after the user has typed "ing".
The salvage is gated behind a hidden, default-off setting. Even while it is off, the coordinator logs the salvage opportunity, so we can measure how many
stale-drops the path would rescue before enabling it for anyone.Validation
The 14 new tests cover exact trim, leading-space preservation, the suffix/prefix overlap fallback and its minimum-overlap floor, deletion past the request baseline, disjoint typing, type-through of the whole continuation, whitespace-only results, and grapheme (emoji) safety.
Linked issues
None.
Risk / rollout notes
cotabbyStaleCompletionSalvageEnabled, defaultfalse. With the flag off the only behavior change is one extra structured log line (stale-salvageable); the result is still dropped exactly as before, so this is pure instrumentation for existing users.defaults write com.jacobfu.tabby cotabbyStaleCompletionSalvageEnabled -bool YESand relaunch. The flag is read into the settings snapshot at launch and is also reactive through the snapshot publisher.stale-salvageableagainststale-dropin~/Library/Logs/Cotabby/cotabby.jsonlto get the rescue rate, split by theexactvsoverlapconfidence noted in each message.StaleCompletionReconciler. The coordinator reuses its existing process-identity, selection, and trailing-text guards before salvaging, then anchors the salvaged session at the live caret and stamps it with the live generation, so partial-acceptance reconciliation keeps working as the user types through the recovered tail.overlap-confidence salvages (a suffix of the typed text matching a prefix of the continuation) are weaker thanexactones; both are logged distinctly so the active path can be restricted toexactlater ifoverlapproves noisy.Cotabby.xcodeprojwas regenerated with XcodeGen to pick up the two new files; there is noproject.ymlchange.Greptile Summary
This PR introduces a salvage path for stale inline completions: when a model result arrives after the user has typed ahead, it trims the typed-ahead characters from the front of the continuation (exact match) or recovers via a suffix/prefix overlap heuristic rather than dropping the result entirely.
StaleCompletionReconcileris a well-designed, pure helper with grapheme-safe string operations, a configurable overlap floor, and 14 thorough unit tests covering all specified edge cases.SuggestionCoordinator+Predictionwires the reconciler in behind the existing process-identity, selection, and trailing-text guardrails.cotabbyStaleCompletionSalvageEnabled) and astale-salvageablemeasurement log stage for the flag-off path, but neither appears in any changed file — the salvage path is unconditionally active for all users from the first build that ships this change.Confidence Score: 4/5
The reconciler logic and coordinator guardrails are sound, but the salvage path is active for all users immediately rather than behind the staged roll-out described in the PR notes.
The coordinator now presents salvaged completions unconditionally for every user who gets a stale result that passes the reconciler. The PR author's own risk notes require a default-off flag and a measurement period before enabling the path for anyone, but neither the flag nor the instrumentation-only log stage were included in the code. The reconciler itself is correct, but the missing gate means the intended phased rollout cannot happen.
SuggestionCoordinator+Prediction.swift — the salvage call site needs the feature flag check and the flag-off stale-salvageable log branch. SuggestionSettingsModel.swift and SuggestionSubsystemContracts.swift need the corresponding setting property.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[apply result, requestContext, workID] --> B{workID current?} B -- No --> Z[return] B -- Yes --> C[refresh focus, get liveContext] C --> D{liveContext.generation == result.generation?} D -- Yes / fresh --> E[normal ready path: start session, present overlay] D -- No / stale --> F[set latestRawModelOutput] F --> G[salvagedStaleCompletion] G --> H{same process? no selection? trailingText unchanged?} H -- No --> L[return nil] H -- Yes --> I[StaleCompletionReconciler.reconcile] I --> J{exact prefix match?} J -- Yes --> K1[finalize trimmed tail, confidence: exact] J -- No --> K2{overlap >= minimumOverlap?} K2 -- Yes --> K3[finalize trimmed tail, confidence: overlap] K2 -- No --> L K1 --> M{tail non-empty and non-whitespace?} K3 --> M M -- No --> L M -- Yes --> N[Reconciled result] L --> O[logStage stale-drop, hideOverlay] N --> P[presentSalvagedCompletion: stamp live generation, start session, present overlay, log stale-salvage]Reviews (2): Last reviewed commit: "Merge main into feat/stale-completion-sa..." | Re-trigger Greptile