feat: grammar-based value completion behind the experimental flag (#467 #343)#477
Open
SJrX wants to merge 2 commits into
Open
feat: grammar-based value completion behind the experimental flag (#467 #343)#477SJrX wants to merge 2 commits into
SJrX wants to merge 2 commits into
Conversation
#343) Cashes in the frontier/expected-set: at the caret, suggest the literal/choice tokens the grammar could accept next. - Combinator.nextTokenChoices(prefix): pure FIRST-set computation — the enumerable choices (LiteralChoiceTerminal / FlexibleLiteralChoiceTerminal) expected at the end of `prefix`, read from the Stuck values at that offset. No IntelliJ types; bounded by maxSteps. - UnitFileValueCompletionContributor: when useGrammarParseEngine is on and the option is a GrammarOptionValue, compute completions from the value text before the caret. It scans split points (largest first) for the tightest word where the grammar expects an enumerable token matching what's typed, then uses resultSet.withPrefixMatcher(word) so the platform filters correctly — otherwise a partial token like "~AF_IN" would be the prefix and match nothing. Falls back to the original getAutoCompleteOptions path otherwise. Polls ProgressManager.checkCanceled between split attempts. Tests: NextTokenChoicesTest (FIRST set incl. context — after "~" offers families not "none"; after "AF_INET " offers another family); GrammarValueCompletionTest drives real completion with the flag on (partial family, and a subsequent family in a list). Full suite green. Refs #467 #343 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first cut used "largest split wins", so for a lenient terminal a partial token
like RootImagePolicy=h was treated as a finished partition identifier and completion
offered "=" instead of "home" (and AF_IN-style cases only worked by luck).
Rework addGrammarCompletions to prefer completing a genuine partial: the longest
non-empty trailing word for which the grammar expects an enumerable choice that
STRICTLY extends it (something left to type). Only if there's no such partial do we
fall back to advancing at a fresh boundary (empty value, or after a complete token
like "~" or "root=") with an empty prefix.
This keeps the previously-working cases (empty value, after "partition=", "AF_IN")
and fixes partial-identifier completion. Adds ImagePolicyCompletionTest covering the
empty boundary, partial-identifier completion ("r" -> root/root-verity/...), and the
post-"root=" policy-flag position.
Refs #467 #343
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 21, 2026
SJrX
added a commit
that referenced
this pull request
Jun 26, 2026
* feat: grammar-based value completion behind the experimental flag
The frontier built for error localization answers exactly the question
completion needs — "what was expected at this offset?" is "what could come next
here?". Adds Combinator.nextTokenChoices(prefix), which reads the Stuck values at
the end of the typed prefix and collects the enumerable choices expected there
(literal / flexible-literal terminals; numbers/regexes/whitespace contribute
nothing to suggest).
Wires it into UnitFileValueCompletionContributor, gated behind
ExperimentalSettings.useGrammarParseEngine AND validator is GrammarOptionValue:
with the flag off the existing completion path is untouched. Two behaviours:
- completing a partial token (sets the prefix matcher so "AF_IN" -> AF_INET/6)
- chaining through forced separators on accept (e.g. accepting "home" inserts
"home=" and re-opens completion, never auto-inserting a content token)
Bundles the stacked PRs #477 (completion) and #478 (forced-separator chaining),
re-cut against current 242.x. New behaviour, so flag-gated; correctness of
existing validators is unchanged. Tests enable the flag and reset it in tearDown
to avoid leaking into the shared light-test project.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: cover the flag-off gating and non-enumerable completion cases
The completion bundle tested the flag-on behaviours but not two things worth
guaranteeing: that grammar completion stays suppressed when the experimental
engine is off (the gating contract), and that a non-enumerable next token
(number/regex) offers nothing rather than crashing or suggesting junk.
- GrammarValueCompletionTest.testFlagOffOffersNoGrammarCompletions: flag off, the
AF_IN partial yields no AF_* names (the flag-on path's suggestions are absent).
- NextTokenChoicesTest.testNonEnumerableNextTokenOffersNothing: an IntegerTerminal
port grammar returns an empty choice set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Steve Ramage <gitcommits@sjrx.net>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The payoff of the frontier work: grammar-based autocomplete. With the experimental flag on, the value completion for any
GrammarOptionValuesuggests the literal/choice tokens the grammar could accept at the caret — e.g. typingRestrictAddressFamilies=AF_IN⎉offersAF_INET/AF_INET6.How
Combinator.nextTokenChoices(prefix)(pure, no IntelliJ): the FIRST set at the end ofprefix— the enumerable choices (LiteralChoiceTerminal/FlexibleLiteralChoiceTerminal) from theStuckvalues at that offset. "What was expected here" (errors) and "what could come next" (completion) are the same question.UnitFileValueCompletionContributor(kept on the simpleextend()path): when the flag is on and the option is grammar-backed, read the value text before the caret and scan split points (largest first) for the tightest word where the grammar expects an enumerable token matching what's typed; thenresultSet.withPrefixMatcher(word)+ add the choices.Why the custom prefix matcher
The platform's default prefix is the leaf text up to the caret. For a partial token like
~AF_INthat prefix is~AF_IN, and no family name starts with it, so everything gets filtered out. Computing the grammar-correct word (hereAF_IN) and setting it viawithPrefixMatcherfixes this without grammar-specific word-splitting heuristics. This was the subtlety flagged before starting.Tests
NextTokenChoicesTest(pure): empty value offersnone/~/families; after~offers families but notnone/~; afterAF_INEToffers another family (context-aware).GrammarValueCompletionTest(fixture, flag on):AF_IN⎉→AF_INET/AF_INET6; and a subsequent family in a listAF_UNIX AF_IN⎉→ same.Full suite green; cancellation is polled (
ProgressManager.checkCanceled) between split attempts.Scope / limits
COMPLETED_VALUEleaves (existing contributor limitation); values mid-line-continuation aren't covered. Fine for the common case.Refs #467 #343
🤖 Generated with Claude Code