Treat ^ and v as operators even without surrounding whitespace - #639
Conversation
parse_boolean_expr has a fast-path that returns the whole string as a single variable name when it contains no operator. It only checked for "!", not the other two classical operators "^" (and) and "v" (or), so an unspaced guard like cond="a^b" skipped operator replacement and was looked up as a variable named "a^b" -- silently evaluating wrong (or raising InvalidDefinition on a StateChart). "!a" already worked unspaced, so this was an inconsistency. Gate the fast-path on the module's existing operator `pattern` (which already matches !, ^ and the word-bounded v) instead of a hand-rolled "!" check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: chuenchen309 <48723787+chuenchen309@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #639 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 52 52
Lines 5505 5507 +2
Branches 869 869
=========================================
+ Hits 5505 5507 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
fgmacedo
left a comment
There was a problem hiding this comment.
Hi @chuenchen309, thanks a lot for this contribution, and congratulations on your first PR to the project! 🎉
Great catch: the fast-path really does mishandle unspaced ^/v, and your diagnosis is spot on. While reviewing I found that gating on pattern introduces two behavior regressions that our test suite doesn't cover (which is why CI stayed green). Details and a suggested alternative inline.
Could you also add a short entry about the fix to the open release notes under docs/releases/? We document every user-facing bugfix there.
| ]: | ||
| got = parse_boolean_expr(unspaced, variable_hook, operator_mapping)() | ||
| want = parse_boolean_expr(spaced, variable_hook, operator_mapping)() | ||
| assert got is want, unspaced |
There was a problem hiding this comment.
Nice test, thanks for including it. Since CI stayed green despite the regressions mentioned in spec_parser.py, could you extend the coverage here to pin the behaviors that must keep working? These two cases would have caught them:
def test_unspaced_not_equal_is_a_comparison():
# "!" is an operator, but "!=" is not a negation: it must keep parsing
# as a comparison even without surrounding whitespace.
got = parse_boolean_expr("frodo_age!=51", variable_hook, operator_mapping)()
want = parse_boolean_expr("frodo_age != 51", variable_hook, operator_mapping)()
assert got is want is True
def test_bare_v_is_a_variable_name():
# "v" is only an operator between operands; a lone "v" is a plain variable.
expr = parse_boolean_expr("v", variable_hook, operator_mapping)
assert expr.__name__ == "v"Note the !=51 (not !=50): a misparse resolves the whole string as an unknown variable, which this variable_hook defaults to False, so a !=50 case would pass for the wrong reason.
The fast-path in `parse_boolean_expr` returned the whole expression as a
single variable name whenever it found no `!`, no space and no `In(`. Any
other operator written without surrounding whitespace was swallowed into a
variable name, so `cond="a^b"` or `cond="items>0"` were resolved as variables
literally named `a^b` / `items>0`.
Gating on the module's `pattern` (as originally proposed) fixes `^`/`v` but
regresses two working cases: unspaced `!=` (skipped by the `\!(?!=)`
lookahead) stops parsing as a comparison, and a guard literally named `v`
starts being replaced by ` or `.
The invariant the fast-path wants is "the whole string is a single variable
name", which `str.isidentifier()` checks directly. This keeps `!=` and a bare
`v` working, makes the `" "` / `"In("` checks redundant, and also fixes the
sibling bug where unspaced comparisons like `x==1` were swallowed too.
Expressions with a structure that is invalid in a boolean context (e.g.
`cond="user.age"`) now reach the AST allowlist and raise `ValueError`, so
`Listeners.build` catches it alongside `SyntaxError` to keep reporting the
friendly `InvalidDefinition`.
Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
|
Hi @chuenchen309, thanks again for spotting this one. Since the PR has been idle for a couple of weeks and I'd like to ship it in 3.2.1, I pushed the review follow-ups directly to your branch (your original commit is untouched, so the fix stays credited to you). What I applied, in a0a19a5:
Full suite is green (1947 passed) with 100% branch coverage, plus ruff, mypy and pyright. Merging once CI goes green. Thanks for the contribution! |
Catching a bare `ValueError` around `parse_boolean_expr` was too broad: the `variable_hook` runs user code while resolving names (`search_name` reads attributes from the model, which may execute a property), so an unrelated `ValueError` raised there was reported as `InvalidDefinition: Failed to parse boolean expression '<name>'`, hiding the real cause. The parser now raises `UnsupportedExpression` (a `ValueError` subclass, so `io.evaluators` and user code catching `ValueError` keep working) when a node kind is rejected by the allowlist, and `Listeners.build` catches only that. Errors from name resolution propagate untouched. Also parametrizes the unspaced-operator tests, adding a falsy case, and drops an inaccurate claim from the release notes: a name that is not a valid identifier can never be silently resolved, since `search_name` only matches names present in `dir(obj)`. Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
|
Follow-up on my own change: I reviewed a0a19a5 again and the The The parser now raises Same commit also parametrizes the unspaced-operator tests (your two cases are the first entries, plus a falsy case and the two comparison cases) and drops an inaccurate sentence from the release note. I also checked the security side, since the change routes more strings into the parser: |
SonarCloud python:S5778: the `with pytest.raises(...)` block held two calls that could raise (`Model()` and the state machine constructor), so the test could pass for the wrong reason if the model constructor started raising `ValueError` on its own. Build the model outside the block. Signed-off-by: Fernando Macedo <fgmacedo@gmail.com>
|



parse_boolean_exprhas a fast-path that returns the whole expression as asingle variable name when it looks operator-free. It only checked for
!, notthe other two documented classical operators
^(and) /v(or), so anunspaced guard skipped operator replacement and was resolved as a variable:
a^bthen evaluates wrong (or raisesInvalidDefinition: Did not found name 'a^b'at class-definition time). Since!aalready worked without spaces, theinconsistency is surprising — users copy operators from the guards docs, where
^/vare listed with no whitespace requirement.Gate the fast-path on the module's existing
pattern(which already matches!,^, and the word-boundedv) instead of the hand-rolled"!" not in exprcheck. Genuine single-name variables like
avc(no word-boundaryv) stilltake the fast-path. Added a regression test asserting the unspaced forms match
their spaced equivalents.
This PR was authored by an AI coding agent (Claude Code) running on this account:
the AI found the bug, ran the repro, wrote the test, and wrote this description.
The human account holder reviews every change and is accountable for it. The
verification is real and re-runnable from the diff. If this isn't the kind of
contribution you want, say so and I'll close it.
Update (maintainer follow-up, a0a19a5): the fast-path is now gated on
expr.isidentifier()instead of the modulepattern. That keeps the reporteda^b/(a)v(b)fix while preserving unspaced!=and a guard literally namedv, and it also fixes the sibling bug where unspaced comparisons (x==1,x>=1) were swallowed as variable names.Listeners.buildnow catchesValueErroralongsideSyntaxErrorso expressions likecond="user.age"keepraising the friendly
InvalidDefinition. Regression tests and a release note indocs/releases/3.2.1.mdwere added.