feat: Support $$() with all element's matchers#1990
Conversation
811216d to
62a200b
Compare
| - name: Run All Checks | ||
| run: npm run checks:all |
| return aliasFn.call(this, toExist, { verb: 'be', expectation: 'existing' }, el, options) | ||
| this.verb = 'be' | ||
| this.expectation = 'existing' | ||
| this.matcherName = 'toBeExisting' |
There was a problem hiding this comment.
Fixed bad matcher name
$$ aka ElementArray$$ with all matchers
$$ with all matchers$$() with all matchers
$$() with all matchers$$() with all element's matchers
| { | ||
| ignoreCase = false, | ||
| trim = false, | ||
| trim = true, |
There was a problem hiding this comment.
Fixed issue where single expect value vs multiple expected value behavior differently by default
| // If no options passed in + children exists | ||
| if ( | ||
| typeof options.lte !== 'number' && | ||
| typeof options.gte !== 'number' && | ||
| typeof options.eq !== 'number' | ||
| ) { | ||
| return { | ||
| result: children.length > 0, | ||
| value: children?.length | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Replaced below by ?? { gte: 1 } + test added
const numberOptions = validateNumberOptionsArray(expectedValue ?? { gte: 1 })There was a problem hiding this comment.
Renaming from toHaveClass
f8f6ba0 to
78a6cf0
Compare
8991b28 to
2c40ce5
Compare
cd23183 to
aec39c2
Compare
cf071bc to
860dedc
Compare
|
What is the status of this? |
|
I was waiting for at least some feedback on the defined behaviour here: https://github.com/webdriverio/expect-webdriverio/blob/860dedcc78c9ede976eae187fdd6a44cdf8b2c4e/docs/MultipleElements.md I moved back to draft and update the list of tasks
|
860dedc to
2bde717
Compare
- Fix double negation in `Expected` with `isNot` when using `enhanceErrorBe` - Fix incorrect Received value with `isNot` when using `enhanceErrorBe` - Code refactoring simplify to better understand the code Ensure mocks used represented wdio framework reality fix bad command Working case of $$ aka ElementArray with `toBeDisplayed` Use Promise.all for ElementArray instead of waiting one by one Add UTs for ElementArray and Element[] Review after rebase + add awaited element case fix regression while trying to support Element[] Code review Add assertions of `executeCommandBeWithArray` and `waitUntil` All to be matchers now compliant with multiple elements + UTs fix rebase Fix ChainablePromiseElement/Array not correctly considered Even if the typing was passing using `'getElement' in received` was not considering Chainable cases Add coverage Enhance toHaveText test cases Working cases of toHaveWidth and toHaveHTML with $$() Ensure all test dans can run fast with `wait: 1` Speed-up unit tests runs Speed-up test execution Add coverage on `executeCommandWithArray` and support edge case Review error message assertions and discover a problem Have failure messages better asserted toHaveAttribute supporting $$() now Make all `toHave` matcher follow same code patterns Align code to use same code pattern and fix case of `Element[]` not working Remove `executeCommand` and simplify executeCommandWithArray + coverage Align typing with expected being an array + trim by default for arrays Support of `toHaveHeigth`, `toHaveSize` & `toHaveWidth`` fix UTs Working support of `$$()` in all element matchers Deprecate `compareTextWithArray` & remove toHaveClassContaining fix possible regression around NumberOptions wait & internal not considered Forget toHaveStyle + code review Code review better function naming Code review Code review + remove `toHaveClass` deprecated since v1, 4 versions ago Support unknown type for `toHaveElementProperty` since example existed doc: official support of `$$()` Review & add coverage + discover problem with isNot fix for other PR, waitunitl + toBeerror - to revert later? fix `.not` cases - Ensure isNot is correct following the backport of other PR fixes - Ensure for multiple elements the not is apply on each element and fails if any case fails - Fix/consider empty elements as an error at all times Finish matchers UT's refactor to be mocked and type safe - Review all matcher UTs to call there real implementation with a this object ensuring the implementation has the right type - Use vi.mocked, to ensure we mock with the proper type Missed some bind Review waitUntil + coverage Better documentation Code review Reivew waitUntil and fixes multiple not tests Code review + make Be/Browser matchers test type safe Mock default option to speed up tests Better way to speed-up tests Add tests around default options Fix $$ mock Ensure matcherName is corectly passed + toExists test correctly Reinforce and test `isElementArray` Add missing coverage for `executeCommand` Migrate test of toBeArraySize + add more robust util - Migrate + add coverage for `toBeElementsArrayOfSize` matcher - Fix wait not correctly considered in `toBeElementsArrayOfSize` - Reinforce isElementArray and similar + add more coverage - By default for `DEFAULT_OPTIONS`use a non 0 wait time - Fix global mock missing element.parent Add note on element not found + add potential case to support Add edge cases Add alternative with parametrized in aPI doc Gracefully fails on invalid element types With selector fixed add back Promise of elements case Code review Code review + add unsupported type to toBe matchers Code review Add unsupported type test coverage Code review + add coverage + better awaitElement mechanism Add asymmetric integration tests Code review & coverage Add more tests for toHaveAttribute Array of array is not supported so adapting test for today Review some TODOs Support better failure msg for multiple results in toBe Matchers Properly handle failure colored message for `.no` for multiple elementst Properly support equal for NumberOptions and .not multiple values failure Code review Review coverage for formatMessage + numberOptions Fix 0 not stringily correctly Add .not elements integration tests Add coverage Review docs Finalize `executeCommandBe` tests Increase coverage More stable tests test Add `toHaveText` non-indexed + non-strict length legacy behavior Test more unknown expected type, but maybe some bug? Use supported type instead of unknown for `toHaveElementProperty` Add case of element not found which throws for single element - Note that for multiple element so ElementArray, there is no exception but an empty array Add missing element case and index out of bound from `$()[x]` Review refresh test after rebase - Ensure we return non modified args elements Review waitUntil implementation Fix bad mocks
d69a855 to
bf769ed
Compare
Greptile SummaryThis PR adds official
Confidence Score: 3/5The core matcher dispatch and waiting logic work correctly for the common cases, but several edge-case contracts are subtly broken. The
|
| Filename | Overview |
|---|---|
| src/util/executeCommand.ts | Refactored to a generic two-strategy pattern (single vs multiple elements); contains a persistent typo mutipleElementsCompareStrategy throughout the file |
| src/util/elementsUtil.ts | New element/array resolution helpers; isElementArrayLike incorrectly returns true for any empty plain array due to vacuous truth in [].every(isElement) |
| src/util/waitUntil.ts | New standalone waitUntil implementation; no longer throws on timeout (silent return instead), and handles both boolean and ConditionResult correctly for .not semantics |
| src/util/numberOptionsUtil.ts | New NumberMatcher class cleanly encapsulates eq/gte/lte comparison logic; numberMatcherTester integrates correctly with jest-matcher-utils equality checks |
| src/matchers/element/toHaveElementProperty.ts | Now supports arrays of elements; removes special value === undefined branch so property-existence checks (no value arg) produce a confusing Expected: undefined error, despite d.ts still marking value as optional |
| src/matchers/element/toHaveAttribute.ts | Correctly migrated to new executeCommand pattern; options not forwarded to toHaveAttributeFn for existence-only check (pre-existing behaviour, but now more visible) |
| src/util/formatMessage.ts | Enhanced with per-element diff formatting for .not with arrays; enhanceErrorBe now maps per-element pass/fail for clear error output |
| src/matchers/element/toHaveText.ts | Legacy compareTextWithArray behaviour preserved for single-element with array; multi-element now uses custom strategy and correctly separates single vs array paths |
| src/matchers/element/toHaveChildren.ts | Migrated to NumberMatcher; default no-arg behaviour preserved (gte: 1 is equivalent to the previous > 0 for integer child counts) |
| src/matchers/elements/toBeElementsArrayOfSize.ts | Migrated to NumberMatcher and awaitElementArray; correctly handles both awaited and non-awaited element arrays; refetch logic preserved |
| types/expect-webdriverio.d.ts | Good expansion of type signatures to MaybeArray; toHaveElementProperty value still marked optional (?) while implementation no longer handles undefined properly |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["expect(received).toHaveMatcher(expected)"] --> B["waitUntil(condition, isNot, opts)"]
B --> C["executeCommand(received, singleStrategy?, multiStrategy?)"]
C --> D["awaitElementOrArray(received)"]
D --> E{Type?}
E -->|Promise| F[await Promise]
F --> E
E -->|Element| G["singleStrategy(element)"]
E -->|ElementArray or Element array| H{Has singleStrategy?}
E -->|empty array| I["return success:false"]
E -->|other| J["return success:false"]
H -->|No| K["multiStrategy(elements)"]
H -->|Yes| L["map elements with singleStrategy"]
G --> M["results with 1 entry"]
K --> N["defaultMultipleElementsIterationStrategy or custom"]
L --> M
N --> O{expected is array?}
O -->|Yes, lengths match| P["compare el-i vs expected-i"]
O -->|Yes, lengths differ| Q["result: false"]
O -->|No| R["compare each el vs expected"]
P --> S["results array"]
R --> S
M --> T["success = all results true"]
S --> T
T --> U["waitUntil checks isNot semantics"]
U --> V["enhanceError for message"]
V --> W["AssertionResult with pass and message"]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A["expect(received).toHaveMatcher(expected)"] --> B["waitUntil(condition, isNot, opts)"]
B --> C["executeCommand(received, singleStrategy?, multiStrategy?)"]
C --> D["awaitElementOrArray(received)"]
D --> E{Type?}
E -->|Promise| F[await Promise]
F --> E
E -->|Element| G["singleStrategy(element)"]
E -->|ElementArray or Element array| H{Has singleStrategy?}
E -->|empty array| I["return success:false"]
E -->|other| J["return success:false"]
H -->|No| K["multiStrategy(elements)"]
H -->|Yes| L["map elements with singleStrategy"]
G --> M["results with 1 entry"]
K --> N["defaultMultipleElementsIterationStrategy or custom"]
L --> M
N --> O{expected is array?}
O -->|Yes, lengths match| P["compare el-i vs expected-i"]
O -->|Yes, lengths differ| Q["result: false"]
O -->|No| R["compare each el vs expected"]
P --> S["results array"]
R --> S
M --> T["success = all results true"]
S --> T
T --> U["waitUntil checks isNot semantics"]
U --> V["enhanceError for message"]
V --> W["AssertionResult with pass and message"]
Reviews (1): Last reviewed commit: "ish stable unit test with some skipped" | Re-trigger Greptile
| return compareText(prop.toString(), expected as string | RegExp | AsymmetricMatcher<string>, options) | ||
| } | ||
|
|
||
| export async function toHaveElementProperty( | ||
| received: WdioElementMaybePromise, | ||
| received: WdioElementOrArrayMaybePromise, | ||
| property: string, | ||
| value?: string | RegExp | AsymmetricMatcher<string> | null, | ||
| expectedValue: MaybeArray<string | number | null | RegExp | AsymmetricMatcher<string>>, | ||
| options: ExpectWebdriverIO.StringOptions = DEFAULT_OPTIONS | ||
| ) { | ||
| const isNot = this.isNot | ||
| const { expectation = 'property', verb = 'have' } = this | ||
| const { expectation = 'property', verb = 'have', isNot, matcherName = 'toHaveElementProperty' } = this | ||
|
|
||
| await options.beforeAssertion?.({ | ||
| matcherName: 'toHaveElementProperty', | ||
| expectedValue: [property, value], | ||
| matcherName, | ||
| expectedValue: [property, expectedValue], | ||
| options, | ||
| }) | ||
|
|
||
| let el = await received?.getElement() | ||
| let el | ||
| let prop: unknown | ||
| const pass = await waitUntil( |
There was a problem hiding this comment.
Property-existence check (
toHaveElementProperty(el, 'prop') without a value) still fails silently, and the error message now shows undefined as expected
The public type in d.ts still marks value as optional (value?:), so users expect expect(el).toHaveElementProperty('checked') to verify that the property exists. In the new code expectedValue lands as undefined in defaultMultipleElementsIterationStrategy, which calls condition(el, undefined, options). Inside condition, the expected === null guard is not reached (because null !== undefined), so for non-string properties the comparison falls through to prop === undefined, which is always false for any defined prop. The old code had a dedicated value === undefined branch that produced a descriptive error message; the new code removed it and now the error shows Expected: undefined which is confusing. Either re-add the undefined guard in condition, or remove the ? from the type declaration to make the breaking change explicit.
There was a problem hiding this comment.
TODO dprevost to review later to allow checking prop existence with a command options without passing undefined
toHaveElementProperty(proName', 'expectValue', {wait: 1})` signature
There was a problem hiding this comment.
The signature conflict is a real TypeScript challenge — if expectedValue is optional, then toHaveElementProperty('prop', { wait: 1 }) becomes ambiguous: is { wait: 1 } the expected value or the options? TypeScript would bind it to expectedValue.
The cleanest resolution without breaking the positional API is function overloads:
| return compareText(prop.toString(), expected as string | RegExp | AsymmetricMatcher<string>, options) | |
| } | |
| export async function toHaveElementProperty( | |
| received: WdioElementMaybePromise, | |
| received: WdioElementOrArrayMaybePromise, | |
| property: string, | |
| value?: string | RegExp | AsymmetricMatcher<string> | null, | |
| expectedValue: MaybeArray<string | number | null | RegExp | AsymmetricMatcher<string>>, | |
| options: ExpectWebdriverIO.StringOptions = DEFAULT_OPTIONS | |
| ) { | |
| const isNot = this.isNot | |
| const { expectation = 'property', verb = 'have' } = this | |
| const { expectation = 'property', verb = 'have', isNot, matcherName = 'toHaveElementProperty' } = this | |
| await options.beforeAssertion?.({ | |
| matcherName: 'toHaveElementProperty', | |
| expectedValue: [property, value], | |
| matcherName, | |
| expectedValue: [property, expectedValue], | |
| options, | |
| }) | |
| let el = await received?.getElement() | |
| let el | |
| let prop: unknown | |
| const pass = await waitUntil( | |
| export async function toHaveElementProperty( | |
| received: WdioElementOrArrayMaybePromise, | |
| property: string, | |
| expectedValue: MaybeArray<string | number | null | RegExp | AsymmetricMatcher<string>>, | |
| options?: ExpectWebdriverIO.StringOptions | |
| ): Promise<ExpectWebdriverIO.MatcherResult> | |
| export async function toHaveElementProperty( | |
| received: WdioElementOrArrayMaybePromise, | |
| property: string, | |
| options?: ExpectWebdriverIO.StringOptions | |
| ): Promise<ExpectWebdriverIO.MatcherResult> | |
| export async function toHaveElementProperty( | |
| received: WdioElementOrArrayMaybePromise, | |
| property: string, | |
| expectedValueOrOptions?: MaybeArray<string | number | null | RegExp | AsymmetricMatcher<string>> | ExpectWebdriverIO.StringOptions, | |
| options: ExpectWebdriverIO.StringOptions = DEFAULT_OPTIONS | |
| ) { |
Then at runtime you disambiguate:
const isOptions = (v: unknown): v is ExpectWebdriverIO.StringOptions =>
v !== null && typeof v === 'object' && !Array.isArray(v) && !(v instanceof RegExp)
&& ('wait' in v || 'interval' in v || /* other option keys */ true)
const expectedValue = isOptions(expectedValueOrOptions) ? null : expectedValueOrOptions
const resolvedOptions = isOptions(expectedValueOrOptions) ? expectedValueOrOptions : optionsThis gives users all three forms:
toHaveElementProperty('checked')— existence checktoHaveElementProperty('checked', { wait: 5000 })— existence check with optionstoHaveElementProperty('value', 'foo', { wait: 5000 })— value + options
Agree it's worth deferring until you also tackle the undefined guard in condition so the existence path is consistent.
|
|
||
| if (error) { throw error } | ||
|
|
||
| // When no results were found, ensure the waitUntil return failure even with `.not` | ||
| if (!isBoolean(result) && result.results.length === 0) { | ||
| // To fails with .not, we need pass=true, so it's inverted later by Jest's expect framework | ||
| return isNot | ||
| } | ||
|
|
There was a problem hiding this comment.
waitUntil no longer throws on timeout; it silently returns the last pass value
The previous waitUntil threw an Error('timeout') after exhausting wait, which the outer try/catch caught to return isNot. The new implementation exits the do/while loop when canWait becomes false and falls through to return pass (the last computed value). For the typical success path this is fine, but any code that previously relied on a thrown timeout to distinguish "timed out" from "condition last returned false" will no longer see a thrown error. Tests or retry wrappers that catch the timeout error to log or rethrow it will silently break.
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
… fix-matchers-with-$$
fcbd425 to
e6ae7ee
Compare
Fixes #1507.
Partially fixes #512.
Fixes #1717.
Summary
Adds official
$$()(element array) support totoBeandtoHavematchers. Previously, TypeScript signatures allowed arrays (by mistake in this PR), but the implementation didn't support them properly.Example:
Current Behaviour Issues
toHaveTextandtoBeElementsArrayOfSizeofficially support arrays$$()or filtered$$().filter()throws errorstoHaveTextwith empty elements incorrectly passestoHaveTextdoesn't trim text for multiple elements (inconsistent with single element behaviour)toHaveTextdoesn't do strict and index-based comparison but only loose comparison (kept)Error handling
$$returns only one element and we have one expected value, the error message (CHANGED)$$returns only one element and an array of expectations is passed, the error message (CHANGED)Note: All the above have been changed to show all the elements' values and not just those not matching
Official
$$()SupportThis PR adds official support for most matchers, with a focus on the
toBeandtoHaveelement matchers.$$()support may incidentally enableexpect()to work with multi-remote, this is not intended and may break at any time. Official multi-remote support is tracked here and is not yet available.Types Support
Behavior
The following must pass when all elements are displayed/have the HTML; otherwise, it fails.
toBematchers, all elements must match the expected boolean (usually true, except for toBeDisabled).toHavematchers, you can provide a single expected value or an array; strict array comparison is used.StringOptions,HTMLOptions,ToBeDisplayedOptionsapply to the whole array (not per element).NumberOptionscan be provided as an array since it's the only "option" treated as expected values.Array Comparison Behaviour
{ trim: false }).toHaveText(deprecated), elements are not compared to any value in the expected array—only by index.isNot
The following must pass when all elements are not displayed/not have the text; otherwise, it fails.
Edge cases
No elements found
When no elements are found, we fail at all times with or without
.not, even if the expected is an empty array.toBeElementsArrayOfSize(0)support empty array elementexpect.arrayContaining
Only
toHaveTextwill do acontainingarray behaviour with the followingWe should consider deprecating the above for
expect.arrayContainingand supporting it, which is not the case at allError handling
Below are examples of failures with colours.
toHaveTextandtoBeDisplayedmatchers.nottoBeare handled by addingnotin the valuestoHavematchers, a more complex method was used to highlight those actuall matching (red highlight)Previous Releases Bugs 🐛
Fixed bugs from previous releases (fixed when checked)
$$()calls likeexpect($$('el')).toHaveText('text')no longer cause function errors.toBeElementsArrayOfSizetypings now work correctly withElement[].toHaveElementPropertyadjusted—using an optional value always failed; future support for existence checks liketoHaveAttributemay be added.$$().toHaveText('C' | ['C'])now fails when there are no elements (e.g.,expect([]).toHaveText('test')); all matchers fail gracefully on empty arrays.beforeandafterhooks.toHaveClassContainingandtoHaveClassmatchers.expect()(e.g.,expect(undefined)) now fail gracefully with clear messages.Future Considerations
$$()(compare each element against multiple values), but this is complex, limited to$$(), and alternatives exist (e.g., regex, stringContaining).$()[x]; POC done and working.Promise<Element|Elements>in typings.expect.arrayContaining()across all matchers and remove custom “containing” logic fromtoHaveText.toBeElementsArrayOfSize.ts, consider updating the array of the non-awaited case by awaiting ittoHaveElementPropertytoHaveAttributesupport properly optional expected value for property existenceTODO
toHaveTextsomehow now or later?toHaveTextlegacy behaviour is kept, including failure reporting!waitUntil.notin waitUntil following regression in mainBlocked and waiting on the merge of
waitUntil#1983enhanceErrorBe+ code refactoring simplification #1987withContextand add Playgrounds for Jasmine, Jest & Mocha #2010Not blockers, but good to have dependent PR