Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,10 @@ jobs:
test-browsers:
name: Test on Browsers (${{ matrix.browser }})
runs-on: ubuntu-latest
timeout-minutes: 20
timeout-minutes: 25
strategy:
# A flake in one browser must not cancel the others, or its cause is lost.
fail-fast: false
matrix:
browser: [edge, chrome, firefox]

Expand All @@ -119,8 +121,36 @@ jobs:
- name: Setup
uses: ./.github/actions/setup

# Chromium under the webdriverio provider sometimes loses a test iframe: the run either fails
# at once with "Cannot connect to the iframe" / "Failed to fetch dynamically imported module",
# or prints nothing more until it is killed. A whole run takes 1 to 3 minutes, so an attempt
# that times out, or fails with one of those two errors, is run again, up to three attempts.
# The iframes are lost when many test files load at once, so the retries run one file at a
# time. Any other failure, including a failing assertion, fails the step at once.
- name: Run tests in ${{ matrix.browser }}
run: vp test --browser.enabled --browser.name=${{ matrix.browser }}
env:
BROWSER: ${{ matrix.browser }}
run: |
attempt() {
timeout "$1" vp test --browser.enabled --browser.name="$BROWSER" "${@:2}" 2>&1 | tee browser-tests.log
return "${PIPESTATUS[0]}"
}

lost_iframe() {
[ "$1" -eq 124 ] || grep -qE "Cannot connect to the iframe|Failed to fetch dynamically imported module" browser-tests.log
}

attempt 300 && exit 0
status=$?

for retry in 1 2; do
lost_iframe "$status" || exit "$status"
echo "::warning::The $BROWSER test run lost a test iframe (exit $status); retry $retry of 2, one file at a time."
attempt 420 --no-file-parallelism && exit 0
status=$?
done

exit "$status"

test-safari:
name: Test on Browsers (safari)
Expand Down
Loading