Skip to content

test: run the E2E suite for both platforms at once - #182

Merged
floyd-soomgo merged 4 commits into
masterfrom
test/e2e-concurrent-platforms
Aug 26, 2026
Merged

test: run the E2E suite for both platforms at once#182
floyd-soomgo merged 4 commits into
masterfrom
test/e2e-concurrent-platforms

Conversation

@floyd-soomgo

Copy link
Copy Markdown
Member

Background

The E2E suite could only cover one platform at a time. Everything a run wrote was shared between the two: the example app's App.tsx, which is rewritten once per release; the mock server's fixed port, served directory and request log; the record the local CLI config appends its artifacts to; the CLI's output root under the app; and the temp directory the CLI sweeps before it bundles. Two platforms at once would have overwritten each other's release markers, thrown away each other's served data, and deleted each other's bundle mid-flight.

CI never ran into this because it gives each platform a job and a runner of its own. Locally it means paying for the two platforms one after the other.

Whether the two drivers can share a machine at all was checked before any of this was written: Maestro against the iOS simulator and maestro-runner against an Android device were run ten times with their invocations interleaved. Every one passed, each stayed on its own device, and no port or lock contention showed up. The Android flow took 36.9s alone and 36.2–41.5s while the other platform was running.

Changes

Every shared resource now follows from the platform

  • The app entry is written as App.ios.tsx or App.android.tsx. Metro resolves a platform extension ahead of the plain name, so each run gets an entry the other cannot touch and App.tsx is left as the source they are patched from. The host baked into the entry differs per platform anyway, which one shared file could never carry.
  • Marker assets go to a directory of their own, because a release clears them by removing them and both platforms write the same file names. That directory's name avoids the e2e-asset prefix the files carry, since the asset assertions match archive entries on the characters both platforms keep.
  • The mock server listens on 18081 for iOS and 18082 for Android, serves a data directory of its own, and keeps its own request log. The served paths inside a data directory are unchanged, so the storage layout the CLI is asserted against stays what it was.
  • Bundling commands are passed -o build/<platform>, which stays under the build directory the example apps already ignore, and every CLI invocation runs with a TMPDIR of its own so the temp sweep and Metro's cache stay inside one platform's run.

--platform both

Runs the two scenario pipelines at the same time. The work that is not per-platform — the watchman reset, the library sync into the app, and the native builds — runs once and up front. Builds stay sequential on purpose: they share the app directory and its node_modules, and one machine gains little from compiling both at once. The pipelines are settled rather than raced, so one platform failing does not cut the other one short and both verdicts are reported.

Phase banners carry the platform they belong to, since the two pipelines write to one terminal. Output from the processes they spawn — gradle, xcodebuild, maestro — arrives as it is.

A CLI bug the concurrency exposed

create-history, update-history and release each wrote the release history to <cwd>/{binaryVersion}.json, handed the config that path, and removed the file afterwards. The name follows from the binary version alone, so two commands releasing the same version of one app for different platforms shared one path: whichever finished first took the file away while the other was still about to read it, and before that each had overwritten the other's contents. The observed failure was ENOENT on the second copy; the quieter one is a platform being handed the other platform's history, which update-history and release can produce because their contents differ per platform.

The history now goes to <cwd>/codepush-release-history/{platform}/. Only the directory changes. The file keeps its name, which is the part of the path a config can read; it is still removed once the history has been stored, and still left behind when storing it failed.

This is not E2E-only — it reproduces whenever two code-push invocations run in one project directory.

The RN CLI matrix

scripts/e2e/run-rn-cli-matrix.sh ran every app for Android and then every app for iOS. Without --only it now runs each app once with --platform both, halving the E2E invocations the matrix makes. --only android and --only ios keep a single platform, which is what to reach for when only one device is up.

scripts/e2e/warm-ios-pods.sh

Opt-in helper for an environment where reading a downloaded artifact back through a file:// URL is not permitted, which is how React Native's pod install unpacks its prebuilt tarballs: the step fails with curl: (37) Couldn't open file and no iOS build can start on a fresh checkout. The script copies an already-extracted Pods directory in from a checkout that has one. --from defaults to the repository's main working tree, so a linked worktree needs no argument beyond the app name. The copy is staged beside the target and swapped in once whole, so a copy that stops partway leaves the existing pods in place.

What existing CLI users see

Scope Change
create-history / update-history / release Options and the data handed to the storage callback are unchanged. The staged JSON path moves from <cwd>/<binaryVersion>.json to <cwd>/codepush-release-history/<platform>/<binaryVersion>.json. A config that reads the path it is given is unaffected; only one that rebuilds the path itself needs a change.
npm run e2e --platform both is new. ios and android behave as before.
scripts/e2e/run-rn-cli-matrix.sh Omitting --only now needs a simulator and an emulator or device up at the same time. A single-device setup uses --only ios or --only android.

Verification

Full suite for both platforms out of one checkout, on a booted iOS simulator and a connected Android device:

=== E2E tests passed: ios ===
=== E2E tests passed: android ===
EXIT=0
28m06s, 141 assertions, 0 retries

Nothing was skipped: the timing-sensitive scenarios (ON_NEXT_RESUME, ON_NEXT_SUSPEND, and the responsiveness check during a patch install) ran on both platforms and passed under the load of two simulators, two bundlers and two Maestro drivers sharing one machine. The Android pipeline's 30 Maestro invocations spanned 21m44s, with the iOS pipeline running inside that window.

Cleanup leaves the example app with App.tsx untouched and nothing else the run wrote.

npm run typecheck, npm run jest (154 tests) and npx eslint --quiet . pass. The one lint error, WAIT_MS in e2e/scripts/sleep.js, predates this branch — it is a Maestro-injected global the config does not know about.

The staging fix carries four tests covering what the collision did: that two platforms writing at the same time get files of their own, that each is handed the history it released, that the file is taken away once stored, and that it survives a failed store.

@floyd-soomgo floyd-soomgo self-assigned this Aug 26, 2026
A run could only cover one platform at a time, because everything it wrote was
shared: the example app's `App.tsx`, which is rewritten once per release; the
mock server's fixed port, served directory and request log; the record the local
CLI config appends its artifacts to; the CLI's output root under the app; and the
temp directory the CLI sweeps before it bundles. Two platforms at once would have
overwritten each other's release markers, thrown away each other's served data,
and deleted each other's bundle mid-flight.

Each of those now follows from the platform:

- The app entry is written as `App.ios.tsx` or `App.android.tsx`, which Metro
  resolves ahead of the plain name, leaving `App.tsx` as the source they are
  patched from. The host the entry is built with differs per platform anyway,
  which one shared file could never carry.
- Marker assets go to a directory of their own, because a release clears them by
  removing them and both platforms write the same file names. That directory's
  name avoids the `e2e-asset` prefix the files carry, since the asset assertions
  match archive entries on the characters both platforms keep.
- The mock server listens on 18081 for iOS and 18082 for Android, serves a data
  directory of its own, and keeps its own request log. The served paths inside a
  data directory are unchanged, so the storage layout the CLI is asserted against
  stays what it was.
- Bundling commands are passed `-o build/<platform>`, which stays under the
  `build` directory the example apps already ignore, and every CLI invocation
  runs with a TMPDIR of its own so the temp sweep and Metro's cache stay inside
  one platform's run.

`--platform both` then runs the two scenario pipelines at the same time. The work
that is not per-platform - the watchman reset, the library sync into the app, and
the native builds - runs once and up front. Builds stay sequential on purpose:
they share the app directory and its node_modules, and one machine gains little
from compiling both at once. The pipelines are settled rather than raced, so one
platform failing does not cut the other one short and both verdicts are reported.

Phase banners carry the platform they belong to, since the two pipelines write to
one terminal; output from the processes they spawn stays as it is. The E2E guide
records what the option needs, what a run writes per platform, what stays
sequential, and where a concurrent run's output interleaves.
create-history, update-history and release each wrote the history to
`<cwd>/{binaryVersion}.json` and handed the config that path. The name follows
from the binary version alone, so two commands releasing the same version of
one app for different platforms shared one path: whichever finished first took
the file away while the other was still about to read it, and before that each
had overwritten the other's contents. The failure was either ENOENT or a
platform being handed the other platform's history.

The history now goes to `<cwd>/codepush-release-history/{platform}/`. Only the
directory changes: the file keeps its name, which is the part of the path a
config can read, and it is still removed once the history has been stored and
still left behind when storing it failed.

Found by running the E2E suite for both platforms at once, where every release
targets binary version 1.0.0 out of one app directory.
…kout

Some environments do not permit reading a downloaded artifact back through a
`file://` URL, which is how React Native's pod install unpacks its prebuilt
tarballs: the step fails with `curl: (37) Couldn't open file` and no iOS build
can start on a fresh checkout.

warm-ios-pods.sh copies an already-extracted Pods directory in from a checkout
that has one, which is enough for pod install to skip the step. --from defaults
to the repository's main working tree, so a linked worktree needs no argument
beyond the app name. The copy clones on APFS, and the generated Pods project
holds no absolute paths, so what gets compiled is still this checkout.
The matrix ran every app for Android and then every app for iOS, two E2E
invocations per app, which is where its wall clock went now that one
invocation can cover both platforms.

Without --only it now runs each app once with --platform both, halving the
runs the matrix makes. --only android and --only ios keep a single platform,
which is what to reach for when only one device is up, and the two loops
collapse into one because the platform is the only thing that differed
between them.

The sibling matrix scripts are left as they are.
@floyd-soomgo
floyd-soomgo force-pushed the test/e2e-concurrent-platforms branch from 1919201 to 1060185 Compare August 26, 2026 05:30
@floyd-soomgo
floyd-soomgo merged commit 4c3bb74 into master Aug 26, 2026
1 check passed
@floyd-soomgo
floyd-soomgo deleted the test/e2e-concurrent-platforms branch August 26, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant