Skip to content

test(e2e): make all contentious e2e resources configurable#9099

Open
mikehardy wants to merge 1 commit into
mainfrom
e2e-parallel
Open

test(e2e): make all contentious e2e resources configurable#9099
mikehardy wants to merge 1 commit into
mainfrom
e2e-parallel

Conversation

@mikehardy

Copy link
Copy Markdown
Collaborator

Summary

Make RNFB e2e ports, devices, and emulator suites configurable via environment variables so multiple platform runs can coexist on one host — as many iOS and Android :test-cover jobs as you provision slots for, plus one macOS, all at once. Default behaviour is unchanged: unset env vars preserve existing serial yarn tests:* commands, ports, and workflows exactly.

  • packages/app/e2e/helpers.js — shared Metro, Jet, and Firebase emulator host/port resolution with serial fallbacks.
  • tests/.detoxrc.js — multi-slot Android AVD / iOS simulator devices and env-driven reversePorts / native Metro build prefix.
  • tests/.jetrc.js — per-platform Jet and Metro ports from env for macOS (and host orchestration).
  • tests/e2e/firebase.test.js — Jet spawn/orchestration and macOS stale-app guard respect slotted ports from env.
  • tests/.babelrc — inline slotted port env vars into app bundles at build time.
  • Package e2e helpers (auth, database, firestore, functions, ai) — route emulator URLs through shared helpers.
  • AppDelegate.mm (iOS/macOS) — honour RCT_METRO_PORT from the environment.
  • package.jsonRNFB_DETOX_ANDROID_CONFIG / RNFB_DETOX_IOS_CONFIG on Detox build/test scripts; generic AVD/sim setup scripts.
  • firebase.emulator.template.json + start-emulator-slotted.sh — generate and start a per-platform Firebase emulator suite from RNFB_*_EMULATOR_* env.
  • create-android-avds.sh / create-ios-simulators.sh — provision cloned AVDs and named simulators for slots 1–4.
  • okf-bundle/testing/running-e2e.md — documents the configurable env var surface.

A future orchestration layer (outside this PR) will use these hooks to reserve slots and export env on demand — including parallel e2e on a single host — while RNFB itself stays coordinator-agnostic.

Test plan

  • Serial sanity: no env overrides → yarn tests:macos:test-cover (and one native platform) unchanged vs main
  • Slotted sanity: export RNFB_ANDROID_METRO_PORT / RNFB_ANDROID_JET_PORT / emulator ports for slot 1 → Android build + :test-cover reaches emulators on non-default ports
  • Docs: env var list in commit message matches running-e2e.md configurable section

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enables parallel E2E testing on a single host by making infrastructure resources—such as Metro bundler ports, Jet WebSocket ports, and Firebase emulator suites—configurable via environment variables. By allowing each test run to operate within a distinct 'slot' with its own ports and device instances, the changes eliminate resource contention while maintaining backward compatibility with existing serial test workflows.

Highlights

  • Configurable E2E Infrastructure: Introduced environment variable support for Metro, Jet, and Firebase emulator ports, allowing multiple E2E test runs to coexist on a single host.
  • Orchestration Scripts: Added helper scripts to provision cloned Android AVDs and named iOS simulators, facilitating parallel testing slots.
  • Dynamic Port Resolution: Updated application helpers and test configurations to dynamically resolve host and port settings based on the environment, replacing hardcoded defaults.
  • Native Build Integration: Updated iOS and macOS AppDelegate files to respect the RCT_METRO_PORT environment variable, ensuring native builds connect to the correct bundler instance.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/scripts/firebase.emulator.template.json
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a configurable e2e environment to support parallel test runs on a single host using slotted ports and dedicated device clones. It includes scripts for cloning simulators and emulators, and updates configurations, test suites, and native AppDelegates to resolve ports dynamically. The review feedback identifies a critical issue where a dynamic require of an e2e helper in the production AI package will cause consumer build failures. Other feedback addresses a version sorting bug in simulator creation, fragile sed usage, a port-reversing limitation in Detox configurations, and inconsistent Detox global references.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread packages/ai/lib/requests/request.ts Outdated
Comment thread tests/.detoxrc.js
Comment thread scripts/e2e/create-android-avds.sh Outdated
Comment thread scripts/e2e/create-ios-simulators.sh
Comment thread tests/e2e/firebase.test.js
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 63.63636% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.62%. Comparing base (3933b45) to head (8d47ad4).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9099      +/-   ##
============================================
+ Coverage     58.61%   58.62%   +0.01%     
- Complexity     1616     1617       +1     
============================================
  Files           503      503              
  Lines         39178    39199      +21     
  Branches       5807     5814       +7     
============================================
+ Hits          22962    22977      +15     
- Misses        14811    14822      +11     
+ Partials       1405     1400       -5     
Flag Coverage Δ
android-native 53.57% <ø> (+0.01%) ⬆️
e2e-ts-android 49.96% <36.37%> (-0.01%) ⬇️
e2e-ts-ios 53.89% <47.37%> (+0.01%) ⬆️
e2e-ts-macos 41.49% <36.37%> (+0.02%) ⬆️
ios-native 53.89% <47.37%> (+0.01%) ⬆️
jest 49.54% <0.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The following environment variables override serial e2e defaults when set
(before Metro build and :test-cover); unset vars keep legacy behaviour:

- RCT_METRO_PORT, RNFB_METRO_PORT, RNFB_{ANDROID,IOS,MACOS}_METRO_PORT
- JET_REMOTE_PORT, JET_METRO_PORT, RNFB_{ANDROID,IOS,MACOS}_JET_PORT, RNFB_JET_CONTROL_PORT
- RNFB_{ANDROID,IOS,MACOS}_EMULATOR_{FIRESTORE,AUTH,DATABASE,FUNCTIONS,STORAGE,HUB,LOGGING}_PORT
- RNFB_DETOX_ANDROID_CONFIG, RNFB_DETOX_IOS_CONFIG
- RNFB_E2E_PLATFORM, RNFB_E2E_SLOT, RNFB_E2E_DEBUG
- RNFB_ANDROID_AVD, RNFB_IOS_SIMULATOR, RNFB_ANDROID_EMULATOR_BOOT_ARGS
- ORG_GRADLE_PROJECT_reactNativeDevServerPort, SIMCTL_CHILD_RCT_METRO_PORT
@mikehardy mikehardy marked this pull request as ready for review July 11, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant