[E2E] Expose native preload state markers by kyle-schellen · Pull Request #626 · Shopify/checkout-kit

kyle-schellen · 2026-08-11T15:10:05Z

What changed

Observe PreloadState in the Swift and Android sample apps using the existing public preload callbacks.
Expose the readiness result through matching invisible test IDs on both platforms:
- preload-state-not-ready
- preload-state-ready
Add unit tests for the shared readiness-to-ID contract.

Why

Maestro cannot read an in-memory Swift/Kotlin preload state. These markers give native E2E tests a stable condition to wait for before presentation.

The identifier changes to preload-state-ready only when the SDK callback reports Ready, so the test does not infer preload completion from timing. The markers add no visible UI and no public SDK API.

Testing

dev swift test sample
dev android test demo
dev swift lint
dev android check detekt
Consumed by the preload E2E test in [E2E] Prepare native preload ready-flow validation #631, green on physical iOS and Android devices.

Stack

[E2E] Expose native preload state markers #626 — this PR: preload state markers in the samples
[E2E] Prove ready preload cache hits on both native samples #627 — ready cache-hit proof and sample markers
[E2E] Prepare native preload ready-flow validation #631 — the E2E test, matrix wiring, and BrowserStack validation

Before you merge

Important

I've added tests to support my implementation
I have read and agree with the Contribution Guidelines
I have read and agree with the Code of Conduct
I've updated the relevant platform README (platforms/swift/README.md and/or platforms/android/README.md)

Releasing a new Swift version?

I have bumped the version in ShopifyCheckoutKit.podspec
I have bumped the version in platforms/swift/Sources/ShopifyCheckoutKit/ShopifyCheckoutKit.swift
I have updated the SwiftPM/CocoaPods version snippets in platforms/swift/README.md (major version only)

Releasing a new Embedded Checkout Protocol version?

I have bumped embeddedCheckoutProtocolAndroid in platforms/android/gradle/libs.versions.toml
I have updated protocol/languages/kotlin/embedded-checkout-protocol/api/embedded-checkout-protocol.api if the public API changed

Releasing a new Android version?

I have bumped checkoutKitAndroid in platforms/android/gradle/libs.versions.toml
I have updated the Gradle/Maven version snippets in platforms/android/README.md

Tip

See the Contributing documentation for the full release process per platform.

kyle-schellen · 2026-08-11T15:10:21Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

github-actions · 2026-08-11T17:43:29Z

Package Size

Platform	Artifact	Base	Head	Delta
Android	release AAR	291.2 KiB	291.2 KiB	0 B

Android file breakdown

File	Base	Head
`classes.jar`	308.4 KiB	308.4 KiB
`res/layout/checkout_view_content.xml`	2.6 KiB	2.6 KiB
`res/layout/checkout_sheet_content.xml`	2.0 KiB	2.0 KiB
`res/values/values.xml`	1.3 KiB	1.3 KiB
`R.txt`	1.2 KiB	1.2 KiB
`AndroidManifest.xml`	922 B	922 B
`proguard.txt`	798 B	798 B
`res/drawable/close.xml`	431 B	431 B
`res/menu/checkout_menu.xml`	354 B	354 B
`META-INF/com/android/build/gradle/aar-metadata.properties`	157 B	157 B

Measured from the PR base SHA and PR head SHA. The file breakdown shows uncompressed sizes within each package artifact, so individual files do not sum to the compressed artifact total. This comment reports package artifact sizes only; it is not a final app binary-size report.

bitrise · 2026-08-11T17:45:10Z

Install this build

Open Tophat, select your target device, then click Install. Links open on the Mac running Tophat.

SDK	Install
Swift	Install with Tophat
Kotlin	Install with Tophat

Checkout Kit E2E results

Status	Suite	Target	Platform	OS version tag	Device
✅	`kotlin-android`	kotlin	android	latest	Google Pixel 11 Android 17.0
✅	`swift-ios`	swift	ios	latest	iPhone 15 iOS 27 Beta

kieran-osgood-shopify · 2026-08-12T12:29:27Z

Do we need a corresponding change to the react-native sample or is that already using this approach?

kyle-schellen · 2026-08-18T00:53:29Z

I might wait for RN because it can't use this approach yet. This PR maps the native PreloadState callbacks to identifiers, and until #654there was no equivalent on RN. #656 is already adding RN sample preload observability, so building on that makes sense imo.

markmur · 2026-08-20T16:03:55Z

+    private static func failedText(for reason: PreloadState.FailureReason) -> String {
+        switch reason {
+        case let .httpError(statusCode):
+            return "failed-http-\(statusCode)"
+        case .navigationFailed:
+            return "failed-navigation"
+        case .webContentUnavailable:
+            return "failed-web-content-unavailable"
+        case .protocolError:
+            return "failed-protocol"
+        }
+    }


This might be overkill. It might be hard to force these states

## What changed Proves that presentation actually reused the preloaded WebView, and exposes that proof to Maestro on both native samples. - **Swift SDK:** emit a debug-only diagnostic through the configured `Logger` when presentation selects a matching cached WebView that reached `.ready`. The existing OSLog message is preserved for dashboard compatibility. - **Swift sample:** observe that diagnostic as it is written and republish it as an invisible `preload-cache-hit-*` identifier. Normal launches hide the diagnostic marker from the accessibility tree; #631 enables preload observability only for diagnostic runs. - **Android SDK/sample:** keep the existing cache-hit log behavior, pin its exact internal diagnostic in an SDK test, and read the app's PID-scoped Logcat. Each watcher writes a unique UUID boundary after its reader starts, ignores every line until that boundary appears, and then republishes the cache-hit identifier only when the latest observed preload state is `Ready`. The debug-only watcher owns and closes its reader/process with the `CartViewModel`. - Tests cover ready hits, still-loading entries, non-debug logging, fresh presentation, URL mismatch, exact diagnostic strings, stale hits before the current UUID boundary, old UUID sessions, Android Ready gating, Logcat failures, and reader/process shutdown. ## Why `preload-state-ready` proves background preload completed. It does not prove presentation used that cached WebView — a fresh one looks identical on screen. This closes that gap. The `.ready` condition matters: a cached entry that matches but is still loading would otherwise count as a hit. Both samples end up exposing the **same identifier**, so the E2E flow has no platform branch. Getting there differs only because the SDKs differ: | | How the sample sees the cache hit | | --- | --- | | Swift | Installs a `Logger`, so it observes the message directly | | Kotlin | SDK log sink is `internal`, so it reads its own PID-scoped Logcat. A per-watcher UUID boundary excludes buffered lines from earlier app processes before the observer accepts a cache hit | Each sample matches the message text against its own copy because the SDK diagnostics are not public API. SDK emission tests and sample parser tests pin the exact literals, so accidental drift fails close to the change instead of surfacing as a 30-second E2E timeout. ## Testing - `dev swift test CheckoutWebViewTests` - `dev swift test sample` - `dev android test demo`, `dev android check detekt` - `PreloadCacheHitMarkerTest`: stale pre-boundary hits, old UUID sessions, current-boundary arming, Ready gating, Logcat failures, and reader shutdown - `dev swift lint` - `dev android e2e --tags preload` - `dev swift e2e --tags preload` - Asserted end to end by #631 on both platforms: preload ready → present → real checkout content → close → `preload-cache-hit-observed`, with `preload-cache-hit-none` asserted beforehand so a stuck marker cannot pass. ## Stack 1. #626 — preload state markers in the samples 2. **#627 — this PR:** proof that the ready cache entry was reused 3. #631 — the E2E test and matrix wiring --- ### Before you merge > [!IMPORTANT] > - [x] I've added tests to support my implementation > - [x] I have read and agree with the [Contribution Guidelines](./CONTRIBUTING.md) > - [ ] I have read and agree with the [Code of Conduct](./CODE_OF_CONDUCT.md) > - [ ] I've updated the relevant platform README (`platforms/swift/README.md` and/or `platforms/android/README.md`) --- <details> <summary>Releasing a new Swift version?</summary> - [ ] I have bumped the version in `ShopifyCheckoutKit.podspec` - [ ] I have bumped the version in `platforms/swift/Sources/ShopifyCheckoutKit/ShopifyCheckoutKit.swift` - [ ] I have updated the SwiftPM/CocoaPods version snippets in `platforms/swift/README.md` (major version only) </details> <details> <summary>Releasing a new Embedded Checkout Protocol version?</summary> - [ ] I have bumped `embeddedCheckoutProtocolAndroid` in `platforms/android/gradle/libs.versions.toml` - [ ] I have updated `protocol/languages/kotlin/embedded-checkout-protocol/api/embedded-checkout-protocol.api` if the public API changed </details> <details> <summary>Releasing a new Android version?</summary> - [ ] I have bumped `checkoutKitAndroid` in `platforms/android/gradle/libs.versions.toml` - [ ] I have updated the Gradle/Maven version snippets in `platforms/android/README.md` </details> > [!TIP] > See the [Contributing documentation](./CONTRIBUTING.md) for the full release process per platform.

## What changed - `e2e/tests/shared/native-preload-ready.yaml` — top-level test, tagged `preload`/`smoke`. - `e2e/flows/checkout/assert-native-preload-ready.yaml` — the reusable subflow, including the existing enabled-button and close-animation safeguards. - `e2e/flows/app/launch.yaml` — enables preload observability for diagnostic runs so the Swift marker stays out of the normal sample accessibility tree. - `e2e/config/matrix.yml` — Swift and Kotlin adopt the `preload` journey. React Native is unchanged, and matrix tests ensure native overrides retain every shared default tag. - `preload` added to the journey-tag taxonomy, with matrix and tag tests. ## Why Presenting checkout is not proof that preload was reused. This asserts the whole chain: ``` preload-state-ready → cached WebView selected → real checkout content ``` Both platforms run the **same flow with no platform branch**, because each sample republishes its SDK's cache-hit diagnostic as the same `preload-cache-hit-*` identifier (#627). The flow pins both ends — `preload-cache-hit-none` before presentation, `preload-cache-hit-observed` after — so a marker stuck on "observed" cannot pass. ## Testing - All `e2e/test/*_test.rb` — 261 runs, 710 assertions, 0 failures - Matrix expansion verified: `launch,checkout,preload` on the two native rows, React Native unchanged - `dev android e2e --tags preload` — passed on Android emulator - `dev swift e2e --tags preload` — passed on iOS simulator ## Stack 1. #626 — preload state markers in the samples 2. #627 — proof that the ready cache entry was reused 3. **#631 — this PR:** the E2E test and matrix wiring --- ### Before you merge > [!IMPORTANT] > - [ ] I've added tests to support my implementation > - [ ] I have read and agree with the [Contribution Guidelines](./CONTRIBUTING.md) > - [ ] I have read and agree with the [Code of Conduct](./CODE_OF_CONDUCT.md) > - [ ] I've updated the relevant platform README (`platforms/swift/README.md` and/or `platforms/android/README.md`) --- <details> <summary>Releasing a new Swift version?</summary> - [ ] I have bumped the version in `ShopifyCheckoutKit.podspec` - [ ] I have bumped the version in `platforms/swift/Sources/ShopifyCheckoutKit/ShopifyCheckoutKit.swift` - [ ] I have updated the SwiftPM/CocoaPods version snippets in `platforms/swift/README.md` (major version only) </details> <details> <summary>Releasing a new Embedded Checkout Protocol version?</summary> - [ ] I have bumped `embeddedCheckoutProtocolAndroid` in `platforms/android/gradle/libs.versions.toml` - [ ] I have updated `protocol/languages/kotlin/embedded-checkout-protocol/api/embedded-checkout-protocol.api` if the public API changed </details> <details> <summary>Releasing a new Android version?</summary> - [ ] I have bumped `checkoutKitAndroid` in `platforms/android/gradle/libs.versions.toml` - [ ] I have updated the Gradle/Maven version snippets in `platforms/android/README.md` </details> > [!TIP] > See the [Contributing documentation](./CONTRIBUTING.md) for the full release process per platform.

github-actions Bot added the #gsd:50662 Rebase Checkout Kit on UCP label Aug 11, 2026

kyle-schellen mentioned this pull request Aug 11, 2026

[E2E] Prove ready preload cache hits on both native samples #627

Merged

11 tasks

kyle-schellen changed the title ~~test(e2e): expose native preload state markers~~ [E2E]: expose native preload state markers Aug 11, 2026

kyle-schellen changed the title ~~[E2E]: expose native preload state markers~~ [E2E] Expose native preload state markers Aug 11, 2026

kyle-schellen marked this pull request as ready for review August 11, 2026 17:40

kyle-schellen requested a review from a team as a code owner August 11, 2026 17:40

kyle-schellen marked this pull request as draft August 11, 2026 17:42

kyle-schellen force-pushed the ks-e2e-preload-sample-markers branch from 558cb61 to 76447c5 Compare August 11, 2026 19:11

kyle-schellen self-assigned this Aug 11, 2026

kyle-schellen marked this pull request as ready for review August 11, 2026 22:39

kyle-schellen marked this pull request as draft August 12, 2026 12:31

kyle-schellen force-pushed the ks-e2e-preload-sample-markers branch from 76447c5 to 770e7de Compare August 12, 2026 17:58

kyle-schellen mentioned this pull request Aug 12, 2026

[E2E] Prepare native preload ready-flow validation #631

Merged

11 tasks

kyle-schellen force-pushed the ks-e2e-preload-sample-markers branch 2 times, most recently from 56cc7f5 to 49478c1 Compare August 17, 2026 23:30

kyle-schellen force-pushed the ks-e2e-preload-sample-markers branch from 49478c1 to 715c1f5 Compare August 18, 2026 13:30

kyle-schellen marked this pull request as ready for review August 19, 2026 19:17

markmur approved these changes Aug 20, 2026

View reviewed changes

markmur reviewed Aug 20, 2026

View reviewed changes

tiagocandido approved these changes Aug 21, 2026

View reviewed changes

kieran-osgood-shopify approved these changes Aug 21, 2026

View reviewed changes

test(e2e): expose native preload state markers

25f45e0

kyle-schellen force-pushed the ks-e2e-preload-sample-markers branch from 715c1f5 to 25f45e0 Compare August 21, 2026 16:22

kyle-schellen merged commit 0fdad6c into main Aug 21, 2026
35 checks passed

kyle-schellen deleted the ks-e2e-preload-sample-markers branch August 21, 2026 19:22

kyle-schellen mentioned this pull request Aug 25, 2026

[E2E] Prove disabled preload avoids cache reuse #694

Draft

Navigation Menu

Uh oh!

[E2E] Expose native preload state markers - #626

[E2E] Expose native preload state markers#626
kyle-schellen merged 1 commit into
mainfrom
ks-e2e-preload-sample-markers

kyle-schellen commented Aug 11, 2026 •

edited

Loading

Uh oh!

kyle-schellen commented Aug 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

bitrise Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

kieran-osgood-shopify commented Aug 12, 2026

Uh oh!

kyle-schellen commented Aug 18, 2026

Uh oh!

markmur Aug 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

kyle-schellen commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Why

Testing

Stack

Before you merge

Uh oh!

kyle-schellen commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Package Size

Uh oh!

bitrise Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Install this build

Checkout Kit E2E results

Uh oh!

kieran-osgood-shopify commented Aug 12, 2026

Uh oh!

kyle-schellen commented Aug 18, 2026

Uh oh!

markmur Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kyle-schellen commented Aug 11, 2026 •

edited

Loading

kyle-schellen commented Aug 11, 2026 •

edited

Loading

github-actions Bot commented Aug 11, 2026 •

edited

Loading

bitrise Bot commented Aug 11, 2026 •

edited

Loading