Skip to content

Drop automatic adb input swipe fallback from Android gesture swipes #690

@thymikee

Description

@thymikee

Problem

PR #681 routes Android gesture swipe through the same provider-native touch injection / bundled touch-helper path as pinch, rotate, and transform, but still keeps an automatic fallback to adb shell input swipe when the helper path fails.

That fallback is useful only as a compatibility escape hatch. Long term, it can hide real regressions in the Android touch-helper path: if the helper artifact is missing, install fails, instrumentation fails, or UiAutomation.injectInputEvent returns false, agent-device should normally surface that as an actionable helper/runtime problem instead of silently downgrading to shell swipe.

Current state

  • src/core/interactors/android.ts now sends swipe, pan, and fling through swipeGestureAndroid.
  • src/platforms/android/multitouch-helper.ts prefers provider-native touch injection, then the bundled Android touch helper, then falls back to swipeAndroid / adb shell input swipe for swipe only.
  • android-multitouch-helper injects swipe via Instrumentation.getUiAutomation().injectInputEvent(...), the same Android API family used for the richer touch gestures.
  • The helper manifest declares minSdkVersion=23 and targetSdkVersion=36.
  • package.json includes android-multitouch-helper/dist in npm package files, and prepack builds/packages the helper. Missing local dist artifacts should be treated as a build/packaging bug, not a supported runtime condition.
  • Android CI packages the multitouch helper before smoke tests.

Research notes

  • Android UiAutomation.injectInputEvent(InputEvent, boolean) is available since API level 18, per Android docs: https://developer.android.com/reference/android/app/UiAutomation#injectInputEvent(android.view.InputEvent,%20boolean)
  • The helper currently supports API 23+ because both the manifest and build script set minSdk 23.
  • No Android API-level blocker is apparent for using the helper on supported agent-device Android environments. Dropping fallback would only exclude Android < 23 unless we intentionally lower helper minSdk.
  • Remote Android providers are not necessarily local adb -s processes. The public provider docs describe ADB-shaped executors routed through ADB tunnels, websocket APIs, or other transports. A remote environment can only use the helper path if it supports the operations needed by the helper lifecycle: install test APKs (adb install -t) and run instrumentation (adb shell am instrument ...).
  • If a remote provider exposes provider-native touch injection, it already bypasses the helper and shell fallback.

What to verify before removal

  1. Packaging guarantee

    • Add/keep an installed-package smoke test that verifies the npm package contains android-multitouch-helper/dist/agent-device-android-multitouch-helper-<version>.manifest.json and APK.
    • Ensure local release/package flows always run package:android-multitouch-helper:npm before publish.
  2. Helper install/instrumentation support

    • Run a real-device/emulator smoke test for gesture swipe left|right that asserts backend metadata is android-multitouch-helper or provider-native-touch, not adb-input-swipe-fallback.
    • Include local emulator and any supported remote Android provider/lease backend in the matrix.
  3. Remote provider capability check

    • Confirm remote Android bridge/lease environments can either:
      • provide AndroidAdbProvider.touch, or
      • support exec/install enough for adb install -t plus adb shell am instrument.
    • If a provider only supports shell commands but not APK install or instrumentation, document that gesture swipe requires provider-native touch injection and should fail clearly.
  4. API-level support decision

    • Decide whether Android < 23 is explicitly unsupported for touch-helper gestures. Current helper minSdk is 23, while UiAutomation.injectInputEvent itself exists since API 18.
    • If API 21/22 support matters, prototype lowering helper minSdk before removing fallback. If not, document API 23+ for helper-backed Android gestures.
  5. Failure semantics

    • Remove the broad catch in swipeGestureAndroid that turns helper failures into adb input swipe.
    • Make helper failures loud and diagnostic-rich: artifact missing, checksum mismatch, install failure, instrumentation timeout/failure, and injectInputEvent returned false should preserve their original failure reason.
    • Keep adb shell input swipe only where the command explicitly means legacy/basic shell input, if that surface is still needed at all.

Proposed end state

Android gesture execution should be:

  1. provider-native touch injection when supplied by the Android provider;
  2. bundled Android touch helper for local ADB and capable remote ADB/provider transports;
  3. no automatic shell-swipe fallback for gesture swipe.

adb shell input swipe can remain as an explicit low-level/basic backend only if agent-device still exposes a command whose contract is intentionally shell-backed. It should not be the hidden recovery path for helper-backed gestures.

Why this matters

The helper path is more consistent with the rest of gesture semantics, can produce lifelike frame-based movement, and lets failures reveal platform/provider gaps. A silent adb input swipe fallback can make React Navigation or Maestro-compat runs appear to pass while masking helper packaging, install, instrumentation, or remote-provider regressions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions