Skip to content

Shared GitHub Actions workflow for integration tests across Harper repos #25

Description

@Ethan-Arrowood

Idea

Most Harper repos that use the integration-testing framework have nearly identical integration-tests GitHub Actions workflows. Comparing five of them:

Repo Node default Build Fixtures Shards Bespoke extras
oauth 22, 24 none (pure simple tier)
generic plugin 20/22/24 optional none
nextjs 20, 22, 24 playwright install, harper_ref override, trace artifacts
harper-pro 22, 24, 26 4 submodules, re2 rebuild, legacy harperdb@4, concurrency cap
harper (core) 22, 24, 26 6 Windows, Bun, uWS, legacy install

The common skeleton — matrix generation → checkout → node setup → npm ci → build/fixtures → run test:integration → upload logs on failure — is identical across all of them. Everything that differs lives in the middle steps. That ~60% boilerplate is a good consolidation target; the bespoke middles are not.

Proposed approach: two shared artifacts, hosted here, versioned with the framework

The divergence lives mid-job, and a reusable workflow is all-or-nothing at the job level (you can't inject a step into its middle). So a single mechanism isn't enough — pair two:

  1. A reusable workflow (.github/workflows/integration-tests.yml, on: workflow_call) that owns the entire simple-tier pipeline (matrix gen, setup, run, logs-on-failure). Inputs: node-version / all-node-versions, build, install-fixtures, test-command, timeout-minutes. Callers shrink to ~8 lines. Covers oauth + any plain plugin.

  2. A composite action (.github/actions/setup/action.yml) that owns just the shared setup steps (node + npm ci + optional build/fixtures), dropped into a caller's own job. For repos that must interleave custom steps — nextjs (playwright, harper_ref), harper-pro/core (sharding, native rebuilds). The action deliberately does not checkout (harper-pro needs submodules: recursive). Caveat: a composite action can't wrap a failure-triggered log upload around the caller's test step, so heavy-tier callers repeat those few lines.

Example: oauth's entire workflow becomes

name: Integration Tests
on:
  push: { branches: [main] }
  pull_request:
  workflow_dispatch:
    inputs:
      node-version: { type: choice, default: all, options: [all, '22', '24'] }
jobs:
  test:
    uses: HarperFast/integration-testing/.github/workflows/integration-tests.yml@v1
    with:
      node-version:      ${{ inputs.node-version || 'all' }}
      all-node-versions: '[22, 24]'
      build:             npm run build
      install-fixtures:  npm run install:fixtures
      timeout-minutes:   5

Example: nextjs / harper-pro keep their own job, reuse only setup

steps:
  - uses: actions/checkout@v6.0.3
    # with: { submodules: recursive }   # harper-pro
  - uses: HarperFast/integration-testing/.github/actions/setup@v1
    with:
      node-version: ${{ matrix.node-version }}
      build:        npm run build
  # --- project-specific middle: playwright / harper_ref / re2 / shards ---
  - run: npx playwright install --with-deps chromium
  - run: npm run test:integration
    env: { HARPER_INTEGRATION_TEST_LOG_DIR: /tmp/harper-test-logs, FORCE_COLOR: '1' }

Where to draw the line

  • Use the reusable workflow if your CI is just matrix → build → run → logs (oauth, most future plugins).
  • Use only the composite action the moment you need a step in the middle (playwright, harper_ref, sharding, native rebuilds) — keep your job, dedupe the setup.
  • harper core stays bespoke — its matrix (Windows × Bun × uWS × 6 shards × legacy install) is divergent enough that routing it through the shared workflow would bury it in conditionals. It can still open each job with the shared setup action.

Open questions / trade-offs to discuss

  • Blast radius vs. drift. Consolidation trades N maintained files for one shared surface — a bad change can break every consumer's CI at once. Mitigate by pinning callers to a tag/SHA (never @main) and versioning the workflow/action alongside the framework release, so CI config and framework move in lockstep. Is that coupling worth it vs. a copy-paste scaffold template (no runtime coupling, but drifts)?
  • Home. This repo (co-located + versioned with the framework) vs. a shared HarperFast/.github repo.
  • Scope of inputs. How much variance (playwright toggle? shard count? legacy install?) belongs in the reusable workflow before it's better left to the composite action?
  • Secrets. Callers add secrets: inherit when tests need them — worth documenting as a convention.

If we pursue this, next step would be the two files + a short docs/ci.md adoption guide, then converting one simple consumer (oauth) as a proof of concept.

sent with Claude Opus 4.8

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions