Skip to content

smoke CI job hard-fails on all fork PRs (forked PRs can't read OPENAI_API_KEY_TEST) #410

@SoundMindsAI

Description

@SoundMindsAI

Summary

The smoke (operator-path tutorial flow) job in .github/workflows/pr.yml hard-fails on every pull request opened from a fork — including all external-contributor PRs — for a reason unrelated to the contributor's code.

If you opened a PR from your fork and see a red ✗ on smoke while every other check is green, your change is almost certainly fine — this is a known CI limitation, not a defect in your PR.

Why it happens

The workflow triggers on pull_request: (not pull_request_target), and GitHub deliberately withholds repository secrets from fork-triggered runs (a security measure against secret exfiltration). So:

  1. OPENAI_API_KEY_TEST: ${{ secrets.OPENAI_API_KEY_TEST }} (pr.yml:536) resolves to an empty string on fork PRs.
  2. It's written to ./secrets/openai_key (pr.yml:553) as a zero-byte file.
  3. The "Sanity-check OPENAI_API_KEY_TEST is populated" step greps for non-whitespace and exit 1s:
    ##[error]OPENAI_API_KEY_TEST secret is empty — smoke gate requires it
    

This reproduces identically on every fork PR regardless of the diff. First observed on #387.

Proposed fix

Make the smoke-test job fork-aware — skip it gracefully with a notice on fork PRs instead of hard-failing, while keeping the upstream-repo path failing loudly on a genuinely empty secret. Candidate approaches (job-level if guard / in-step soft-skip / vetted pull_request_target) are written up in the planned-feature idea file:

docs/00_overview/planned_features/02_mvp2/infra_smoke_fork_pr_secret_skip/idea.md

Related

  • Sibling "smoke stays red" issue (independent failure mode — reseed wall-clock budget): infra_smoke_reseed_runtime_budget
  • Surfaced by use aclose for arq pool shutdown #387 (chore_arq_pool_aclose_deprecation)

Metadata

Metadata

Assignees

No one assigned

    Labels

    mvp2MVP2 backlog itemneeds-preflightIdea-stage; run /idea-preflight before /pipelinepriority/P2P2 — important to file, not blockingtype/infraInfra — tooling, CI, deploy

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions