Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed images/docs/images/cicd/github_setup_1.png
Binary file not shown.
Binary file removed images/docs/images/cicd/github_setup_2.png
Binary file not shown.
Binary file removed images/docs/images/cicd/github_setup_3.png
Binary file not shown.
Binary file removed images/docs/images/cicd/github_setup_4.png
Binary file not shown.
Binary file removed images/docs/images/cicd/github_setup_5.png
Binary file not shown.
Binary file removed images/docs/images/cicd/github_setup_6.png
Binary file not shown.
296 changes: 187 additions & 109 deletions integrations/ci-cd/github/actions.mdx
Original file line number Diff line number Diff line change
@@ -1,157 +1,235 @@
---
title: Integrating Checkly in GitHub Actions
description: "Run Checkly checks from GitHub Actions, triggering on deployment_status or push events to validate deployments before promoting your checks to monitors."
description: "Run Checkly checks from GitHub Actions and report test session results in pull requests."
sidebarTitle: GitHub Actions
canonical: 'https://www.checklyhq.com/docs/integrations/ci-cd/github/actions/'
---

## Workflow scenarios
Use the Checkly GitHub Action to run `checkly test` or `checkly trigger` from GitHub Actions. This lets you validate Monitoring as Code changes in pull requests, run checks against preview deployments, and report the test session result back to GitHub.

GitHub Actions workflows can run on different events. In most cases, you will want to trigger the Checkly CLI after a
**deployment** is done based on a recent `git push` to a branch, when a branch is merged or when a pull request is created.
<Accordion title="Prerequisites">
- A Checkly project with checks managed in code or deployed checks you want to trigger.
- A GitHub repository with GitHub Actions enabled.
- `CHECKLY_API_KEY` stored as a GitHub Actions secret.
- `CHECKLY_ACCOUNT_ID` stored as a GitHub Actions variable.
- Checkly CLI `8.15.0` or newer. The action uses the latest compatible CLI version by default.
- Optional: install the Checkly GitHub integration from [Account settings > Integrations](https://app.checklyhq.com/settings/account/integrations) to receive a separate GitHub Check for detached Checkly runs.
</Accordion>

GitHub created the [deployments API](https://docs.github.com/en/rest/deployments/deployments) for this and many 3rd party
integrators like Vercel and Heroku use this to relay deployment status to GitHub. However, you can also call this API
yourself and trigger a `deployment_status` event.
<Warning>
Install the Checkly GitHub App from inside the Checkly web app: go to [Account settings > Integrations](https://app.checklyhq.com/settings/account/integrations) and click **Integrate with GitHub**. Installing the app directly from GitHub does not create the matching Checkly integration, so GitHub Check reporting will not work correctly.
</Warning>

> Make sure to set your `CHECKLY_API_KEY` as a [secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets) and `CHECKLY_ACCOUNT_ID` as a [variable](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables) in your GitHub Actions settings before you get started.
## Choose a GitHub workflow

## Running on `deployment_status` events
There are two GitHub CI/CD paths in Checkly:

The `deployment_status` event is the preferred event to trigger a GH Actions workflow that executes your checks. However,
this comes with some peculiarities that are native to GH Actions and a bit different from using the `push` event.
| Use case | Guide |
| --- | --- |
| Run a Checkly test session from a GitHub Actions workflow and report the result on a pull request. | Continue with this guide. |
| Listen for GitHub deployment events and trigger checks linked to a repository, using the deployment environment URL as the test target. | Use [GitHub deployment hooks](/integrations/ci-cd/github/deployments). |

- The deployment event holds core information about your deployment, i.e. the environment name and an optional `ENVIRONMENT_URL`.
- The full git repo with full history is not available. We have to jump through some hoops to properly set the branch name
for instance.
- We have no access to the original pull request that triggered the deployment event.
## Run checks on pull requests

Use `command: test` when your checks are defined in the repository. The action checks out the pull request code, installs dependencies, and runs `checkly test` on Checkly's infrastructure.

```yaml .github/workflows/checkly.yml
name: 'checkly'
on: [deployment_status]

# Set the necessary credentials and export variables we can use to instrument our test run. Use the ENVIRONMENT_URL
# to run your checks against staging, preview or production.
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
ENVIRONMENT_URL: ${{ github.event.deployment_status.environment_url }}
CHECKLY_TEST_ENVIRONMENT: ${{ github.event.deployment_status.environment }}
name: Checkly

on:
pull_request:

jobs:
test-e2e:
if: github.event.deployment_status.state == 'success' # Only run when the deployment was successful.
name: Test E2E on Checkly
checkly:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v3
with:
ref: "${{ github.event.deployment_status.deployment.ref }}"
fetch-depth: 0
- name: Set branch name # workaround to detect branch name in "deployment_status" actions
run: echo "CHECKLY_TEST_REPO_BRANCH=$(git show -s --pretty=%D HEAD | tr -s ',' '\n' | sed 's/^ //' | grep -e 'origin/' | head -1 | sed 's/\origin\///g')" >> $GITHUB_ENV
- uses: actions/setup-node@v3
- uses: actions/checkout@v4

- uses: checkly/checkly-action@v1
with:
node-version-file: '.nvmrc'
- name: Restore or cache node_modules
id: cache-node-modules
uses: actions/cache@v3
command: test
cli-version: latest
install-command: npm ci
reporting: auto
github-check-name: Checkly PR checks
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
```

With `reporting: auto`, the action uses GitHub Check reporting when the Checkly GitHub App can report on the repository. If GitHub Check reporting is unavailable, the action waits for the test session and reports through the GitHub Actions job summary instead.

## Run selected checks

Filter the checks that run in a pull request with `tags`, `grep`, or file patterns.

```yaml .github/workflows/checkly-filtered.yml
name: Checkly

on:
pull_request:

jobs:
checkly:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: checkly/checkly-action@v1
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}
- name: Install dependencies
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: npm ci
- name: Run checks # run the checks passing in the ENVIRONMENT_URL and recording a test session.
id: run-checks
run: npx checkly test -e ENVIRONMENT_URL=${{ env.ENVIRONMENT_URL }} --reporter=github --record
- name: Create summary # export the markdown report to the job summary.
id: create-summary
run: cat checkly-github-report.md > $GITHUB_STEP_SUMMARY
- name: Deploy checks # if the test run was successful and we are on Production, deploy the checks
id: deploy-checks
if: steps.run-checks.outcome == 'success' && github.event.deployment_status.environment == 'Production'
run: npx checkly deploy --force
command: test
cli-version: latest
install-command: npm ci
reporting: auto
tags: |
production,webapp
production,api
grep: checkout
files: |
__checks__/**/*.check.ts
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
```

## Usage with mono repos and Vercel deployments
Multiple `tags` lines are treated as OR filters. Comma-separated tags on one line are treated as AND filters.

If you are running a mono repo, you might bump into issues that multiple `deployment_status` events will randomly trigger
your workflow. GitHub Actions does not have a great way to filter for this, but there are two strategies you can follow.
## Report results

### Filtering with `if` statements.
The `reporting` input controls where the Checkly result is reported.

If part of the URL for your deployments deterministically maps to one of the apps in your repo, you can just extend
the `if` statement in your workflow, as shown below.
| Value | Behavior |
| --- | --- |
| `auto` | Use a detached Checkly run with GitHub Check reporting when available. Otherwise, wait in the GitHub Actions job and report there. |
| `github-check` | Require detached GitHub Check reporting. The action fails before running checks if the Checkly GitHub App cannot report on the repository. |
| `github-actions` | Always wait in the GitHub Actions job and write the Checkly summary to the GitHub Actions job summary. |

Use `github-check` when the Checkly GitHub Check should be a required status check on pull requests.

```yaml .github/workflows/checkly-required.yml
name: Checkly required

on:
pull_request:

```yaml .github/workflows/checkly-filtered.yml
jobs:
test-e2e:
if: github.event.deployment_status.state == 'success' && contains(github.event.deployment_status.environment_url, 'webapp')
name: Test E2E on Checkly
checkly:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4

- uses: checkly/checkly-action@v1
with:
command: test
cli-version: latest
reporting: github-check
github-check-name: Checkly required checks
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
```

### Using branch protection rules
Use `github-check-name` to set the GitHub Check name that appears on pull requests. It defaults to `Checkly`.

<Note>
GitHub Check reporting requires the Checkly GitHub App to be installed through [Account settings > Integrations](https://app.checklyhq.com/settings/account/integrations) in Checkly. Without the app, use `reporting: auto` or `reporting: github-actions`.
</Note>

## Test preview deployments

For pull request preview deployments, pass the preview URL to your checks as `ENVIRONMENT_URL`. In Browser checks and Playwright Check Suites, read `process.env.ENVIRONMENT_URL` in your test code.

In many cases, the above solution with `if` statements is not enough. When using Vercel and mono repos for instance, GitHub
Actions will recognize any "skipped" statuses as "passed". This can cause havoc. A way to sidestep this issue is by setting
a branch protection rule on your repo that is generated by your GH Action workflow.
```yaml .github/workflows/checkly-preview.yml
name: Checkly preview

You do this by adding the [LouisBrunner/checks-action@v1.6.0](https://github.com/marketplace/actions/github-checks) action
to your workflow and assigning some name. This name can be fixed or dynamic. See the example below.
on:
pull_request:

```yaml .github/workflows/checkly-branch-protection.yml
name: 'Run Checkly MaC Checks'
on: [deployment_status]
env:
ENVIRONMENT_URL: ${{ github.event.deployment_status.environment_url }}
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
run-name: Run MaC checks against ${{ github.event.deployment_status.environment_url }}
jobs:
test-e2e:
if: github.event.deployment_status.state == 'success' && contains(github.event.deployment_status.environment_url, 'webapp')
name: Test E2E on Checkly
checkly:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
# All steps as normal...
- uses: LouisBrunner/checks-action@v1.6.0
if: always()
- uses: actions/checkout@v4

- id: preview
run: echo "url=https://preview.example.com" >> "$GITHUB_OUTPUT"

- uses: checkly/checkly-action@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
name: Test E2E Passed Notification
conclusion: ${{ job.status }}
command: test
cli-version: latest
reporting: auto
env: |
ENVIRONMENT_URL=${{ steps.preview.outputs.url }}
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
CHECKLY_TEST_ENVIRONMENT: preview
ENVIRONMENT_URL: ${{ steps.preview.outputs.url }}
```

The will generate a custom status for your called **Test E2E Passed Notification**.
If your deployment provider already emits GitHub deployment events with an environment URL, you can also use [GitHub deployment hooks](/integrations/ci-cd/github/deployments) instead of a GitHub Actions workflow.

Then, add a [branch protection rule](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/managing-a-branch-protection-rule)
for your repo.
## Trigger deployed checks

![GitHub Branch Protection Ruke](/images/docs/images/cicd/gh_branch_protection_rule.png)
Use `command: trigger` when you want to run checks that are already deployed in your Checkly account. This is useful for smoke testing existing production or staging checks from a release workflow.

Together with any optional `if` statements, your GH Actions will now run only against the apps you want in your mono repo.
```yaml .github/workflows/checkly-trigger.yml
name: Checkly trigger

## Using the GitHub Markdown Reporter
on:
workflow_dispatch:

You can print a user friendly summary report to your GitHub Actions Job Summary by adding the `--reporter=github` flag
to the `test` command and export the resulting `checkly-github-report.md` file to the `$GITHUB_STEP_SUMMARY` variable.
jobs:
checkly:
runs-on: ubuntu-latest
steps:
- uses: checkly/checkly-action@v1
with:
command: trigger
cli-version: latest
reporting: auto
tags: production
check-id: abc123,def456
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
```

## Use with monorepos

In monorepos, combine GitHub Actions path filters with the action's `working-directory`, `tags`, `grep`, or `files` inputs.

```yaml .github/workflows/checkly-monorepo.yml
name: Checkly webapp

In the example, you can see this in the following step:
on:
pull_request:
paths:
- apps/webapp/**
- packages/checks/**

```yaml .github/workflows/checkly-reporter.yml
- name: Run checks
id: run-checks
run: npx checkly test
-e ENVIRONMENT_URL=${{ env.ENVIRONMENT_URL }}
--reporter=github
--record
- name: Create summary
id: create-summary
run: cat checkly-github-report.md > $GITHUB_STEP_SUMMARY
jobs:
checkly:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: checkly/checkly-action@v1
with:
command: test
cli-version: latest
working-directory: apps/webapp
install-command: npm ci
reporting: auto
tags: webapp
env:
CHECKLY_API_KEY: ${{ secrets.CHECKLY_API_KEY }}
CHECKLY_ACCOUNT_ID: ${{ vars.CHECKLY_ACCOUNT_ID }}
```

## Next steps

- Learn more about [`checkly test`](/cli/checkly-test).
- Learn more about [`checkly trigger`](/cli/checkly-trigger).
- Use [GitHub deployment hooks](/integrations/ci-cd/github/deployments) when you want Checkly to react to GitHub deployment webhook events without a GitHub Actions workflow.
Loading
Loading