Skip to content

chore: mirror repository to AWS CodeCommit #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jmgasper wants to merge 1 commit into from
Closed

chore: mirror repository to AWS CodeCommit #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 144 additions & 0 deletions .github/workflows/codecommit-mirror.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
name: Mirror GitHub to AWS CodeCommit
# Runbook: docs/github-codecommit-mirror.md

on:
push:
branches:
- develop
- master
schedule:
- cron: '17 */6 * * *'
workflow_dispatch:

permissions:
contents: read
id-token: write

concurrency:
group: codecommit-mirror-${{ github.repository }}
cancel-in-progress: false

jobs:
mirror:
name: Synchronize Git refs
if: >-
${{
vars.CODECOMMIT_MIRROR_ENABLED == 'true' &&
(github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/master')
}}
runs-on: ubuntu-24.04
timeout-minutes: 20
env:
MIRROR_AWS_REGION: ${{ vars.CODECOMMIT_MIRROR_AWS_REGION }}
MIRROR_REPOSITORY: ${{ vars.CODECOMMIT_MIRROR_REPOSITORY }}
MIRROR_ROLE_ARN: ${{ vars.CODECOMMIT_MIRROR_ROLE_ARN }}
SOURCE_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}

steps:
- name: Validate mirror configuration
shell: bash
run: |
set -euo pipefail

configuration_is_valid=true

if [[ -z "${MIRROR_AWS_REGION}" ]]; then
echo "::error::Repository variable CODECOMMIT_MIRROR_AWS_REGION is not configured."
configuration_is_valid=false
fi

if [[ -z "${MIRROR_REPOSITORY}" ]]; then
echo "::error::Repository variable CODECOMMIT_MIRROR_REPOSITORY is not configured."
configuration_is_valid=false
fi

if [[ -z "${MIRROR_ROLE_ARN}" ]]; then
echo "::error::Repository variable CODECOMMIT_MIRROR_ROLE_ARN is not configured."
configuration_is_valid=false
fi

if [[ -z "${SOURCE_DEFAULT_BRANCH}" ]]; then
echo "::error::The GitHub event did not identify the repository's default branch."
configuration_is_valid=false
fi

if [[ "${configuration_is_valid}" != "true" ]]; then
exit 1
fi

if [[ ! "${MIRROR_AWS_REGION}" =~ ^[a-z]{2}(-[a-z0-9]+)+-[0-9]+$ ]]; then
echo "::error::CODECOMMIT_MIRROR_AWS_REGION is not a valid AWS Region name."
exit 1
fi

if [[ ! "${MIRROR_REPOSITORY}" =~ ^[A-Za-z0-9._-]{1,100}$ ]]; then
echo "::error::CODECOMMIT_MIRROR_REPOSITORY is not a valid CodeCommit repository name."
exit 1
fi

if [[ ! "${MIRROR_ROLE_ARN}" =~ ^arn:(aws|aws-us-gov|aws-cn):iam::[0-9]{12}:role/.+$ ]]; then
echo "::error::CODECOMMIT_MIRROR_ROLE_ARN is not a valid IAM role ARN."
exit 1
fi

- name: Check out every branch and tag
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ env.SOURCE_DEFAULT_BRANCH }}
fetch-depth: 0
persist-credentials: false

- name: Configure short-lived AWS credentials
uses: aws-actions/configure-aws-credentials@e6de054238d6b7531b4efff3b6587d9aade6a06c # v6.2.3
with:
role-to-assume: ${{ env.MIRROR_ROLE_ARN }}
aws-region: ${{ env.MIRROR_AWS_REGION }}
role-session-name: GitHubActions-${{ github.event.repository.name }}-mirror

- name: Synchronize branches, tags, and the default branch
shell: bash
run: |
set -euo pipefail

codecommit_url="https://git-codecommit.${MIRROR_AWS_REGION}.amazonaws.com/v1/repos/${MIRROR_REPOSITORY}"
git remote add codecommit "${codecommit_url}"
git config --local credential.helper '!aws codecommit credential-helper $@'
git config --local credential.UseHttpPath true

while read -r object_name remote_ref; do
if [[ "${remote_ref}" == "refs/remotes/origin/HEAD" ]]; then
continue
fi

branch_name="${remote_ref#refs/remotes/origin/}"
git update-ref "refs/heads/${branch_name}" "${object_name}"
done < <(git for-each-ref --format='%(objectname) %(refname)' refs/remotes/origin)

git push --force codecommit \
'refs/heads/*:refs/heads/*' \
'refs/tags/*:refs/tags/*'

aws codecommit update-default-branch \
--region "${MIRROR_AWS_REGION}" \
--repository-name "${MIRROR_REPOSITORY}" \
--default-branch-name "${SOURCE_DEFAULT_BRANCH}"

git push --force --prune codecommit \
'refs/heads/*:refs/heads/*' \
'refs/tags/*:refs/tags/*'

- name: Verify mirrored ref object IDs
shell: bash
run: |
set -euo pipefail

source_refs="${RUNNER_TEMP}/source-refs"
destination_refs="${RUNNER_TEMP}/codecommit-refs"

git for-each-ref --format='%(objectname) %(refname)' refs/heads refs/tags \
| sort > "${source_refs}"
git ls-remote --refs codecommit 'refs/heads/*' 'refs/tags/*' \
| awk '{ print $1, $2 }' \
| sort > "${destination_refs}"

diff -u "${source_refs}" "${destination_refs}"
80 changes: 80 additions & 0 deletions docs/github-codecommit-mirror.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# GitHub backup mirror to AWS CodeCommit

The [mirror workflow](../.github/workflows/codecommit-mirror.yml) maintains an exact secondary Git copy of `topcoder-platform/groups-api-v6` in the dedicated `groups-api-v6` AWS CodeCommit repository.

## Provisioned configuration

Provisioning and the initial verified seed were completed on 24 August 2026:

- AWS account: `811668436784`
- AWS Region: `us-east-1`
- CodeCommit repository: `groups-api-v6`
- CodeCommit default branch: `develop`
- IAM role: `GitHubActions-groups-api-v6-CodeCommitMirror`
- Trusted GitHub branches: `develop` and `master`
- Authentication: GitHub OpenID Connect (OIDC), with no long-lived AWS access key
- GitHub repository variables: configured and enabled

## Synchronization behavior

A push to `develop` or `master` starts a full reconciliation. A reconciliation also runs every six hours and can be started manually from either trusted branch.

Every run:

1. Fetches all GitHub branches and tags with their reachable Git history.
2. Obtains short-lived AWS credentials by presenting a GitHub-signed OIDC token.
3. Force-updates CodeCommit branches and tags to the same Git object IDs.
4. Deletes CodeCommit branches and tags that no longer exist in GitHub.
5. Sets CodeCommit's default branch to GitHub's default branch.
6. Fails if any final branch or tag object ID differs.

Each run mirrors all current refs. A push to another branch or tag therefore appears in CodeCommit at the next `develop`/`master` push or scheduled reconciliation; workflows on those other refs cannot obtain the AWS role.

Runs are serialized so an older run cannot finish after a newer one and move the mirror backwards.

## Trust and permissions

The account-wide IAM OIDC provider is:

`arn:aws:iam::811668436784:oidc-provider/token.actions.githubusercontent.com`

The `GitHubActions-groups-api-v6-CodeCommitMirror` trust policy accepts only OIDC subjects for this repository's `develop` and `master` branches. It accepts both GitHub's legacy name-based subject and the repository's immutable owner/repository-ID subject. Public forks and workflows on any other upstream ref have different subjects and are denied.

The role's inline `CodeCommitMirrorAccess` policy permits only:

- `codecommit:GitPull`
- `codecommit:GitPush`
- `codecommit:UpdateDefaultBranch`

Those permissions apply only to `arn:aws:codecommit:us-east-1:811668436784:groups-api-v6`. The role cannot access other CodeCommit repositories, delete repositories, manage IAM, or use other AWS services.

GitHub stores only resource identifiers in these repository variables:

- `CODECOMMIT_MIRROR_AWS_REGION`
- `CODECOMMIT_MIRROR_REPOSITORY`
- `CODECOMMIT_MIRROR_ROLE_ARN`
- `CODECOMMIT_MIRROR_ENABLED`

The role ARN and repository variables are not credentials. AWS validates GitHub's signed token and issues credentials for at most one hour. GitHub Actions logs and AWS CloudTrail provide an audit trail.

## Operations

- Treat a failed mirror run as a backup warning; it does not block or roll back the GitHub push.
- Use **Actions → Mirror GitHub to AWS CodeCommit → Run workflow** from `develop` or `master` for an immediate repair.
- The next trusted-branch push or six-hour run performs a complete repair, so retries are safe.
- Keep both referenced third-party Actions pinned to reviewed full commit SHAs.
- Protect `develop` and `master` and require review for changes under `.github/workflows/`.
- If a trusted branch or repository identity changes, update both the workflow filters and IAM `sub` conditions before the next run.
- Do not develop in the CodeCommit repository; GitHub is authoritative and synchronization overwrites destination-only refs.

## Recovery limitations

This is an exact warm Git replica, not immutable point-in-time retention:

- Force-pushes, moved tags, and deleted branches or tags are reproduced.
- A commit on another ref that becomes unreachable before reconciliation can be missed.
- Pull requests, issues, releases, Actions artifacts, repository settings, secrets, and other GitHub metadata are not copied.
- Git LFS pointer files are copied, but external LFS objects are not.
- Provider-specific refs such as GitHub pull-request refs are not copied.

For immutable retention, add a separate scheduled `git bundle` archive in versioned, object-locked storage. For disaster recovery, an AWS administrator with CodeCommit read access can clone the mirror and push its branches and tags into a replacement GitHub repository.
Loading