Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 48 additions & 8 deletions .github/workflows/self-host-nightly.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
# Manual maintenance hook for an externally reachable self-host stack. This is deliberately not scheduled while
# Manual maintenance hook for externally reachable self-host stacks. This is deliberately not scheduled while
# the review stack is running without colocated GitHub Actions runners.
#
# #4899: generalized from a single hardcoded instance to a fleet matrix. Each self-host instance is modeled
# as its own GitHub Environment (Settings -> Environments), carrying that instance's own `SELF_HOST_URL`
# environment variable and `INTERNAL_JOB_TOKEN` environment secret -- this is the supported way to give a
# matrix job a different secret VALUE per entry while every job still reads the same secret NAME
# (`secrets.INTERNAL_JOB_TOKEN`), since workflow expressions cannot index `secrets.*` by a computed key.
#
# Setup for N instances:
# 1. Create one GitHub Environment per instance (any name, e.g. "primary", "edge-nl-01", "customer-acme").
# 2. On each environment, set the `SELF_HOST_URL` variable and the `INTERNAL_JOB_TOKEN` secret for that
# instance.
# 3. Set the repo-level `SELF_HOST_ENVIRONMENTS` variable to a comma-separated list of those environment
# names, e.g. `primary,edge-nl-01`.
# A single-instance setup works exactly the same way -- one environment, one name in the list.
name: self-host maintenance

on:
Expand All @@ -13,30 +27,56 @@ concurrency:
cancel-in-progress: false

jobs:
discover:
if: ${{ vars.SELF_HOST_ENVIRONMENTS != '' }}
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
environments: ${{ steps.list.outputs.environments }}
steps:
- name: Parse SELF_HOST_ENVIRONMENTS into a JSON matrix list
id: list
env:
SELF_HOST_ENVIRONMENTS: ${{ vars.SELF_HOST_ENVIRONMENTS }}
run: |
json=$(printf '%s' "$SELF_HOST_ENVIRONMENTS" | tr ',' '\n' | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' -e '/^$/d' | jq -R . | jq -sc .)
echo "environments=$json" >> "$GITHUB_OUTPUT"
echo "Discovered instances: $json"

maintenance:
if: ${{ vars.SELF_HOST_URL != '' }}
needs: discover
if: ${{ needs.discover.outputs.environments != '' && needs.discover.outputs.environments != '[]' }}
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
environment: ${{ fromJson(needs.discover.outputs.environments) }}
environment: ${{ matrix.environment }}
env:
SELF_HOST_URL: ${{ vars.SELF_HOST_URL }}
INTERNAL_JOB_TOKEN: ${{ secrets.INTERNAL_JOB_TOKEN }}
steps:
- name: Self-host health
- name: Self-host health (${{ matrix.environment }})
run: |
if [ -z "$SELF_HOST_URL" ]; then
echo "✗ SELF_HOST_URL not set on the '${{ matrix.environment }}' environment — skipping."
exit 1
fi
if curl -fsS "$SELF_HOST_URL/ready" >/dev/null; then
echo "✓ self-host ready"
echo "✓ self-host ready (${{ matrix.environment }})"
else
echo "✗ self-host not ready"
echo "✗ self-host not ready (${{ matrix.environment }})"
exit 1
fi

- name: Refresh the RAG index (fan out over configured repos)
- name: Refresh the RAG index (fan out over configured repos) (${{ matrix.environment }})
run: |
if [ -z "$INTERNAL_JOB_TOKEN" ]; then
echo "INTERNAL_JOB_TOKEN not set — skipping RAG refresh (set it as a repo secret to enable)."
echo "INTERNAL_JOB_TOKEN not set on the '${{ matrix.environment }}' environment — skipping RAG refresh."
exit 0
fi
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$SELF_HOST_URL/v1/internal/jobs/rag-index" \
-H "authorization: Bearer $INTERNAL_JOB_TOKEN" -H "content-type: application/json" -d '{}')
echo "RAG re-index request → HTTP $code"
echo "RAG re-index request (${{ matrix.environment }}) → HTTP $code"
[ "$code" = "202" ] || [ "$code" = "200" ]
31 changes: 31 additions & 0 deletions apps/loopover-ui/content/docs/self-hosting-operations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -845,6 +845,37 @@ avoids piling duplicate fan-outs). In metrics, break down deferrals by
`job_type=agent-regate-pr` or `agent-regate-sweep` when GitHub
rate-limit pressure spikes.

## Self-host maintenance workflow (GitHub Actions)

[`self-host-nightly.yml`](https://github.com/JSONbored/loopover/blob/main/.github/workflows/self-host-nightly.yml)
is a manual (`workflow_dispatch`) GitHub Actions workflow that checks
`/ready` against your externally reachable self-host instance and,
if `INTERNAL_JOB_TOKEN` is configured, triggers a RAG re-index fan-out. It is
deliberately not scheduled while the review stack runs without colocated
Actions runners — trigger it by hand from the Actions tab, or wire it into
your own external scheduler if you want it automated.

It validates a **fleet** of instances, not just one. Each instance is
modeled as its own [GitHub
Environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment)
(repo Settings → Environments), carrying that instance's own
`SELF_HOST_URL` environment variable and `INTERNAL_JOB_TOKEN` environment
secret — this lets every matrix job read the same secret name while each
instance supplies its own value, since workflow expressions cannot index
`secrets.*` by a computed key.

<CodeBlock
lang="text"
code={`1. Create one GitHub Environment per instance (any name — e.g. "primary", "edge-nl-01").
2. On each environment, set the SELF_HOST_URL variable and the INTERNAL_JOB_TOKEN secret for that instance.
3. Set the repo-level SELF_HOST_ENVIRONMENTS variable to a comma-separated list of those environment names,
e.g. \`primary,edge-nl-01\`.`}
/>

A single-instance setup works exactly the same way — one environment, one
name in the list. The workflow no-ops (skips entirely) when
`SELF_HOST_ENVIRONMENTS` is unset.

## Routine checks

- Queue pending count is not growing without processing.
Expand Down
Loading