Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ jobs:
runs-on: ubuntu-latest
outputs:
rust: ${{ steps.filter.outputs.rust }}
helm: ${{ steps.filter.outputs.helm }}
steps:
- uses: actions/checkout@v7
- uses: dorny/paths-filter@v4
Expand All @@ -28,6 +29,9 @@ jobs:
- 'Cargo.lock'
- 'deny.toml'
- '.github/workflows/ci.yml'
helm:
- 'chart/**'
- '.github/workflows/ci.yml'

lint:
name: Lint
Expand Down Expand Up @@ -87,6 +91,44 @@ jobs:
- name: Check dependencies
run: cargo deny check

helm:
name: Lint and validate Helm chart
needs: changes
if: needs.changes.outputs.helm == 'true'
runs-on: ubuntu-latest
env:
# kubeconform release used to validate rendered manifests against the
# Kubernetes schemas.
KUBECONFORM_VERSION: v0.6.7
steps:
- uses: actions/checkout@v7
- uses: azure/setup-helm@v5
- name: Install kubeconform
run: |
curl -sSL "https://github.com/yannh/kubeconform/releases/download/${KUBECONFORM_VERSION}/kubeconform-linux-amd64.tar.gz" \
| tar -xz -C /usr/local/bin kubeconform
- name: Lint, render, and validate the chart
shell: bash
run: |
set -euo pipefail
helm lint chart
# Render under a few value combinations so the conditional templates
# (Ingress, emptyDir, ServiceMonitor, CA trust, auth secrets) are all
# exercised, then validate each render against the Kubernetes schemas.
# -strict rejects unknown fields; -ignore-missing-schemas skips CRDs
# (the ServiceMonitor).
for args in \
"" \
"--set ingress.enabled=true" \
"--set persistence.enabled=false" \
"--set serviceMonitor.enabled=true" \
"--set caTrust.enabled=true,caTrust.configMapName=ca" \
"--set upstreamAuth.existingSecret=up,serveToken.existingSecret=srv"; do
echo "-- helm template chart $args"
helm template release chart $args \
| kubeconform -strict -summary -ignore-missing-schemas
done

# Spelling runs on everything, including docs.
typos:
name: Typos
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ env:
# instead of drifting (the hardcoded name once lagged a rename and published under
# the old name). `github.repository` is `owner/repo`, lowercase as GHCR requires.
IMAGE: ghcr.io/${{ github.repository }}
# OCI namespace for the Helm chart, shared across this owner's charts. The chart is
# pushed as `<owner>/charts/git-cache-proxy`.
CHART_REPO: oci://ghcr.io/${{ github.repository_owner }}/charts

jobs:
release:
Expand Down Expand Up @@ -142,3 +145,30 @@ jobs:
run: cargo publish --locked
env:
CARGO_REGISTRY_TOKEN: ${{ steps.auth.outputs.token }}

# Package and push the Helm chart to GHCR as an OCI artifact, in the same run that
# cut the release, at the tag knope just created. The published GHCR package starts
# private; make it public once to allow anonymous `helm pull`.
publish-chart:
name: Publish chart
needs: release
if: needs.release.outputs.released == 'true'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v7
with:
ref: v${{ needs.release.outputs.version }}
- uses: azure/setup-helm@v5
# Version the chart in lockstep with the app: package at the released version
# (overriding the static Chart.yaml version) with the matching appVersion, so a
# pulled chart deploys the image it was cut with.
- name: Package and push the chart
run: |
helm package chart -d dist \
--version "${{ needs.release.outputs.version }}" \
--app-version "${{ needs.release.outputs.version }}"
echo "${{ secrets.GITHUB_TOKEN }}" | helm registry login ghcr.io -u "${{ github.actor }}" --password-stdin
helm push dist/git-cache-proxy-*.tgz "${{ env.CHART_REPO }}"
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,17 @@ rather than a fully distroless/`FROM scratch` image. Removing that dependency
(and enabling a git-free image) means moving the git plumbing in-process to a
Rust library - see the roadmap below.

On Kubernetes, a Helm chart lives in [`chart/`](./chart) (single-writer
Deployment, `/healthz`+`/readyz` probes, cache PVC, optional Ingress and
Prometheus `ServiceMonitor`):

```shell
helm install git-cache-proxy oci://ghcr.io/rolandjitsu/charts/git-cache-proxy \
--set upstream=https://your-git-host.example.com
```

See the [chart README](./chart/README.md) for the full values reference.

## Status / scope

Working and end-to-end tested against both Git wire protocol versions — the
Expand All @@ -237,8 +248,6 @@ rely on it.

Not yet implemented, in rough priority order:

- A Helm chart for Kubernetes deployment (liveness/readiness probes,
single-writer RWO PVC, metrics scrape), shipped in-repo.
- A background/scheduled refresh option (today every `info/refs` triggers an
on-demand, TTL-coalesced fetch).
- No external `git` binary: move the plumbing in-process to a Rust library
Expand Down
6 changes: 6 additions & 0 deletions chart/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.DS_Store
.git/
.gitignore
*.tmproj
*.orig
*.bak
22 changes: 22 additions & 0 deletions chart/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
apiVersion: v2
name: git-cache-proxy
description: Read-only caching proxy for Git - serves clones/fetches from an in-region mirror, pulling only deltas from upstream.
type: application
# Chart version. Overridden at release time with the app version (see
# release.yml); this static value is for installs straight from the git source.
version: 0.1.0
# The git-cache-proxy release this chart deploys by default - the image tag used
# when image.tag is empty. Set to the released version at publish time.
appVersion: "0.1.8"
home: https://github.com/rolandjitsu/git-cache-proxy
sources:
- https://github.com/rolandjitsu/git-cache-proxy
keywords:
- git
- cache
- proxy
- ci
- mirror
maintainers:
- name: Roland Groza
url: https://github.com/rolandjitsu
124 changes: 124 additions & 0 deletions chart/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# git-cache-proxy

A Helm chart that deploys [git-cache-proxy](https://github.com/rolandjitsu/git-cache-proxy),
a read-only caching proxy for Git. It sits between CI machines and an origin git server,
keeps a local bare mirror fresh (incremental pull from upstream), and serves clones/fetches
from that mirror - so the bulk history is served in-region and only the delta crosses the
WAN. It is strictly pull-only: it never pushes and never proactively replicates.

## Install

```sh
helm install git-cache-proxy oci://ghcr.io/rolandjitsu/charts/git-cache-proxy \
--set upstream=https://your-git-host.example.com
```

Pin a version with `--version`, and override defaults with `-f my-values.yaml` or `--set`.
See [Configuration](#configuration).

Clients then clone through the Service as if it were the origin:

```sh
git clone http://git-cache-proxy.<namespace>.svc:8080/<owner>/<repo>.git
```

## Upstream and auth

`upstream` is the origin base URL (default `https://github.com`); requested repo paths are
appended to it. To cache private repos, give the proxy a read-only upstream credential as a
Secret holding the full HTTP `Authorization` header, and reference it:

```sh
kubectl create secret generic gcp-upstream \
--from-literal=auth-header="Authorization: Bearer $TOKEN"

helm install git-cache-proxy ... \
--set upstreamAuth.existingSecret=gcp-upstream
```

The header is injected via an env var, so the token never appears in the process argv. For
GitLab, a personal access token goes in as HTTP Basic: the header value is
`Authorization: Basic <base64 of oauth2:$PAT>`.

## Serving auth

By default the proxy serves anonymously, which is intended for a network-restricted
deployment. Anyone who can reach the port can read **every mirrored repo** (the proxy holds
one upstream credential). Restrict who can reach it (private network, NetworkPolicy, mTLS at
the ingress), and/or require a client bearer token via `serveToken.existingSecret`. The
proxy speaks plain HTTP, so terminate TLS in front of it on any untrusted network.

## Single writer (not HA)

The chart runs a single replica with the `Recreate` strategy. The bare mirrors live on one
`ReadWriteOnce` volume and concurrent fetches are already coalesced in-process, so a second
replica would only contend for the same PVC; `Recreate` ensures the PVC detaches from the
old pod before the new one attaches. `replicas` is therefore not exposed. An evicted or lost
mirror is transparently re-cloned on the next request, so an `emptyDir`
(`persistence.enabled=false`) is a valid choice for a pure accelerator.

## Persistence and eviction

With `persistence.enabled` (default), the chart creates a PVC of `persistence.size` on
`persistence.storageClassName` (empty = cluster default), or mounts
`persistence.existingClaim` if set. Bound the cache with `config.cacheMaxMb`: when the total
exceeds it, least-recently-used idle mirrors are evicted in the background until back under.
Leaving it `0` (unlimited) lets the volume grow until full, so set it whenever the volume is
bounded.

## Metrics

The proxy exposes Prometheus metrics at `/metrics` on the Service port: per-repo request and
upstream counters, cache-size gauges, and `*_duration_seconds` fetch/serve latency
histograms. Scrape it with pod annotations (`podAnnotations`) or, with the Prometheus
Operator, set `serviceMonitor.enabled=true`.

## Ingress

A `ClusterIP` Service is exposed by default; reach it in-cluster or port-forward it. Set
`ingress.enabled=true` for a standard `networking.k8s.io/v1` Ingress (configurable
`className`, `host`, `annotations`, `tls`). For a non-standard controller (e.g. a Traefik
`IngressRoute` CRD), leave the Ingress off and manage the route as a separate manifest
pointing at the Service.

## Configuration

| Key | Default | Description |
| --- | --- | --- |
| `image.repository` | `ghcr.io/rolandjitsu/git-cache-proxy` | Image; override for a fork/mirror |
| `image.tag` | `""` | Image tag; empty uses the chart `appVersion` |
| `image.pullPolicy` | `IfNotPresent` | |
| `imagePullSecrets` | `[]` | Pull secrets for a private registry |
| `nameOverride` / `fullnameOverride` | `""` | Override the generated names |
| `upstream` | `https://github.com` | Origin git base URL; repo paths are appended |
| `upstreamAuth.existingSecret` | `""` | Secret with the full upstream `Authorization` header (empty = anonymous) |
| `upstreamAuth.key` | `auth-header` | Key in that Secret |
| `serveToken.existingSecret` | `""` | Secret with a client bearer token to require (empty = anonymous) |
| `serveToken.key` | `token` | Key in that Secret |
| `config.fetchTtlSeconds` | `10` | Skip upstream fetch if refreshed within this window (`0` = always) |
| `config.cacheMaxMb` | `0` | Cap on on-disk cache, MiB; evicts LRU idle mirrors (`0` = unlimited) |
| `config.maxConcurrentRequests` | `64` | Max concurrent in-flight requests (`0` = unlimited) |
| `config.maxDecodedBodyMb` | `512` | Cap on a decoded upload-pack request body, MiB |
| `config.logLevel` | `info` | Log filter directive |
| `config.logFormat` | `text` | `text` or `json` |
| `persistence.enabled` | `true` | Mount a PVC for the cache; `false` uses an emptyDir |
| `persistence.existingClaim` | `""` | Use this PVC instead of creating one |
| `persistence.size` | `20Gi` | Created PVC size |
| `persistence.storageClassName` | `""` | StorageClass for the PVC; empty = cluster default |
| `persistence.accessModes` | `[ReadWriteOnce]` | PVC access modes |
| `persistence.emptyDirSizeLimit` | `20Gi` | emptyDir limit when persistence is off |
| `service.type` | `ClusterIP` | Service type |
| `service.port` | `8080` | Service port and the port the container binds |
| `resources` | `{}` | Pod resource requests/limits |
| `terminationGracePeriodSeconds` | `60` | Drain window for in-flight clones on shutdown |
| `podAnnotations` | `{}` | Extra pod annotations (e.g. a metrics scraper) |
| `podSecurityContext` / `securityContext` | `{}` | Pod/container security contexts (empty = image defaults) |
| `nodeSelector` / `tolerations` / `affinity` | `{}` / `[]` / `{}` | Scheduling |
| `caTrust.enabled` | `false` | Mount a private-CA bundle and point git at it |
| `caTrust.configMapName` / `caTrust.key` | `""` / `ca-certificates.crt` | Source ConfigMap and key |
| `serviceMonitor.enabled` | `false` | Render a Prometheus Operator ServiceMonitor |
| `serviceMonitor.interval` / `.scrapeTimeout` / `.labels` | `30s` / `10s` / `{}` | ServiceMonitor config |
| `ingress.enabled` | `false` | Render a standard Ingress |
| `ingress.className` / `.host` / `.path` / `.pathType` / `.annotations` / `.tls` | see `values.yaml` | Ingress config |

Full defaults and inline comments: [`values.yaml`](./values.yaml).
30 changes: 30 additions & 0 deletions chart/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
git-cache-proxy is running as {{ include "git-cache-proxy.fullname" . }}, caching
{{ .Values.upstream }}.

Point CI clients at it in-cluster (it speaks git smart-HTTP, so clone/fetch as if
it were the origin):

git clone http://{{ include "git-cache-proxy.fullname" . }}.{{ .Release.Namespace }}.svc:{{ .Values.service.port }}/<owner>/<repo>.git

{{ if .Values.ingress.enabled -}}
Externally, via the Ingress:

http://{{ .Values.ingress.host }}{{ .Values.ingress.path }}
{{ else -}}
No Ingress is enabled. Reach it in-cluster (above), port-forward the Service, or
set ingress.enabled=true.
{{ end }}
Health and metrics:

kubectl -n {{ .Release.Namespace }} port-forward svc/{{ include "git-cache-proxy.fullname" . }} {{ .Values.service.port }}:{{ .Values.service.port }}
# then: curl localhost:{{ .Values.service.port }}/readyz and /metrics

{{ if not .Values.upstreamAuth.existingSecret -}}
No upstream auth is set, so only public repos on {{ .Values.upstream }} will work.
Set upstreamAuth.existingSecret to a Secret holding the full Authorization header
to cache private repos.
{{ end -}}
{{ if and (eq (int .Values.config.cacheMaxMb) 0) .Values.persistence.enabled -}}
config.cacheMaxMb is 0 (unlimited): the cache volume can grow until full. Set it
to bound on-disk usage with LRU eviction.
{{ end -}}
24 changes: 24 additions & 0 deletions chart/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{{/*
Chart name, overridable via nameOverride.
*/}}
{{- define "git-cache-proxy.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end }}

{{/*
Fully qualified app name. Truncated at 63 chars for the Kubernetes name limit.
Used as the resource name and as the stable `app` selector label - do not change
its shape, since editing a Deployment selector is a breaking in-place upgrade.
*/}}
{{- define "git-cache-proxy.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
Loading