Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
bde6b79
Initial plan
Copilot May 21, 2026
1a20410
Add operator architecture draft plan to README
Copilot May 21, 2026
b5fc486
Polish wording consistency in README plan
Copilot May 21, 2026
2455ee4
Update CRD plan fields per PR feedback
Copilot May 21, 2026
d518a57
Clarify repull and concurrency fields in README
Copilot May 21, 2026
fcb7ebf
Add feature-sliced AI planning docs
Copilot May 21, 2026
910370f
Expand CRD docs with slow-pull guidance
Copilot May 21, 2026
d291fe3
Add policy redesign proposals for pull pacing
Copilot May 21, 2026
4177d07
Clarify concurrency with concrete operator use cases
Copilot May 21, 2026
c3808e5
Simplify pull policy plans and drop redundant concurrency knob
Copilot May 21, 2026
850bfd6
Normalize pull policy field naming in docs
Copilot May 21, 2026
1cf446a
Clarify rollout limit field naming
Copilot May 21, 2026
1352a5d
Refine policy and worker semantics wording
Copilot May 21, 2026
b86e78f
Clarify pacing field semantics in policy docs
Copilot May 21, 2026
791ec52
Add concrete CR scenario examples with per-pool policy binding
Copilot May 22, 2026
078b3df
Fix grammar in example scenarios doc
Copilot May 22, 2026
d44a5a4
Add CRD naming and structure proposals with PrePullImageSet
Copilot May 22, 2026
a41cfd3
Simplify CRD naming proposals: drop PrePull prefix, add NodeImage and…
Copilot May 22, 2026
44b82d2
consolidate to Proposal C: CachedImage + CachedImageSet + PullPolicy …
Copilot May 22, 2026
a2e9fd3
remove maxUnavailableNodes: pulls never affect node schedulability, s…
Copilot May 22, 2026
72187f9
redesign DiscoveryPolicy: extensible sources list, secretRef for auth…
Copilot May 22, 2026
bf540ee
add discovery architecture plan: reconciliation flow, query contract,…
Copilot May 22, 2026
eb0abd7
Add system architecture plan (ai-docs/14-architecture.md)
Copilot May 22, 2026
4014f98
Add detailed implementation plan (ai-docs/15-implementation-plan.md)
Copilot May 22, 2026
5127b33
feat: implement puller operator with CRD types, reconcilers, pod buil…
Copilot May 22, 2026
f28f570
fix: address code review feedback - remove dead loop variable, fix er…
Copilot May 22, 2026
18f11f9
fix: resolve all golangci-lint issues
Copilot May 22, 2026
88a3632
feat: add observability, docs, dev tooling, E2E tests, and demo script
Copilot May 22, 2026
1c23c86
fix: decrement ActivePulls gauge on pod completion, improve E2E pacin…
Copilot May 22, 2026
bf4ea5e
Add proof-of-operation script and expected output documentation
Copilot May 22, 2026
06410bd
Add Prometheus + Registry to e2e tests and dev setup, fix docs workflow
Copilot May 22, 2026
252bc98
fix: update tests and E2E assertions for namespace-aware pod management
Copilot May 22, 2026
0623eec
feat: add local CI verification targets and docs to dev environment
Copilot May 22, 2026
0660ced
fix: E2E discovery tests - fix prometheus assertion, fix registry see…
Copilot May 22, 2026
a751be8
fix e2e
Breee May 23, 2026
614211c
feat(api): add imagePullPolicy, imagePullSecrets, repullInterval, res…
Breee May 24, 2026
643531c
feat(controller): backoff, pacing, pod watch, repull, digest capture
Breee May 24, 2026
7419e5b
feat(discovery): lookback, step params, error categorization
Breee May 24, 2026
5f5e004
test: e2e infra, kind config, failure/discovery test suites
Breee May 24, 2026
b409c64
feat(observability): grafana dashboard, helm updates, docs
Breee May 24, 2026
380d753
chore: gitignore hugo build lock
Breee May 24, 2026
5d227f2
feat(docs): add asciinema recordings to landing page
Breee May 24, 2026
54c8214
chore: ignore gen-ai-docs binary
Breee May 24, 2026
7def682
feat: add docs-gen tooling and generated agent files
Breee May 24, 2026
c916596
refactor(ai-docs): consolidate into single reference
Breee May 24, 2026
0214e7d
feat(docs): restructure Hugo site with new pages and layouts
Breee May 24, 2026
3e16cbe
Fix Makefile recipe syntax causing CI build failure
Copilot May 24, 2026
c4695aa
Finalize CI build fix validation
Copilot May 24, 2026
3a6b306
fix: add missing tabs to Makefile tool-install recipe lines
Copilot May 25, 2026
db11976
Fix cachedimage e2e manifest field name
Copilot May 25, 2026
3eb97a6
feat: add critical rules to agent instructions and split dev guide
Breee May 25, 2026
3eb5285
fix(e2e): use JMESPath condition filters and fix assertion values
Breee May 25, 2026
ab0de88
test: stabilize e2e scenarios
Copilot May 25, 2026
c5bf66f
fix: make e2e wait loops POSIX-safe
Copilot May 25, 2026
11f7fbb
test: harden e2e polling scripts
Copilot May 25, 2026
a35c643
feat: upgrade to Go 1.24, add renovate.json, add weekly release workflow
Copilot May 25, 2026
d0ea7ef
chore: upgrade project Go version pins to 1.26
Copilot May 25, 2026
34b9a51
fix: migrate golangci-lint to v2 and chainsaw to v0.2.15 for Go 1.26
Breee May 25, 2026
c1b8a1a
feat!: rebrand to Drop operator (drop.corewire.io)
Breee May 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .cursorrules
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Cursor Rules for Puller

## Critical Rules

1. ALWAYS read project files (Tiltfile, Makefile, source) before acting. Never guess.
2. Documentation: short, concise, high-level. No volatile details.
3. Simplicity over complexity. DRY is NOT always best. No premature optimization.
4. Kubernetes: use kubectl explain or read CRD types before suggesting specs.
5. Security: never expose secrets in code or docs.
6. Tilt handles the dev loop. tilt up does everything. Don't suggest manual commands for automated steps.

## Project Context
Kubernetes operator (Go 1.23.0, Kubebuilder, controller-runtime).
Module: github.com/Breee/puller
API group: puller.corewire.io/v1alpha1. All CRDs cluster-scoped.

## Key Commands
- Build: go build ./...
- Test: make test
- Lint: make lint
- CRD gen: make manifests
- Deepcopy gen: make generate
- All codegen: make codegen
- AI docs gen: make docs-gen

## Structure
- api/v1alpha1 — Package v1alpha1 contains API Schema definitions for the puller v1alpha1 API group.
- internal/controller — Reconciler implementations (one per CRD)
- internal/discovery — Discovery source interface + implementations
- internal/metrics — Prometheus metrics registration
- internal/pacing — Shared pacing engine for rate-limited pulls
- internal/podbuilder — Pure Pod construction function (no k8s client)
- charts/puller/ — Helm chart
- test/e2e/ — Chainsaw E2E tests
- hack/gen-ai-docs/ — generates all docs from source

## CRDs → Controllers
- CachedImage → internal/controller/cachedimage_controller.go
- CachedImageSet → internal/controller/cachedimageset_controller.go
- PullPolicy (config-only, no controller)
- DiscoveryPolicy → internal/controller/discoverypolicy_controller.go

## Conventions
- All CRDs are cluster-scoped
- Status uses metav1.Condition with type "Ready"
- No privileged containers — kubelet-based image pulls only
- Single responsibility reconcilers — one controller per CRD
- Pod builder is a pure function in internal/podbuilder/ (no k8s client)
- Pacing logic lives exclusively in internal/pacing/
- ownerReferences: CachedImageSet→CachedImage, controller→Pod
- Table-driven tests preferred; envtest for controllers
- Pods use nodeName placement + command: ["true"]
- Don't manually edit generated files — run make docs-gen

## Don't
- Edit generated files (zz_generated.deepcopy.go, config/crd/bases/, llms.txt, llms-full.txt, knowledge.yaml)
- Add privileged containers or CRI socket mounts
- Create namespaced CRDs
- Put pacing logic outside internal/pacing/
24 changes: 24 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"name": "Kubebuilder DevContainer",
"image": "docker.io/golang:1.26",
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {},
"ghcr.io/devcontainers/features/git:1": {}
},

"runArgs": ["--network=host"],

"customizations": {
"vscode": {
"settings": {
"terminal.integrated.shell.linux": "/bin/bash"
},
"extensions": [
"ms-kubernetes-tools.vscode-kubernetes-tools",
"ms-azuretools.vscode-docker"
]
}
},

"onCreateCommand": "bash .devcontainer/post-install.sh"
}
23 changes: 23 additions & 0 deletions .devcontainer/post-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash
set -x

curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
mv ./kind /usr/local/bin/kind

curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/linux/amd64
chmod +x kubebuilder
mv kubebuilder /usr/local/bin/

KUBECTL_VERSION=$(curl -L -s https://dl.k8s.io/release/stable.txt)
curl -LO "https://dl.k8s.io/release/$KUBECTL_VERSION/bin/linux/amd64/kubectl"
chmod +x kubectl
mv kubectl /usr/local/bin/kubectl

docker network create -d=bridge --subnet=172.19.0.0/24 kind

kind version
kubebuilder version
docker --version
go version
kubectl version --client
11 changes: 11 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# More info: https://docs.docker.com/engine/reference/builder/#dockerignore-file
# Ignore build and test binaries.
bin/

# Docs and dev artifacts
docs/
ai-docs/
hack/
test/
.github/
*.md
19 changes: 19 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
root = true

[*]
indent_style = tab
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

[*.{yaml,yml}]
indent_style = space
indent_size = 2

[*.md]
trim_trailing_whitespace = false

[Makefile]
indent_style = tab
79 changes: 79 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Copilot Instructions for Drop

## Critical Rules

1. **ALWAYS read project files before acting.** Read the Tiltfile, Makefile, and relevant source before writing docs, suggesting workflows, or describing how things work. Never guess based on general knowledge.
2. **Documentation must be short and concise.** Focus on high-level overview and usage. Avoid volatile implementation details. Avoid information that will change frequently.
3. **Simplicity over complexity.** If a simple solution exists, use it. DRY is NOT always best. No premature optimization.
4. **Kubernetes: always verify.** Use `kubectl explain` or read the CRD types before suggesting field values or resource specs.
5. **Security-conscious.** Never expose secrets in code or docs. Follow secure coding practices.
6. **Tilt handles the dev loop.** `tilt up` does everything: cluster creation, build, deploy, port-forwards, Hugo docs, e2e infra, dev samples. Don't suggest manual commands for things Tilt automates.

## Project

Kubernetes operator (Go 1.23.0, Kubebuilder, controller-runtime) that pre-caches container images on cluster nodes.
API group: `drop.corewire.io/v1alpha1`. All CRDs are cluster-scoped.

## Build Commands

```bash
make generate # regenerate deepcopy
make manifests # regenerate CRD + RBAC YAML
make codegen # both of the above
go build ./... # compile
make test # unit tests (envtest)
make test-e2e # e2e tests (chainsaw, needs kind)
make lint # golangci-lint
make docs-gen # regenerate AI docs from source
```

## Code Conventions

- All CRDs are cluster-scoped
- Status uses metav1.Condition with type "Ready"
- No privileged containers — kubelet-based image pulls only
- Single responsibility reconcilers — one controller per CRD
- Pod builder is a pure function in internal/podbuilder/ (no k8s client)
- Pacing logic lives exclusively in internal/pacing/
- ownerReferences: CachedImageSet→CachedImage, controller→Pod
- Table-driven tests preferred; envtest for controllers
- Pods use nodeName placement + command: ["true"]
- Don't manually edit generated files — run make docs-gen

## Testing Patterns

- Controller tests use envtest (`internal/controller/*_test.go`)
- Table-driven tests preferred
- E2E uses Kyverno Chainsaw in `test/e2e/`
- Test fixtures in `config/samples/` and `hack/dev-samples.yaml`

## CRD Quick Reference

| Kind | Controller | Purpose |
|------|-----------|---------|
| CachedImage | internal/controller/cachedimage_controller.go | CachedImage is the Schema for the cachedimages API. |
| CachedImageSet | internal/controller/cachedimageset_controller.go | CachedImageSet is the Schema for the cachedimagesets API. |
| PullPolicy | | PullPolicy is the Schema for the pullpolicies API. It is a configuration-only resource with no status. |
| DiscoveryPolicy | internal/controller/discoverypolicy_controller.go | DiscoveryPolicy is the Schema for the discoverypolicies API. |

## Package Dependency Graph

```
api/v1alpha1 — Package v1alpha1 contains API Schema definitions for the drop v1alpha1 API group.
internal/controller — Reconciler implementations (one per CRD)
imports: api/v1alpha1, internal/discovery, internal/metrics, internal/pacing, internal/podbuilder
internal/discovery — Discovery source interface + implementations
internal/metrics — Prometheus metrics registration
internal/pacing — Shared pacing engine for rate-limited pulls
imports: api/v1alpha1, internal/podbuilder
internal/podbuilder — Pure Pod construction function (no k8s client)
imports: api/v1alpha1
```

## Don'ts

- Don't add CRI socket access or privileged containers — we use kubelet image pulls only
- Don't put pacing logic outside `internal/pacing/`
- Don't create namespaced CRDs — all resources are cluster-scoped
- Don't manually edit generated files (`zz_generated.deepcopy.go`, `config/crd/bases/`)
- Don't manually edit `llms.txt`, `llms-full.txt`, `.cursorrules`, `AGENTS.md` — run `make docs-gen`
121 changes: 121 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]

permissions:
contents: read

# Each job maps to a local make target for easy debugging:
# lint → make lint
# test → make test
# build → make build
# helm-lint → make helm-lint && make helm-template
# docs-build → make docs-build
# e2e → make e2e-local

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- uses: golangci/golangci-lint-action@v9
with:
version: v2.12.2

test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: Run tests
run: make test

build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: Build
run: make build

helm-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: azure/setup-helm@v4
- name: Lint Helm chart
run: helm lint charts/drop
- name: Template Helm chart
run: helm template drop charts/drop

docs-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.26'
cache: false
- name: Setup Hugo
uses: peaceiris/actions-hugo@v2
with:
hugo-version: 'latest'
extended: true
- name: Build docs
working-directory: docs
run: |
hugo mod get
hugo --minify

e2e:
runs-on: ubuntu-latest
needs: [build]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: Install kind
run: |
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.24.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
- name: Create kind cluster
run: make kind-create KIND=kind
- name: Build and load image
run: |
make docker-build IMG=controller:ci
make kind-load KIND=kind IMG=controller:ci
- name: Install CRDs
run: |
make controller-gen
make manifests
kubectl apply -f config/crd/bases/
- name: Deploy E2E infrastructure (Prometheus + Registry)
run: make e2e-infra
- name: Deploy operator
run: |
helm install drop charts/drop \
--namespace drop-system \
--create-namespace \
--set image.repository=controller \
--set image.tag=ci \
--set image.pullPolicy=Never \
--set leaderElection.enabled=false \
--set metrics.enabled=true \
--set metrics.secureServing=false \
--wait --timeout 120s
- name: Run E2E tests
run: make test-e2e
Loading
Loading