Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions cmd/galactic-agent/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,19 +30,13 @@ func newRootCommand() *cobra.Command {
},
}

cmd.Flags().StringVar(&opts.MetricsAddr, "metrics-addr", ":8082", "Address to serve Prometheus metrics on")
cmd.Flags().StringVar(&opts.HealthAddr, "health-addr", ":8083", "Address to serve health/readiness probes on")
cmd.Flags().StringVar(&opts.NodeName, "node-name", "", "Override node name (default: NODE_NAME env var)")
cmd.Flags().StringVar(&opts.Plane, "plane", "overlay",
"BGP plane label published on the BGPProvider (e.g. overlay, overlay-rr)")
cmd.Flags().BoolVar(&opts.GoBGPEnabled, "gobgp-enabled", false,
"Enable embedded GoBGP and publish BGPProvider")
cmd.Flags().IntVar(&opts.GoBGPAPIPort, "gobgp-api-port", 50051,
"Port for the embedded GoBGP gRPC API (cosmos dials this)")
cmd.Flags().StringVar(&opts.GoBGPLogLevel, "gobgp-log-level", "panic",
"GoBGP internal log level (debug, info, warn, error, panic)")
cmd.Flags().IntVar(&opts.GRPCHealthPort, "grpc-health-port", 8084,
"Port for the gRPC health service (used by Kubernetes readiness probes)")
cmd.Flags().StringVar(&opts.Role, "role", "overlay",
"Agent role published on the BGPProvider label galactic.io/role (overlay, overlay-rr)")
cmd.Flags().IntVar(&opts.Port, "port", 33438,
"Port for the gRPC server that cosmos uses to configure the BGP provider")
cmd.Flags().IntVar(&opts.HealthPort, "health-port", 5000,
"Port for the gRPC health server (Kubernetes liveness and readiness probes)")

return cmd
}
91 changes: 0 additions & 91 deletions deploy/containerlab/examples/nginx-vpc-test.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/dfw/bgpinstance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
spec:
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Auto
Expand Down
2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/dfw/bgppeer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ spec:
instanceRef: overlay
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
address: "fc00:0:8::1"
asNumber: 65000
remotePort: 1179
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
spec:
providerSelector:
matchLabels:
galactic.io/plane: overlay-rr
galactic.io/role: overlay-rr
asNumber: 65000
routerIDSource: Manual
routerID: "10.255.255.4"
Expand Down
2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/iad/bgpinstance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
spec:
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Auto
Expand Down
6 changes: 3 additions & 3 deletions deploy/containerlab/resources/bgp/iad/bgppeer-rr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ spec:
instanceRef: overlay-rr
providerSelector:
matchLabels:
galactic.io/plane: overlay-rr
galactic.io/role: overlay-rr
address: "fc00:0:2::1"
asNumber: 65000
passive: true
Expand All @@ -23,7 +23,7 @@ spec:
instanceRef: overlay-rr
providerSelector:
matchLabels:
galactic.io/plane: overlay-rr
galactic.io/role: overlay-rr
address: "fc00:0:3::1"
asNumber: 65000
passive: true
Expand All @@ -40,7 +40,7 @@ spec:
instanceRef: overlay-rr
providerSelector:
matchLabels:
galactic.io/plane: overlay-rr
galactic.io/role: overlay-rr
address: "fc00:0:4::1"
asNumber: 65000
passive: true
Expand Down
2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/iad/bgppeer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ spec:
instanceRef: overlay
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
address: "fc00:0:8::1"
asNumber: 65000
remotePort: 1179
Expand Down
2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/sjc/bgpinstance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
spec:
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Auto
Expand Down
2 changes: 1 addition & 1 deletion deploy/containerlab/resources/bgp/sjc/bgppeer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ spec:
instanceRef: overlay
providerSelector:
matchLabels:
galactic.io/plane: overlay
galactic.io/role: overlay
address: "fc00:0:8::1"
asNumber: 65000
remotePort: 1179
Expand Down
11 changes: 11 additions & 0 deletions deploy/containerlab/resources/cosmos/daemonset-patch.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: cosmos
namespace: bgp-system
spec:
template:
spec:
containers:
- name: cosmos-operator
imagePullPolicy: Never
13 changes: 13 additions & 0 deletions deploy/containerlab/resources/cosmos/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../cosmos-config/deploy/
images:
- name: ghcr.io/milo-os/cosmos
newName: cosmos
newTag: latest
patches:
- path: daemonset-patch.yaml
target:
kind: DaemonSet
name: cosmos
21 changes: 8 additions & 13 deletions deploy/containerlab/resources/overlay/base/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,9 @@ spec:
image: galactic-agent:latest
imagePullPolicy: Never
args:
- --gobgp-enabled=true
- --gobgp-api-port=50051
- --gobgp-log-level=debug
- --metrics-addr=:8082
- --health-addr=:8083
- --grpc-health-port=8084
- --plane=overlay
- --port=33438
- --health-port=5000
- --role=overlay
env:
- name: NODE_NAME
valueFrom:
Expand All @@ -43,14 +39,13 @@ spec:
add:
- NET_ADMIN
livenessProbe:
httpGet:
path: /healthz
port: 8083
grpc:
port: 5000
initialDelaySeconds: 15
periodSeconds: 20
readinessProbe:
httpGet:
path: /readyz
port: 8083
grpc:
port: 5000
service: readyz
initialDelaySeconds: 5
periodSeconds: 10
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,13 @@ spec:
containers:
- name: galactic-agent
args:
- --gobgp-enabled=true
- --gobgp-api-port=50052
- --gobgp-log-level=debug
- --metrics-addr=:9085
- --health-addr=:9086
- --grpc-health-port=9087
- --plane=overlay-rr
- --port=33440
- --health-port=5000
- --role=overlay-rr
livenessProbe:
httpGet:
path: /healthz
port: 9086
grpc:
port: 5000
readinessProbe:
httpGet:
path: /readyz
port: 9086
grpc:
port: 5000
service: readyz
15 changes: 3 additions & 12 deletions deploy/containerlab/scripts/install-overlay.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,9 @@ deploy_cosmos() {
echo "Deploying cosmos operator to ${node}..."
# Copy the entire config/ directory so that the kustomization's ../crd reference resolves
docker cp "${COSMOS_DIR}/config" "${node}:/galactic/resources/cosmos-config/"
# Use locally built image; patch imagePullPolicy to Never
docker exec "${node}" kubectl apply -k /galactic/resources/cosmos-config/deploy/
docker exec "${node}" kubectl patch daemonset bgp \
-n bgp-system \
--type='json' \
-p='[{"op":"replace","path":"/spec/template/spec/containers/0/image","value":"cosmos:latest"},{"op":"replace","path":"/spec/template/spec/containers/0/imagePullPolicy","value":"Never"}]'
# Copy the local overlay (image + imagePullPolicy overrides for Kind)
docker cp "${RESOURCES_DIR}/cosmos" "${node}:/galactic/resources/cosmos-overlay/"
docker exec "${node}" kubectl apply -k /galactic/resources/cosmos-overlay/
}

apply_overlay dfw-control-plane dfw
Expand All @@ -37,10 +34,4 @@ deploy_cosmos dfw-control-plane
deploy_cosmos sjc-control-plane
deploy_cosmos iad-control-plane

# iad hosts the overlay-rr BGPInstance (route reflector), which requires infra role.
echo "Patching cosmos clusterRole to infra on iad-control-plane..."
docker exec iad-control-plane kubectl patch configmap cosmos-config \
-n cosmos-system --type=merge -p '{"data":{"clusterRole":"infra"}}'
docker exec iad-control-plane kubectl rollout restart daemonset bgp -n bgp-system

echo "Done."
30 changes: 11 additions & 19 deletions deploy/galactic-agent/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,37 +22,29 @@ spec:
- name: galactic-agent
image: ghcr.io/datum-cloud/galactic:latest
args:
- --gobgp-enabled=true
- --gobgp-api-port=50051
- --gobgp-log-level=panic
- --metrics-addr=:8082
- --health-addr=:8083
- --grpc-health-port=8084
- --port=33438
- --health-port=5000
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
ports:
- name: metrics
containerPort: 8082
- name: grpc
containerPort: 33438
protocol: TCP
- name: healthz
containerPort: 8083
protocol: TCP
- name: grpc-health
containerPort: 8084
- name: health
containerPort: 5000
protocol: TCP
livenessProbe:
httpGet:
path: /healthz
port: healthz
grpc:
port: 5000
initialDelaySeconds: 10
periodSeconds: 30
readinessProbe:
httpGet:
path: /readyz
port: healthz
grpc:
port: 5000
service: readyz
initialDelaySeconds: 5
periodSeconds: 10
securityContext:
Expand Down
13 changes: 6 additions & 7 deletions docs/agent-startup.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ sequenceDiagram
participant GoBGP
participant Kubernetes

Agent->>Agent: register Prometheus metrics
Agent->>Agent: start gRPC health server (NOT_SERVING)
Agent->>Agent: start health gRPC server (--health-port, liveness SERVING immediately)
Agent->>Agent: start provider gRPC server (--port, BGPProviderService only)
Agent->>GoBGP: start embedded server
Agent->>GoBGP: WaitReady — poll gRPC API port (30s timeout)
Agent->>Kubernetes: EnsureGoBGPProvider (create/update BGPProvider CR)
Agent->>Agent: mark gRPC health SERVING
Agent->>Agent: mgr.Start — controller-runtime loop
Note over Agent: on shutdown: DeleteGoBGPProvider, GracefulStop gRPC
Agent->>Kubernetes: EnsureGoBGPProvider (create/update BGPProvider CR with --port address)
Agent->>GoBGP: WaitReady — poll in-process API (30s timeout)
Agent->>Agent: mark readyz SERVING
Note over Agent: on shutdown: mark readyz NOT_SERVING, GracefulStop both gRPC servers
```
Loading
Loading