Skip to content

feat(cni): replace GoBGP direct calls with BGPVRFInstance CRD #113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
privateip merged 1 commit into from
Jun 16, 2026
Merged

feat(cni): replace GoBGP direct calls with BGPVRFInstance CRD #113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions deploy/containerlab/resources/bgp/dfw/bgpinstance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
galactic.io/plane: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Manual
routerID: "10.255.255.2"
routerIDSource: Auto
addressFamilies:
- afi: L2VPN
safi: EVPN
3 changes: 1 addition & 2 deletions deploy/containerlab/resources/bgp/iad/bgpinstance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
galactic.io/plane: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Manual
routerID: "10.255.255.1"
routerIDSource: Auto
addressFamilies:
- afi: L2VPN
safi: EVPN
3 changes: 1 addition & 2 deletions deploy/containerlab/resources/bgp/sjc/bgpinstance.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ spec:
galactic.io/plane: overlay
asNumber: 65000
listenPort: 1790
routerIDSource: Manual
routerID: "10.255.255.3"
routerIDSource: Auto
addressFamilies:
- afi: L2VPN
safi: EVPN
23 changes: 16 additions & 7 deletions docs/cni-sequence.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,42 +8,51 @@ sequenceDiagram
participant Veth as veth
participant Route as route
participant HD as host-device CNI
participant GoBGP as GoBGP
participant K8s as Kubernetes API
participant Kernel as Kernel (SRv6)
participant Cosmos as cosmos operator

rect rgb(220, 240, 220)
note over Multus,Kernel: cmdAdd — Container Attach
note over Multus,Cosmos: cmdAdd — Container Attach
Multus->>CNI: ADD (VPC, VPCAttachment, SRv6Locator, IPAM, terminations)
CNI->>CNI: parseConf()
CNI->>CNI: validate NODE_NAME env var
CNI->>VRF: Add(VPC, VPCAttachment)
CNI->>Veth: Add(VPC, VPCAttachment, MTU)
loop for each termination
CNI->>Route: Add(network, via, host-side dev)
end
CNI->>HD: ADD — move guest veth into pod netns, assign IPs via IPAM
CNI->>CNI: Base62ToHex(VPC, VPCAttachment)
CNI->>CNI: getNetworks() — IPAM addresses + termination networks
CNI->>GoBGP: AddPaths(SRv6Locator, VPCHex, VPCAttachmentHex, networks)
CNI->>K8s: List BGPProviders (by bgp.miloapis.com/node label)
CNI->>K8s: List BGPInstances (find instance whose providerSelector matches)
CNI->>CNI: compute RD = ASN:uint32(vpcHex)
CNI->>CNI: compute RT = ASN:uint32(vpcHex)
CNI->>K8s: CreateOrUpdate BGPVRFInstance (instanceRef, providerSelector, RD, RT)
CNI->>CNI: EncodeSRv6Endpoint(SRv6Locator, VPCHex, VPCAttachmentHex)
CNI->>Kernel: RouteIngressAdd(srv6Endpoint)
CNI->>Multus: PrintResult
K8s-->>Cosmos: BGPVRFInstance created/updated
note over Cosmos: reconciles VRF onto BGP provider (async)
end

rect rgb(240, 220, 220)
note over Multus,Kernel: cmdDel — Container Detach
note over Multus,Cosmos: cmdDel — Container Detach
Multus->>CNI: DEL (VPC, VPCAttachment, SRv6Locator, IPAM, terminations)
CNI->>CNI: parseConf()
CNI->>CNI: Base62ToHex(VPC, VPCAttachment)
CNI->>CNI: getNetworks()
CNI->>GoBGP: DeletePaths(SRv6Locator, VPCHex, VPCAttachmentHex, networks)
CNI->>CNI: EncodeSRv6Endpoint(SRv6Locator, VPCHex, VPCAttachmentHex)
CNI->>Kernel: RouteIngressDel(srv6Endpoint)
CNI->>K8s: Delete BGPVRFInstance
note over CNI,K8s: withdrawal signalled before local teardown so remote peers stop sending sooner
CNI->>HD: DEL — release IPAM, remove veth from pod netns
loop for each termination
CNI->>Route: Delete(network, via, host-side dev)
end
CNI->>Veth: Delete(VPC, VPCAttachment)
CNI->>VRF: Delete(VPC, VPCAttachment)
CNI->>Multus: PrintResult
K8s-->>Cosmos: BGPVRFInstance deleted
note over Cosmos: withdraws VRF from BGP provider (async)
end
```
161 changes: 0 additions & 161 deletions docs/plans/gobgp-provider.md
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ require (
github.com/prometheus/client_golang v1.23.2
github.com/spf13/cobra v1.10.2
github.com/vishvananda/netlink v1.3.2-0.20260610182031-c05a276ed0e0
go.miloapis.com/cosmos v0.0.0-20260613151556-25ccb5c88f4d
go.miloapis.com/cosmos v0.0.0-20260615212649-d08f7867c312
golang.org/x/sys v0.46.0
google.golang.org/grpc v1.81.1
k8s.io/api v0.36.0
k8s.io/apimachinery v0.36.1
k8s.io/client-go v0.36.0
sigs.k8s.io/controller-runtime v0.24.1
Expand Down Expand Up @@ -88,7 +89,6 @@ require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.3.2 // indirect
k8s.io/api v0.36.0 // indirect
k8s.io/apiextensions-apiserver v0.36.0 // indirect
k8s.io/klog/v2 v2.140.0 // indirect
k8s.io/kube-openapi v0.0.0-20260317180543-43fb72c5454a // indirect
Expand Down
12 changes: 2 additions & 10 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,6 @@ github.com/onsi/gomega v1.39.1 h1:1IJLAad4zjPn2PsnhH70V4DKRFlrCzGBNrNaru+Vf28=
github.com/onsi/gomega v1.39.1/go.mod h1:hL6yVALoTOxeWudERyfppUcZXjMwIMLnuSfruD2lcfg=
github.com/orcaman/concurrent-map/v2 v2.0.1 h1:jOJ5Pg2w1oeB6PeDurIYf6k9PQ+aTITr/6lP/L/zp6c=
github.com/orcaman/concurrent-map/v2 v2.0.1/go.mod h1:9Eq3TG2oBe5FirmYWQfYO5iH1q0Jv47PLaNK++uCdOM=
github.com/osrg/gobgp/v4 v4.5.0 h1:1jS4cMxUYSo36UfDyigLOLYEg6Oh+9I2mrnjjgAGCFk=
github.com/osrg/gobgp/v4 v4.5.0/go.mod h1:pgu8waqTvZUYl4eQuPrKNOaVwhHv7Zt9YymuzCaX7f8=
github.com/osrg/gobgp/v4 v4.6.0 h1:9ga/Pn3NUiM0Sv0K7YI+dy/uMYKyXOsu3dalXTmnX8I=
github.com/osrg/gobgp/v4 v4.6.0/go.mod h1:j1GLEuE20jm2YAoGmaHGb3y9lGH/KBgCBT4Ss5RY/wQ=
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
Expand Down Expand Up @@ -160,18 +158,14 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/vishvananda/netlink v1.3.2-0.20260505210927-99e979749d3e h1:P2oj+7IBc6qc2Ie+Ie07RRKdShl+6PN6MW87OQaUOHA=
github.com/vishvananda/netlink v1.3.2-0.20260505210927-99e979749d3e/go.mod h1:lEui7SPMd9fgxzHVGRAvTxsBGCF6PRH81o2kLWLWHgw=
github.com/vishvananda/netlink v1.3.2-0.20260610182031-c05a276ed0e0 h1:ljtwwUHGey840KIaywKtF5FHMuOtmw7xST3ZLIxT7kY=
github.com/vishvananda/netlink v1.3.2-0.20260610182031-c05a276ed0e0/go.mod h1:lEui7SPMd9fgxzHVGRAvTxsBGCF6PRH81o2kLWLWHgw=
github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY=
github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
go.miloapis.com/cosmos v0.0.0-20260609211521-d6cebe28f2aa h1:4Fp5JhfyG0HvD8FfqNid5xqW9vQ4XaBLZNnlM6HCtEg=
go.miloapis.com/cosmos v0.0.0-20260609211521-d6cebe28f2aa/go.mod h1:q7znijrggZ/Ac1wTE2bkV6DY/vQ4hU4D4t2Ts3blsVI=
go.miloapis.com/cosmos v0.0.0-20260613151556-25ccb5c88f4d h1:YIlEkBBOnQKEtn6F92qXU586DhZY8msxN+/V/7OmkIA=
go.miloapis.com/cosmos v0.0.0-20260613151556-25ccb5c88f4d/go.mod h1:q7znijrggZ/Ac1wTE2bkV6DY/vQ4hU4D4t2Ts3blsVI=
go.miloapis.com/cosmos v0.0.0-20260615212649-d08f7867c312 h1:bIGAtAgfMg8vUni2Te5m4qT2be4mxz9Gr3OJGJK+k7M=
go.miloapis.com/cosmos v0.0.0-20260615212649-d08f7867c312/go.mod h1:c99/Iy0c7lwmI46d4brOQyRc1JKjzkl9xRIcU3hnHo4=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
Expand Down Expand Up @@ -214,8 +208,6 @@ golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
Expand Down
Loading
Loading