feat(gateway): Add galactic-gateway binary - #352
Conversation
9a8c9b2 to
d34659a
Compare
bed30c2 to
19e0d98
Compare
d34659a to
623af9e
Compare
19e0d98 to
2a23577
Compare
623af9e to
b9c9b57
Compare
2a23577 to
66f6fa4
Compare
ecv
left a comment
There was a problem hiding this comment.
Approving on content. CI here is red from the module replace inherited from #351, tracked in #358, not from anything in this PR.
The binary follows the galactic-router precedent closely, and the two things I most wanted to check are right: no leader election, which is correct for a per-node DaemonSet where every node needs its own datapath, and complete cleanup on every error path in setupGatewayDatapath.
The keep-alive var deserves the comment it has. A GC'd link silently detaching while every control-plane signal still reads healthy is a genuinely nasty failure, and it is worth having found that live rather than in production.
Four follow-ups filed as #360. The one worth acting on is ordering: the health server answers SERVING before the datapath is attached, so there is a window where the probe says ready and no traffic can be intercepted. Bounded, since a failed attach exits the process, and there is no Service routing on readiness. Still the wrong way round.
The rest are small: a health server that stops serving is logged past rather than fatal, an IPv4 gateway address passes startup validation and fails deeper in, and the interface lookup sits after the attach that would already have failed on it.
Introduces galactic-gateway as its own binary, hosting the edge XDP NAT+LB gateway engine (previous two branches) and the NetworkGateway/NetworkRule reconcilers (previous branch) separately from galactic-router's tenant BGP process, so a crash on either side no longer takes the other down with it. Tenant BGP (GoBGP + the BGPRouter/BGPPeer/BGPAdvertisement/BGPPolicy/BGPVRFInstance reconcilers) still runs in galactic-router, co-located on the same gateway node. - cmd/galactic-gateway/main.go, root.go: manager setup, flags/env config, RBAC pre-flight -- no BGP runtime/RuntimeManager/BGP-family reconciler here at all. - cmd/galactic-gateway/gateway.go: setupGatewayDatapath, the load/attach/wire-up entry point, moved unchanged from cmd/galactic-router/gateway.go. - internal/config/gateway.go: GatewayConfig (metrics/grpc-health ports distinct from every other galactic-* process on the same hostNetwork node, node name, public interface, SRv6 address). - Taskfile.yaml: builds bin/galactic-gateway alongside the other binaries. Fourth branch in the edge-gateway stack; builds on feat/edge-gateway-03-controllers.
66f6fa4 to
de11cb4
Compare
galactic-gateway's health server set SERVING right after registering it, before setupGatewayDatapath ever ran. grpchealth.NewServer() additionally defaults the "" overall-health service to SERVING on its own, so the process was already answering healthy from the moment the server started -- not merely from that explicit call -- for the entire window before the XDP program was loaded and attached. Any probe watching during that window saw ready before the thing being probed existed, masking a silent failure: traffic that should have been intercepted routed normally instead. Three smaller issues sat alongside it: - The health server's Serve() goroutine logged a failure and kept going. If it stopped serving, the process ran on with no health signal at all and nothing restarted it. - The gateway's own SRv6 address was checked for being parseable, not for being the right family. An IPv4 address passed GatewayConfig.Validate and only failed later, deeper in, at kerneldatapath.go's identical Is6()/Is4In6() check. - setupGatewayDatapath looked up the public interface with net.InterfaceByName after edgeattach.Attach had already resolved the same interface. Attach would already have failed if it were missing, so the check could never fire. Fixes, in cmd/galactic-gateway/root.go, gateway.go, and internal/config/gateway.go: - Explicitly set NOT_SERVING when the health server is registered (overriding grpchealth's own SERVING default), and only flip to SERVING once setupGatewayDatapath has attached the datapath and constructed the rule table. - Wrap ctrl.SetupSignalHandler()'s context with context.WithCancelCause. A Serve() failure now cancels it with that error as the cause, which mgr.Start propagates out of runCmd as a fatal error -- the same as any other startup failure -- instead of a dropped log line. context.Cause is checked against context.Canceled to still treat an ordinary signal-triggered shutdown as success. - GatewayConfig.Validate now parses SRv6Address and rejects anything that isn't a native IPv6 address, with a message that names the problem instead of surfacing it later as a kernel-datapath error. - Dropped the dead net.InterfaceByName check. Adds two GatewayConfig.Validate cases (unparseable address, IPv4 address) covering the new validation. Fixes #360 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Summary
The gateway engine and reconcilers from the previous PRs need a process to run in. This adds galactic-gateway as its own binary, separate from galactic-router's tenant BGP process, so a crash in the XDP-holding gateway engine no longer takes BGP down with it and vice versa. Tenant BGP keeps running in galactic-router, co-located on the same gateway node. Fourth branch in the edge-gateway stack; builds on the controllers and webhook PR.
Test plan
task test:unit)task buildproducesbin/galactic-gatewayandtask lintis cleanRelated to #17