feat(seed): add seed node mode to SeiNode - #499
Merged
Conversation
A seed is a peer-discovery server: seid runs only the P2P transport and the PEX reactor, binding no RPC, gRPC, REST or EVM listener. It joins no consensus and stores no chain state. SeedSpec carries one field, a required NodeKey. Unlike the validator's optional key this is mandatory because a seed's NodeID is published -- operators dial NodeID@host:port and the handshake verifies the pinned value, so a regenerated key silently breaks every client carrying the old one. It is declared on SeedSpec rather than hoisted to SeiNodeSpec because ValidatorSpec's key-pairing CEL rules could no longer reach a top-level field; SeiNodeSpec.NodeKeySecret() resolves the split so downstream callers stay mode-blind. seedPlanner mirrors the archive planner. A nil snapshot source selects the genesis progression and, via needsStateSyncWitnesses, omits configure-state-sync; validate-node-key pre-flights the identity Secret before the StatefulSet is applied. The pod needed shaping. Readiness gated on seid's /lag_status and the cosmos-exporter container blocked on gRPC, so on a seed the pod would never reach Ready and the exporter would CrashLoopBackOff. servesSeidRPC gates both; ValidateSeedProbes fails the render closed if a seid probe ever targets RPC or gRPC again. GOMEMLIMIT is set from the resolved memory limit -- with request==limit the Go GC, blind to the cgroup, turns an allocation burst into an OOMKill rather than a throttle. scheduling.nodepoolSeed has no fallback: the default pool is RPC-class, so a 2Gi seed landing there costs an order of magnitude more than a seed is worth and karpenter.sh/do-not-disrupt then pins that node against consolidation. GenerateStatefulSet rejects a seed without it, which keeps the requirement off Config.Validate so clusters running no seeds still boot on an app-config file predating the key. storage.sizeSeed stays genuinely optional -- 20Gi is a correct value, not a placeholder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bump is what makes the controller's own sei-config constants match the released library; the sidecar carries the seed config defaults. It also lets ServicePorts compare against seiconfig.PortNameGRPC instead of a bare literal, which the constant was added for. Raise the seed memory profile from 2Gi to 4Gi. RFC 006 Appendix A gives "2 GB (4 GB for a primary seed)" and 1000 peers; the profile took the floor of the memory range while sei-config configures the top of the peer range. 1000 connections sets maxInbound, and with it the concurrent pre-auth handshake count, each reading up to a 1 MiB peer-declared size no config key bounds -- a burst that dwarfs live connection state and lands on the full cosmos app seid builds before dispatching on mode. 4Gi is the same t4g.medium class the RFC budgets. GOMEMLIMIT follows automatically: it derives from the resolved limit. ServicePorts keeps rendering from NodePorts() rather than NodePortsForMode(): the latter reports validator as p2p+metrics only, while applyValidatorOverrides enables gRPC for the price feeder and cosmos-exporter, so adopting it would strip a port validators serve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…strap invariant Review findings from the seed-mode cross-review. A StatefulSet render failure returned before the status flush, so a seed created before scheduling.nodepoolSeed exists left no phase and no condition -- the operator saw a stuck SeiNode with nothing explaining it, and the reason only in controller logs. Infra config loads once at startup, so applying a seed before restarting the controller reaches this on ordinary operator sequencing. Emit an Event so the reason reaches kubectl describe. Correct the sizing documentation. Four comments still described the seed as 2Gi after the profile moved to 4Gi, and the profile claimed a t4g.medium class that a 4Gi request==limit cannot schedule onto once kube/system reservation is taken -- the pool provisions the next size up. The rationale also cited connection counts and buffer sizes that live in sei-config and sei-tendermint, where this repo cannot track them. GOMEMLIMIT's doc oversold it: it governs churn, not live memory, and does not see non-heap RSS at all -- so absolute headroom is what absorbs the handshake burst, not the soft limit. The stated reason for scoping it to seed was also wrong; GC slack is proportional to the live set, not absolute, so the honest reason is caution about retuning prod-proven modes. Multiply before dividing, and skip the env entirely rather than emit a degenerate soft limit for a pathologically small override. ValidateSeedProbes claimed a failing readiness probe drops the pod from Service endpoints; PublishNotReadyAddresses means it does not. What it actually costs is the Ready condition and anything gating on it. A seed stays out of the bootstrap-Job path only because its SnapshotSource is nil -- bootstrapNodeMode has no seed arm and would render it as a full node with RPC probes and no ValidateSeedProbes call, with nothing failing to compile. TestSeedNeverBootstraps pins that invariant instead of a comment guarding it. NodeKeySource and SecretNodeKeySource are reused by SeedSpec but their docs described a validator's key as not slashing-relevant and low-stakes. Those land verbatim in kubectl explain seinode.spec.seed.nodeKey, where a published, effectively permanent NodeID is the opposite of low-stakes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consistency with the pools beside it: nodepoolArchive and nodepoolValidator are already required whether or not a cluster runs those modes. Treating the seed pool as optional was the exception, and it pushed the only enforcement into the render path, where a missing key surfaced as a stuck SeiNode rather than a startup error naming the field. Validate now rejects a config without it, so a cluster states its seed pool the same way it states the others. The render check stays as defence-in-depth alongside the images.kubeRBACProxy check it mirrors -- NodepoolForMode gives a seed no fallback, so an empty value there would silently schedule it onto the RPC-class default pool. Deploy ordering: an app-config file predating this key now fails the controller at startup, so platform adds scheduling.nodepoolSeed to each cell before the image rolls. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Back to 2Gi. 4Gi was sizing around seid's unbounded pre-auth handshake read, and buying that off here cost two of the RFC's own numbers: a 4Gi request==limit cannot schedule on a 4 GiB node once kube/system reservation is taken, so the pool provisions 8 GiB and three seeds land near $190/month against a $50-150 line. The exposure is real and now tracked as PLT-850 -- bound the handshake read to roughly the largest legitimate message. The burst is live memory, so no soft limit reclaims it and only absolute headroom would absorb it, which no figure in the RFC provides. Accepting the risk against a high-priority upstream fix keeps all four Appendix A figures consistent; when PLT-850 lands this footprint carries real headroom. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR SummaryMedium Risk Overview Controller behavior for seeds: Also bumps Reviewed by Cursor Bugbot for commit c37f7cf. Bugbot is set up for automated code reviews on this repo. Configure here. |
gofmt realigned two fields in testhelpers_test.go, which was enough for --new-from-patch to attribute the pre-existing goconst findings on those lines to this branch. Use the constants instead of silencing it: the const block already documented that defaultTestChainID has to match what these fixtures hardcode and that nothing enforced it, so this closes that gap and adds the image alongside. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Jul 27, 2026
bdchatham
added a commit
to sei-protocol/seictl
that referenced
this pull request
Jul 27, 2026
Patch bump `version.json` v0.0.68 → v0.0.69 to cut a release. On merge, the Releaser workflow tags and publishes v0.0.69, and Containerize pushes `ghcr.io/sei-protocol/seictl:v0.0.69` for amd64/arm64. ## Includes since v0.0.68 - **#240** — sei-config v0.0.23 → v0.0.24, which fixes seed mode's config defaults. ## Why this cut matters more than a usual dependency bump **The sidecar resolves `ConfigIntent`, so the sidecar's sei-config version — not the controller's — is what a node actually boots on.** Seed support in [sei-k8s-controller#499](sei-protocol/sei-k8s-controller#499) / [#500](sei-protocol/sei-k8s-controller#500) is merged and released, but it stays inert until this image ships. The failure mode if it doesn't is quiet. On sei-config v0.0.23, seed mode inherits the base default of a **loopback** P2P listen address, so a seed binds `127.0.0.1:26656` and is reachable by nobody — while booting clean, passing its TCP readiness probe against localhost, and reporting healthy. A seed that no peer can dial looks identical to a working one from inside the cluster. v0.0.24 also disables the listeners a seed has no business running (REST, gRPC, gRPC-web, EVM HTTP+WS), sets the tx indexer to `null`, turns off StateStore with pruning everything, and adds a validation guard rejecting seed mode with pex disabled — a seed exists to exchange peer addresses, and seid will not start without it. ## Sidecar code changes **None.** The bump carries only `go.mod`/`go.sum` plus two tests (`sidecar/tasks/config_apply_test.go`) asserting a seed resolves to a reachable listen address and that a pex-disabled seed is rejected. That the sidecar needed no changes to support an entirely new node mode is the mode-config layering working as designed: the controller emits `{Mode, Overrides}`, the sidecar calls `ResolveIntent`, and sei-config owns every per-mode default. ## Verification `go build ./...` clean; `go test ./...` green across all 16 packages. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a
seedmode toSeiNode— a peer-discovery node running only the P2P transport and the PEX reactor. Last repo in the RFC 006 chain (PLT-813 / PLT-815); sei-config v0.0.24 and the seictl bump are already merged.What a seed is
seedNodeImplstarts pprof, the p2p router and PEX, and nothing else. No RPC, gRPC, REST or EVM listener; no consensus, blocksync, mempool or statesync reactor. It loadsgenesis.jsonand opens block/state DBs it never writes. That shapes everything below.The CRD
SeedSpeccarries one field: a requiredNodeKey. A seed's NodeID is published — operators dialNodeID@host:portand the handshake verifies the pinned value — so a regenerated key silently breaks every client holding the old one. Left toseid initit regenerates onto the data volume; sourced from a Secret it survives pod recreation and PVC loss.It lives on
SeedSpecrather than being hoisted toSeiNodeSpecbecauseValidatorSpec's key-pairing CEL rules cannot reach a top-level field, and rewriting slashing-adjacent CEL to ship a seed feature is the wrong trade.SeiNodeSpec.NodeKeySecret()resolves the split so downstream callers stay mode-blind.Backward-compatible:
seedis new, so no existing object can carry it, and widening the exactly-one rule still evaluates true for every existing single-mode object. Exercised in envtest against a real apiserver.The pod
Readiness gated on seid's
/lag_statusand cosmos-exporter blocked on gRPC — on a seed the pod would never reach Ready and the exporter would CrashLoopBackOff. A singleservesSeidRPCpredicate gates both;ValidateSeedProbesfails the render closed if a seid probe ever targets RPC/gRPC again.Sizing, and a known exposure
2Gi per RFC 006 Appendix A, which also keeps the seed on the t-class instances and inside the cost line that RFC budgets.
Accepted risk: seid's pre-auth handshake read (
handshake.go:40) allocates a peer-declared size up to a hardcoded 1 MiB beforeNodeAuth.Verify, andMaxConcurrentAcceptsscales it withmax_connections— ~980 MB of unauthenticated concurrent allocation at the seed's 1000. That memory is live, soGOMEMLIMITcannot reclaim it and only absolute headroom would absorb it, which no RFC figure provides. Sizing around it would break the instance class and the budget, so it is tracked upstream as PLT-850 (Platform, High) rather than bought off here. When that lands, 2Gi carries real headroom.GOMEMLIMITis still set, derived from the resolved limit so aresources.seedoverride stays consistent — it governs churn, which is the part a soft limit can actually help.scheduling.nodepoolSeedis required at startup, matchingnodepoolArchiveandnodepoolValidatorwhich are already required whether or not a cluster runs those modes. An app-config file predating this key fails the controller at startup, so platform must add it to each cell before this image rolls. A seed must not inherit the RPC-class default pool, soNodepoolForModegives it no fallback.Review
Three blinded xreview rounds across sei-config and this repo, six lenses. Findings that changed the code here: a render failure returned before the status flush, leaving a stuck SeiNode with no phase, condition or event (now emits
StatefulSetRenderFailed);ValidateSeedProbesclaimed a failing readiness drops the pod from Service endpoints, whichPublishNotReadyAddressescontradicts; the shared node-key type docs described the key as low-stakes and land verbatim inkubectl explain seinode.spec.seed.nodeKey, where a published permanent NodeID is the opposite; andTestSeedNeverBootstrapsnow pins the invariant that was the only thing keeping a seed out of the bootstrap-Job path —bootstrapNodeModehas no seed arm and would render one as a full node with nothing failing to compile.Ledger:
designs/seed-node-mode/xreview/seed-node-mode.mdin bdchatham-designs.Not included
External reachability —
spec.externalAddress, a TCP/26656 load balancer, DNS, and an ingress allowance are GitOps-owned since PLT-451. Documented onSeedSpec, deliberately not CEL-enforced: an in-cluster-only seed on an ephemeral chain has a routable pod IP and needs none. Topology spread across the three seeds is also unaddressed — nothing stops Karpenter bin-packing all three bootstrap anchors onto one node.Test
Unit and full envtest suites green. New: seed planner progression asserting the negatives (no snapshot-restore, no configure-state-sync, no signing-key validate), pod shape, probe guard,
GOMEMLIMIT, nodepool fail-closed, storage sizing, and five admission tests against a real apiserver. No new lint findings in any touched file.🤖 Generated with Claude Code