Skip to content

Document preflight checks for Stacklok Enterprise Platform - #1068

Draft
glageju wants to merge 15 commits into
mainfrom
gautam/preflight-documentation
Draft

Document preflight checks for Stacklok Enterprise Platform#1068
glageju wants to merge 15 commits into
mainfrom
gautam/preflight-documentation

Conversation

@glageju

@glageju glageju commented Jul 29, 2026

Copy link
Copy Markdown

!! HOLD until upstream implementation is released: stacklok/stacklok-enterprise-platform#2513

Description

Documents the preflight checks feature for the Stacklok Enterprise Platform
Helm chart: the shipped Preflight custom resource, the advisory
kubectl preflight workflow on the Helm CLI install path, and the opt-in
preflight.enforce=true in-cluster blocking mode. This lets customers catch
install-time misconfigurations, an unsupported Kubernetes version, tight
node capacity, an unreachable OIDC issuer, a missing signing-key Secret,
before helm install fails partway through a release.

docs/platform/enterprise-platform/deployment.mdx

Adds a "Run preflight checks" section between "What the chart includes" and
"Deploy with Helm", covering:

  • Registry authentication and a real values.yaml are prerequisites for
    running the check, since helm template against an oci:// reference is
    itself a registry pull; the section points readers to the relevant
    numbered steps below
  • Installing the kubectl-preflight CLI plugin with krew as the primary
    path (any recent release works), with a pinned direct-download fallback
    for air-gapped or krew-less environments
  • Running the spec against real values so opt-in analyzers (OIDC,
    signing-key, registry database) actually render
  • A table of every check, when it renders, and its outcome
  • A credential-handling warning for supplying the registry database URI as
    a one-off --set flag alongside the real values file, using placeholder
    values rather than a literal-looking connection string
  • The advisory-only nature of the Helm CLI path
  • The optional preflight.enforce=true in-cluster blocking mode: its
    prerequisites, how to diagnose a blocked install, and how to bypass a
    known false positive
  • The two gaps preflight can't check (installer RBAC rights, CRD collisions
    with a prior install) and how to verify them manually

docs/platform/enterprise-platform/airgap-install.mdx

Adds a "Preflight, install, and verify" step (renamed from "Install and
verify") with two subsections in order: "Run preflight checks against the
mirrored chart" runs first, templating the local chart archive from the
mirror step instead of an oci:// reference, followed by "Install and
verify" (the prior content, unchanged). Preflighting runs before install so
it can catch problems ahead of time.

Type of change

  • New documentation

Related issues/PRs

Closes #1067

Upstream implementation: stacklok/stacklok-enterprise-platform#2513

Screenshots

Not applicable, no navigation or formatting changes.

Submitter checklist

Content and formatting

  • I have reviewed the content for technical accuracy
  • I have reviewed the content for spelling, grammar, and
    style

Reviewer checklist

Content

  • I have reviewed the content for technical accuracy
  • I have reviewed the content for spelling, grammar, and
    style

Vercel Deployment: CLICK HERE

glageju added 4 commits July 29, 2026 11:17
Implements documentation for issue #1067:
- Add "Run preflight checks" section covering the shipped Preflight CR
- Cover CLI plugin install, running against real values, and all checks
- Document advisory-by-default behavior and preflight.enforce=true
- Note the RBAC and CRD-collision gaps preflight cannot verify
Implements documentation for issue #1067:
- Add a subsection under "Step 5: Install and verify" showing that
  the default preflight spec runs entirely offline
- Show templating the locally transferred chart instead of oci://
- Note mirroring the CLI plugin release asset for the transfer workstation
Follow-up for issue #1067:
- Prefix idpConfig.issuer and signingConfig.existingSecret with the
  enterprise-manager subchart key, matching the rest of the page
- Rephrase the enforcement heading to plain English
Follow-up for issue #1067, iteration 2:
- Qualify preflightDatabaseUri with the toolhive-registry-server prefix
  in the checks table, matching the other conditional rows
- Distinguish the preflight.enforce values-file form from the --set flag
- Restructure two run-on sentences for clarity
Copilot AI review requested due to automatic review settings July 29, 2026 18:30
@vercel

vercel Bot commented Jul 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs-website Ready Ready Preview Jul 30, 2026 9:52pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds documentation for the Stacklok Enterprise Platform Helm chart preflight checks so readers can validate cluster prerequisites (and optionally enforce checks in-cluster) before running an install/upgrade that fails partway through.

Changes:

  • Added a new “Run preflight checks” section to the standard deployment guide, including plugin install, running against real values, a checks table, advisory vs. enforced modes, and manual verification gaps.
  • Added an air-gapped install subsection showing how to run preflight checks against a locally mirrored chart directory and mirror the CLI plugin asset if needed.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
docs/platform/enterprise-platform/deployment.mdx Adds comprehensive preflight-check guidance (install, run, interpret results, enforcement mode, and limitations).
docs/platform/enterprise-platform/airgap-install.mdx Adds air-gap specific preflight workflow using the mirrored chart directory and mirrored plugin assets.

Comment thread docs/platform/enterprise-platform/deployment.mdx Outdated
Comment thread docs/platform/enterprise-platform/deployment.mdx Outdated
Comment thread docs/platform/enterprise-platform/deployment.mdx
Comment thread docs/platform/enterprise-platform/airgap-install.mdx Outdated
Follow-up for issue #1067 / PR #1068:
- Fix the CLI download URL to match real Troubleshoot release assets
  (darwin ships a single universal binary, not per-arch ones)
- Reword two awkward sentences flagged in review
Match how the docs already treat third-party tool versions (helm,
cosign): a minimum/compatible version, not a pin tied to Stacklok's
internal CI. krew becomes the primary install path; the pinned
release download moves to a fallback for air-gapped or krew-less
setups, without claiming to match a CI-validated version.
Match the <ALL_CAPS> placeholder convention used elsewhere on this
page instead of a literal-looking connection string.
- Explain <CHANNEL> and <VERSION> at first use, since the preflight
  section now runs before Step 1 where they were previously introduced
- Replace the vague <CHART> placeholder with the concrete oci:// reference
Preflight is meant to catch problems before helm install, but the
prior placement ran it after install already succeeded and pods were
verified Running, defeating the point. Move "Run preflight checks"
ahead of "Install and verify" within Step 5, and rename the step to
"Preflight, install, and verify" to reflect both subsections.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

docs/platform/enterprise-platform/deployment.mdx:130

  • This line encourages using whatever is "current"/"latest" on the Troubleshoot releases page. That conflicts with the pinned-version guidance described in the PR (pick a specific version that Stacklok validates, rather than chasing latest). Consider rephrasing to avoid implying readers should update to the newest version opportunistically.
Check the [releases page](https://github.com/replicatedhq/troubleshoot/releases)
for the current version rather than assuming `v0.131.1` above is still latest.

docs/platform/enterprise-platform/deployment.mdx:93

  • This section says to run preflight "before you authenticate", but the workflow here uses helm template against an OCI chart URL, which requires authenticating to oci.stacklok.com (as described in Step 1 later in this file). This is inconsistent and can block readers from running preflight at the intended point in the install flow.
Before you authenticate and install, run the platform's **preflight** spec
against your target cluster. The umbrella chart ships a `Preflight` custom
resource as a labeled Secret, so rendering the chart and piping it to the
`kubectl-preflight` plugin analyzes real prerequisites automatically. It catches
misconfigurations before they surface later as failed pods or a stuck rollout: a

docs/platform/enterprise-platform/deployment.mdx:155

  • helm template doesn't typically "prompt" for registry credentials; it will fail unless you've already authenticated (e.g., helm registry login) or otherwise configured credentials. Since this page already documents helm registry login in Step 1, reword this to avoid implying an interactive prompt and to keep the flow consistent.
`helm template` prompts for the same registry credentials you use in
[Step 1](#1-authenticate-to-the-replicated-registry): your license email as the
username and your **License ID** as the password.

docs/platform/enterprise-platform/deployment.mdx:105

  • This suggests "Any recent release works" for the kubectl-preflight plugin, but the PR description/issue scope calls out recommending a pinned Troubleshoot release (the version Stacklok CI validates) and noting krew as an unpinned alternative that can drift. As written, readers will likely default to krew/latest and miss the pinned-version guidance.

This issue also appears on line 129 of the same file.

Any recent release works; the chart's `Preflight` spec doesn't depend on a
specific CLI version.

- Add the missing --values values.yaml flag to the registry database
  credential example; without it, copying that block standalone drops
  the real values file and silently skips every other analyzer
- Reword the "Step 4" cross-reference, since no numbered steps are
  visible from the preflight section itself
- deployment.mdx: explain where global.replicated.dockerconfigjson
  comes from (injected by the registry at pull time, not hand-set),
  verified against stacklok-enterprise-platform's appendix-b-helm-
  distribution.md and troubleshoot.md
- airgap-install.mdx: fix the preflight template command to reference
  the actual .tgz this flow produces, not an invented unpacked
  directory that never exists in this walkthrough
"Before you authenticate and install" was wrong: helm template
against an oci:// reference is itself a registry pull, so it needs
the same authentication as helm install. Verified against Replicated's
own docs and the upstream install-helm.md, which orders preflight
right after authentication, not before it. Fix the framing to point
the reader at Step 1 and Step 3 first, rather than claiming preflight
runs before authenticating.
@glageju
glageju marked this pull request as ready for review July 30, 2026 00:11

@danbarr danbarr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough work here, especially the credential-handling warnings and the enforcement-mode RBAC detail. A few things to work through before merge, and separately I don't think this should merge yet regardless of content, given the timing of the upstream feature.

Hold: merge timing

The preflight checks feature (stacklok-enterprise-platform#2513) hasn't shipped in an enterprise release yet. Documenting it now would put instructions in front of customers for a capability they can't use. Please hold this PR until the next enterprise release ships preflight checks.

Primary issues

1. The CLI install guidance conflicts with the upstream source, and the note framing buries a full script

deployment.mdx's "Install the CLI plugin" section leads with kubectl krew install preflight ("any recent release works"), with the pinned direct-download as a :::note fallback. The upstream stacklok-enterprise-platform repo (private, but the source of truth for this behavior) documents the opposite in docs/replicated/troubleshoot.md on current main:

Recommended: install the same pinned CLI version Stacklok validates the spec against. kubectl krew install preflight always resolves whatever the krew index currently points at, krew has no version-pin flag, so a krew install can silently drift ahead of the CLI version this spec was last tested with.

That repo also has a CI check that fails the build if the pinned version in its own docs drifts from build/replicated/Taskfile.yml's PREFLIGHT_CLI_VERSION, so this reads as a deliberate, tested recommendation, not a style choice. This page had the pinned-first ordering with the drift caveat until f2c25814 ("Lead with krew for preflight CLI install") intentionally swapped it, reasoning from how the docs treat Helm/cosign versions elsewhere. That's a reasonable analogy in isolation, but it now disagrees with the specific source this section is otherwise verified against. Can you confirm whether the swap was meant to intentionally diverge from the upstream recommendation, or reconcile the two?

Separately, and regardless of which method ends up primary: the pinned-install path is a multi-line script (curl, tar, sudo install, plus "check the releases page" guidance), and right now it's the entire content of a :::note. That's too much primary, standalone content for an admonition; whichever method is the fallback still deserves its own subsection rather than living inside a note.

2. The registry-database check's only worked example lives inside a credential warning

The :::warning[Registry database credential] block is the only place on the page that shows the actual command to run the registry-database check (--set registryServer.enabled=true --set 'toolhive-registry-server.preflightDatabaseUri=...'); the table just names the check, and the main "Run it against your real values" example above doesn't include those flags. A reader who wants to exercise that check has to find it inside a block titled around credential handling, not "how to run this." Please promote this to its own subsection (for example, "Check registry database reachability") with the command as the primary content, and keep the credential-handling caution as a smaller callout inside that section rather than the container for the whole example.

3. The CRD-collision check command misses AI Gateway CRDs

The "What preflight can't check" section's manual CRD-collision check says kubectl get crd | grep toolhive.stacklok.dev. The upstream repo's equivalent, in docs/replicated/troubleshoot.md on current main, is kubectl get crd | grep -E 'aigateway|toolhive'. Since aiGateway.enabled is an optional component with its own CRDs, a reader reinstalling over a prior AI-Gateway-enabled release who copies the narrower command would miss a real collision. Please widen the grep to match.

Secondary issues

  • The node-capacity table row says "warn if tight" without a threshold. The upstream source specifies concrete numbers (< 4 cores, < 6Gi memory). Not wrong as written, but stating the actual numbers would let readers self-diagnose without running the check. Optional.

What's solid

I verified this fairly extensively against the upstream stacklok-enterprise-platform source, and most of it checks out precisely: the checks table's gating conditions (including the OIDC row's enterpriseManager.enabled=true and idpConfig.issuer condition), the preflightDatabaseUri field name, the preflight.enforce RBAC scope and global.replicated.dockerconfigjson explanation, the bypass and log commands, and the "what preflight can't check" scoping (RBAC and CRD collision, correctly omitting the air-gap image-completeness item that only applies on that page) all match. Also confirmed every #N-heading cross-reference in both files resolves correctly, including the airgap-install.mdx step renumbering. All four earlier Copilot review threads look properly fixed and resolved, so I haven't re-raised those.

glageju and others added 2 commits July 30, 2026 14:33
Restore the pinned direct-download as the primary CLI install path, with
krew demoted from a note to its own subsection. This matches the upstream
recommendation in stacklok-enterprise-platform's
docs/replicated/troubleshoot.md, which is version-gated by CI against
PREFLIGHT_CLI_VERSION in build/replicated/Taskfile.yml.

Neither install method belonged in an admonition, so both are now real
subsections. The air-gapped page points at the pinned path explicitly,
since krew needs index access an air-gapped host doesn't have.

Also promote the registry-database check to its own subsection so its
command is primary content rather than living inside a credential
warning, widen the CRD-collision grep to cover the AI Gateway CRDs
(ai-gateway.stacklok.dev), and state the concrete node-capacity
thresholds (4 cores, 6Gi).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Correct three claims that contradicted the tooling or the upstream
source, and document two failure modes the sections omitted:

- helm template has no interactive credential prompt. It accepts
  --username/--password and otherwise reuses the helm registry login
  from Step 1, so a reader waited on a prompt that never appears.
- The dockerconfigjson guidance told readers to confirm the value in a
  "pulled values.yaml", but Step 3 has them hand-author that file and
  no step pulls one. Describe the license injection without pointing at
  an artifact the guide never produces.
- Note that the OIDC check probes the issuer from wherever the CLI runs,
  so a cluster-internal issuer can fail from a workstation while the
  Enterprise Manager pod reaches it fine. Without this, the "don't
  install while anything reports fail" rule blocks a valid install.
- Enabling enforcement on the air-gapped path needs more than a mirrored
  image: preflight.image.repository defaults to a Stacklok-hosted host
  an air-gapped cluster can't reach, and the hook Job renders its pull
  secret from global.replicated.dockerconfigjson, which is still
  required even though that path sets replicated.enabled: false.
- Enforcement also widens the preflightDatabaseUri exposure into the
  long-lived release Secret, not just the hook-deleted one.

Also link preflight checks from the platform deployment sequence, which
had no inbound reference to the new section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@glageju

glageju commented Jul 30, 2026

Copy link
Copy Markdown
Author

Thanks for verifying this against the upstream source rather than just reading the prose — the CLI ordering and CRD grep catches were both real.

Merge timing: you're right, and it's worse than "hasn't shipped yet." #2513 merged 2026-07-21, one day after the v0.9.0 tag, so git merge-base --is-ancestor confirms it's not in that release. Holding until the next enterprise release ships preflight.

1. CLI install ordering. Reconciled with upstream rather than defending the swap. Pinned direct-download is primary again, krew is now its own ### Alternative: install with krew subsection. Neither method sits in an admonition anymore, which covers your point that the fallback still deserves a real subsection either way.

To answer the direct question: the f2c2581 swap wasn't a deliberate divergence, it was reasoning from the Helm/cosign convention without re-checking this specific source. Your CI-gate point settles it — build/replicated/Taskfile.yml:865 fails the build on doc drift, so the pinned recommendation is tested, not stylistic.

One intentional divergence I kept: our script branches on preflight_darwin_all.tar.gz for macOS. Upstream's snippet uses preflight_${os}_${arch} unconditionally, which 404s on darwin — Troubleshoot ships one universal binary there. Ours is more correct than the source it's verified against; I'll open an upstream issue.

2. Registry-database example. Promoted to ### Check registry database reachability with the command as primary content. The credential caution is now a smaller :::warning inside that section instead of the container for the whole example.

3. CRD grep. Widened to grep -E 'aigateway|toolhive'. Confirmed it matters: the AI Gateway ships ai-gateway.stacklok.dev CRDs (aigateways, aipolicies, pricingcatalogs), none of which the old pattern catches. Left the kubectl get crd at line 564 alone — that's pre-existing verify-the-install code checking registration, not collisions.

Node capacity. Took the optional suggestion: now warn below 4 allocatable cores or 6Gi.

Three more things surfaced from reading it as a first-time user rather than against upstream:

  1. helm template doesn't prompt for credentials — it takes --username/--password or reuses the Step 1 helm registry login. The old text had readers waiting on a prompt that never appears.
  2. The dockerconfigjson bullet told readers to confirm the value in a "pulled values.yaml", but Step 3 has them hand-author that file and nothing in the guide pulls one.
  3. Air-gap enforcement needs more than a mirrored image: preflight.image.repository defaults to image-proxy.stacklok.com, and the hook Job renders its pull secret from global.replicated.dockerconfigjson — still required even though that path sets replicated.enabled: false. Mirroring alone still hangs the install.

Also added the OIDC workstation-vantage caveat, since a cluster-internal issuer fails from a laptop and the "don't install while anything reports fail" rule would have blocked a valid install.

Pushed in fd2d22f.

@glageju
glageju marked this pull request as draft July 30, 2026 21:49
@glageju
glageju requested a review from danbarr July 30, 2026 21:49

@danbarr danbarr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the review, LGTM now! Ready to merge once it releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Gap]: Document preflight checks for Stacklok Enterprise Platform

3 participants