feat(install-dynamic-plugins): add @red-hat-developer-hub/cli-module-install-dynamic-plugins#3246
Open
gustavolira wants to merge 8 commits into
Open
Conversation
Migrates scripts/install-dynamic-plugins/ from redhat-developer/rhdh#4574 into this repo as @red-hat-developer-hub/install-dynamic-plugins so it can be published to npm and consumed by the RHDH init-container without curl-by-SHA. Runtime contract (CLI args, env vars, plugin-hash format, on-disk layout, tar/OCI security guards) preserved verbatim. Build remains a single self-contained .cjs via esbuild. Tests migrated from vitest to jest to align with the repo's backstage-cli pipeline (14 suites / 166 tests pass). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
This pull request adds a new top-level directory under |
|
Important This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior. Changed Packages
|
Code Review by Qodo
1.
|
Review Summary by QodoAdd @red-hat-developer-hub/install-dynamic-plugins workspace with TypeScript/Node.js port
WalkthroughsDescription• Imports scripts/install-dynamic-plugins/ from redhat-developer/rhdh as a new workspace package @red-hat-developer-hub/install-dynamic-plugins for npm publication • Implements TypeScript/Node.js port of the original Python init-container CLI with byte-compatible behavior • Core functionality: materializes plugins on disk from dynamic-plugins.yaml config, supporting OCI (via skopeo) and NPM sources • Preserves runtime contract unchanged — same CLI surface, environment variables, and plugin hash values for in-place upgrades • Implements comprehensive security guards: path-traversal rejection, zip-bomb detection, link-target containment, and allowed-type whitelisting in tar-extract.ts and catalog-index.ts • Migrated test suite from vitest to jest with 14 test suites and 166 passing tests • Configured single bundled .cjs via esbuild for init-container image compatibility • Nested workspace layout (workspaces/install-dynamic-plugins/packages/install-dynamic-plugins/) matches monorepo structure and release pipeline • Implements concurrency control with semaphore-based worker management for OCI and NPM installations • Includes OCI image caching, manifest handling, pull policy support (IfNotPresent/Always), and registry fallback logic • Provides SRI integrity verification for NPM packages with streaming hash computation • Implements lock file management with signal cleanup to prevent concurrent installs Diagramflowchart LR
A["Python init-container<br/>install-dynamic-plugins.py"] -->|"Port to TypeScript/Node.js"| B["@red-hat-developer-hub/<br/>install-dynamic-plugins"]
B --> C["CLI orchestration<br/>index.ts"]
C --> D["Config merging<br/>merger.ts"]
C --> E["OCI installation<br/>installer-oci.ts"]
C --> F["NPM installation<br/>installer-npm.ts"]
E --> G["Image caching<br/>image-cache.ts"]
E --> H["Skopeo wrapper<br/>skopeo.ts"]
F --> I["Integrity verification<br/>integrity.ts"]
E --> J["Secure extraction<br/>tar-extract.ts"]
F --> J
C --> K["Catalog index<br/>catalog-index.ts"]
K --> H
C --> L["Lock management<br/>lock-file.ts"]
C --> M["Finalization<br/>finalize-install.ts"]
B -->|"Published to npm"| N["RHDH init-container<br/>consumes via npm install"]
File Changes1. workspaces/install-dynamic-plugins/packages/install-dynamic-plugins/src/index.ts
|
- Add bin/install-dynamic-plugins shim and have package.json bin point at it (matches the convention used by extensions-cli, translations-cli, and rhdh-repo-tools). Split src/cli.ts as the esbuild entry so the bundle no longer needs the require.main guard or a shebang banner. - Stop committing dist/install-dynamic-plugins.cjs; the release pipeline rebuilds via the customBuild path, and a new prepack script makes yarn npm publish self-healing for local runs. - Drop the .js suffix from relative imports across src/ so the package matches the rest of the repo and the jest moduleNameMapper workaround is no longer needed. - Consolidate the tsconfigs: the inner package extends the workspace tsconfig and only declares what differs. - Add why-it's-intentional comments to the two eslint-disable lines (PullPolicy const+type pair, tar.x filter inside a sequential loop). - README now leads with the npm/npx usage path; the RHDH init-container section is below. tar/yaml stay in dependencies (not devDependencies as the review suggested) — @backstage/no-undeclared-imports flagged the source imports, and the repo convention treats bundling as opaque. 166/166 tests pass, tsc/lint/prettier clean, bin shim and bundle both exit 0 on the empty-config smoke run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CI step "check api reports and generate API reference" runs
`backstage-repo-tools api-reports --ci` before the build, and that tool
requires the bin file to introspect the CLI. The previous shim did a
plain `require('../dist/install-dynamic-plugins.cjs')`, which failed
under CI because dist/ is no longer committed and the build hasn't run
yet.
- Switch the bin shim to the local-vs-installed pattern used by every
other CLI in the repo (extensions-cli, translations-cli,
rhdh-repo-tools): when `src/` exists (monorepo), load TS directly via
`@backstage/cli/config/nodeTransform`; otherwise require the built
bundle (npm-installed scenario).
- Add `--help` / `-h` handling to main() so the api-reports tool can
introspect the CLI usage without creating a stray `--help/` directory.
- Commit the generated `cli-report.md`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3246 +/- ##
==========================================
+ Coverage 53.23% 53.28% +0.04%
==========================================
Files 2413 2435 +22
Lines 86358 87615 +1257
Branches 23897 24195 +298
==========================================
+ Hits 45975 46683 +708
- Misses 40049 40598 +549
Partials 334 334
*This pull request uses carry forward flags. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
- Use String.raw for strings containing backslash literals so the source
reads with one '\' instead of '\\\\' (catalog-index.ts log message,
extra-catalog-index.test.ts subdirectory fixtures, skopeo.test.ts shell
escape).
- Switch the OCI regex builder to a joined string array — eliminates the
nested template literals SonarCloud was flagging on oci-key.ts (and
reads much better).
- Object.prototype.hasOwnProperty.call -> Object.hasOwn in
merger.test.ts (ES2022, available since Node 16.9).
- String#replace(/'/g, ...) -> String#replaceAll("'", ...) in
skopeo.test.ts (ES2021).
- Hoist test helpers (stageLayer, fakeImageCache) out of their describe
blocks so they aren't re-defined on every test.
- Drop the redundant parseMaxEntrySize(undefined) call in types.test.ts —
the parameter already defaults to process.env.MAX_ENTRY_SIZE.
166/166 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches the hand-rolled `process.argv` + USAGE-string handling in main() to `cleye` — the same parser every `@backstage/cli-module-*` package uses (already in our transitive deps). Aligns with the Backstage CLI convention requested during PR review. Existing surface preserved: - positional `<dynamic-plugins-root>` (required, exit 1 if absent) - `--help` / `-h` prints usage and exits 0 - normal run still exits with the installer's status code Bundle grew from 226 kB -> 267 kB (cleye + type-flag minified). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
…keeping the bundled bin Aligns with the @backstage/cli-module-* convention without losing the self-contained bundled artifact: - Rename to @red-hat-developer-hub/cli-module-install-dynamic-plugins, backstage.role: cli-module. - src/installer.ts holds the install pipeline and main() (formerly src/index.ts). - New src/index.ts default-exports `createCliModule(...)` registering an `install` command whose loader is src/command.ts. Exposes the package through backstage-cli discovery — `backstage-cli install <dir>` works when the package is a dependency. - src/cli.ts (the esbuild entry) keeps invoking installer.main() directly, so the bundled .cjs stays self-contained: no @backstage/cli-node and no keytar gymnastics in the bin path. - Build is dual now — `backstage-cli package build && node esbuild.config.mjs`. backstage-cli emits dist/index.cjs.js (the cli-module export) and the unbundled supporting modules; esbuild emits dist/install-dynamic-plugins.cjs (the standalone bin). Both are published. - bin shim's installed branch now requires the bundled .cjs explicitly rather than going through `main` — that keeps direct/npx/init-container invocations at ~60 ms cold start instead of paying the cli-module dispatch cost. - main() now takes optional `args` and `programName` so the cli-module loader can pass the command's argv slice and have `--help` print the real invocation (`install-dynamic-plugins install …`). - @backstage/cli-node added as a runtime dependency. It is only loaded by the cli-module discovery path; the bundled bin never imports it. 166/166 tests pass; tsc/lint/prettier/api-reports clean. Bundle size unchanged at ~267 KB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gustavolira
changed the title
…nbundled cli-module Replaces the dual build (esbuild bundle + backstage-cli) with plain `backstage-cli package build` — the standard Backstage cli-module pattern. The bundle and the keytar gymnastics only existed to satisfy RHDH's init-container `COPY` of a single self-contained `.cjs`; that consumption model is moving to `npm install` (redhat-developer/rhdh#4908), so the single-file requirement is going away. What's left in the package: - `src/installer.ts`: install pipeline + `main(args, programName)`. - `src/index.ts`: `createCliModule(...)` default export, registers the `install` command. Discovered by `backstage-cli` when this package is a dependency of a host project. - `src/command.ts`: thin loader that calls `installer.main`. - `bin/install-dynamic-plugins`: fast-path shim that loads `dist/installer.cjs.js` directly and runs `main(process.argv.slice(2))`, bypassing `@backstage/cli-node`'s `runCliModule` dispatch — saves ~80 ms of cold start for direct/`npx`/init-container invocations. The cli-module discovery path still goes through `runCliModule` and pays the dispatch cost where it belongs. Removed: - `esbuild.config.mjs` (the custom bundle config) - `src/cli.ts` (the esbuild entry) - `dist/install-dynamic-plugins.cjs` from `files` (no longer produced) - `esbuild` devDependency 166/166 tests pass; tsc/lint/prettier/api-reports clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…undle The README and the createCliModule docstring still referenced the self-contained bundle that the package no longer ships: - `npm run build` description and committed-bundle CI-check section. - `node install-dynamic-plugins.cjs "$1"` wrapper line in "How RHDH consumes it" — replaced with a pointer to redhat-developer/rhdh#4908. - `src/index.ts` describing the bin path as the bundle. - Source layout was missing `index.ts` / `command.ts` / `installer.ts`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Imports
scripts/install-dynamic-plugins/from redhat-developer/rhdh#4574 into this repo and packages it as a Backstage CLI module —@red-hat-developer-hub/cli-module-install-dynamic-plugins— so the overlay repo (and any other RHDH consumer) cannpm installit instead of vendoring the script.Builds with the standard
backstage-cli package buildand follows the@backstage/cli-module-*convention. No custom esbuild config, no keytar stub.Architecture
src/installer.ts— the install pipeline +main(args, programName). Single source of truth for the install logic.src/index.ts— default-exportscreateCliModule(...)registering theinstallcommand. Hosts that load this package viabackstage-cliget the command auto-discovered.src/command.ts— loader for theinstallcommand; callsinstaller.main.bin/install-dynamic-plugins— fast-path shim that loadsdist/installer.cjs.jsand invokesmain(process.argv.slice(2))directly. Bypasses@backstage/cli-node'srunCliModuledispatch for direct/npxinvocations — saves ~80 ms cold start (commander, zod, @manypkg/get-packages, @yarnpkg/lockfile, proper-lockfile, fs-extra, …). Thebackstage-clidiscovery path still goes throughrunCliModule.What's preserved verbatim
MAX_ENTRY_SIZE,SKIP_INTEGRITY_CHECK,CATALOG_INDEX_IMAGE,EXTRA_CATALOG_INDEX_IMAGES,DYNAMIC_PLUGINS_WORKERS,DYNAMIC_PLUGINS_NPM_WORKERS,DYNAMIC_PLUGINS_LOCK_TIMEOUT_MS,CATALOG_ENTITIES_EXTRACT_DIR).plugin-hash.ts: byte-compatible with the previous Python implementation — existing RHDH installs upgrade in place.tar-extract.tsandcatalog-index.tsuntouched.What changed during the move
@backstage/cli-module-*uses internally).backstage-cli package test. Mostly mechanical from the original vitest setup (dropvi.spyOn→jest.spyOn, co-locate tests intosrc/). 14 suites / 166 tests pass.@spotify/prettier-config.Consumption story
This package is built to be consumed two ways, both running the same install pipeline:
npx:npx @red-hat-developer-hub/cli-module-install-dynamic-plugins ./dynamic-plugins-root. Runs the fast-path bin.backstage-clihost: when listed as a dependency of a project that usesbackstage-cli, the command is registered automatically.backstage-cli install ./dynamic-plugins-rootworks out of the box.The follow-up RHDH PR (redhat-developer/rhdh#4908) switches RHDH's init-container from
COPY scripts/install-dynamic-plugins/dist/install-dynamic-plugins.cjsto consuming this package throughdynamic-plugins/package.json(yarn). After both PRs land,scripts/install-dynamic-plugins/can be deleted from RHDH in a subsequent cleanup.Test plan
yarn tsc/lint/prettier:check/build:api-reports:only --cicleanbin/install-dynamic-plugins ./tmp-empty-direxits 0 on an empty configrequire(dist/index.cjs.js).default.\$\$type === '@backstage/CliModule'🤖 Generated with Claude Code