Skip to content

feat(scorecard): score first in-window DORA deployments for lead time and CFR - #4732

Open
PatAKnight wants to merge 3 commits into
redhat-developer:mainfrom
PatAKnight:rhidp-16305
Open

PatAKnight wants to merge 3 commits into
redhat-developer:mainfrom
PatAKnight:rhidp-16305

Conversation

@PatAKnight

Copy link
Copy Markdown
Member

Hey, I just made a Pull Request!

Median lead time and change failure rate only paired successful production deployments inside the 30-day window, so the first in-window deploy was never fully scored.
This change loads the latest persisted successful production deployment immediately before windowFrom and uses it as:

  • Lead time: baseCommitSha for PRs into the first in-window deploy
  • CFR: start of the interval ending at the first in-window deploy, so incidents in that gap are counted

Collector sync stays on the 30-day window. If no prior deploy is still in the DB, behavior is unchanged. Incident sync is not expanded; incidents before windowFrom count only if already retained.

Also included the knip-reports and removed some of the unused dependencies across the plugins. I can drop them in the event that we a cleaner, more focused PR.

✔️ Checklist

  • A changeset describing the change and affected packages. (more info)
  • Added or Updated documentation
  • Tests for new functionality and regression tests for bug fixes
  • Screenshots attached (for UI changes)

… and CFR

Signed-off-by: Patrick Knight <pknight@redhat.com>
Signed-off-by: Patrick Knight <pknight@redhat.com>
@rhdh-gh-app

rhdh-gh-app Bot commented Sep 14, 2026

Copy link
Copy Markdown

Important

This PR includes changes that affect public-facing API. Please ensure you are adding/updating documentation for new features or behavior.

Changed Packages

Package Name Package Path Changeset Bump Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-catalog workspaces/scorecard/plugins/scorecard-backend-module-catalog patch v0.0.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dora workspaces/scorecard/plugins/scorecard-backend-module-dora minor v0.0.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-filecheck workspaces/scorecard/plugins/scorecard-backend-module-filecheck patch v1.0.2
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-github workspaces/scorecard/plugins/scorecard-backend-module-github patch v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-jira workspaces/scorecard/plugins/scorecard-backend-module-jira patch v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-openssf workspaces/scorecard/plugins/scorecard-backend-module-openssf patch v1.0.2
@red-hat-developer-hub/backstage-plugin-scorecard-node workspaces/scorecard/plugins/scorecard-node patch v4.2.0
@red-hat-developer-hub/backstage-plugin-scorecard workspaces/scorecard/plugins/scorecard patch v4.2.0

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 14, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 1:50 PM UTC · Ended 1:54 PM UTC

Commit: 01bc00a · View workflow run →

Signed-off-by: Patrick Knight <pknight@redhat.com>
@sonarqubecloud

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 14, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 1:56 PM UTC · Completed 2:12 PM UTC

Commit: 7b3d8a0 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $6.77

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.63%. Comparing base (450960f) to head (7b3d8a0).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4732   +/-   ##
=======================================
  Coverage   59.62%   59.63%           
=======================================
  Files        2627     2628    +1     
  Lines      104976   104996   +20     
  Branches    29553    29556    +3     
=======================================
+ Hits        62594    62614   +20     
  Misses      41819    41819           
  Partials      563      563           
Flag Coverage Δ *Carryforward flag
adoption-insights 84.77% <ø> (ø) Carriedforward from 450960f
ai-integrations 78.80% <ø> (ø) Carriedforward from 450960f
app-defaults 53.07% <ø> (ø) Carriedforward from 450960f
augment 46.67% <ø> (ø) Carriedforward from 450960f
boost 83.46% <ø> (ø) Carriedforward from 450960f
bulk-import 73.12% <ø> (ø) Carriedforward from 450960f
cost-management 13.35% <ø> (ø) Carriedforward from 450960f
dcm 73.47% <ø> (ø) Carriedforward from 450960f
e2e-adoption-insights 60.00% <ø> (ø) Carriedforward from 450960f
e2e-extensions 62.31% <ø> (ø) Carriedforward from 450960f
e2e-global-header 49.71% <ø> (ø) Carriedforward from 450960f
e2e-homepage 61.11% <ø> (ø) Carriedforward from 450960f
e2e-intelligent-assistant 46.09% <ø> (ø) Carriedforward from 450960f
e2e-orchestrator 49.52% <ø> (ø) Carriedforward from 450960f
e2e-orchestrator-plugin 49.51% <ø> (ø) Carriedforward from 450960f
e2e-quickstart 55.21% <ø> (ø) Carriedforward from 450960f
e2e-scorecard 50.05% <ø> (ø) Carriedforward from 450960f
e2e-theme 16.36% <ø> (ø) Carriedforward from 450960f
extensions 57.37% <ø> (ø) Carriedforward from 450960f
global-floating-action-button 71.18% <ø> (ø) Carriedforward from 450960f
global-header 67.88% <ø> (ø) Carriedforward from 450960f
homepage 48.39% <ø> (ø) Carriedforward from 450960f
install-dynamic-plugins 71.77% <ø> (ø) Carriedforward from 450960f
intelligent-assistant 77.26% <ø> (ø) Carriedforward from 450960f
konflux 91.98% <ø> (ø) Carriedforward from 450960f
lightspeed 69.02% <ø> (ø) Carriedforward from 450960f
mcp-integrations 84.46% <ø> (ø) Carriedforward from 450960f
orchestrator 72.02% <ø> (ø) Carriedforward from 450960f
quickstart 63.74% <ø> (ø) Carriedforward from 450960f
sandbox 79.56% <ø> (ø) Carriedforward from 450960f
scorecard 88.25% <100.00%> (+0.04%) ⬆️
theme 87.91% <ø> (ø) Carriedforward from 450960f
translations 5.12% <ø> (ø) Carriedforward from 450960f
x2a 13.91% <ø> (ø) Carriedforward from 450960f

*This pull request uses carry forward flags. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 450960f...7b3d8a0. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Low

  • [scope-creep] — The PR bundles two distinct concerns: (1) the DORA pre-window deployment scoring feature, and (2) a workspace-wide knip cleanup that removes unused dependencies across 7 package.json files, adds 14 knip-report.md files, and modifies yarn.lock. The author acknowledges this in the PR body and offers to split them.
    Remediation: Split the knip dependency cleanup into a separate PR for cleaner review and bisect history.

  • [edge-case] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/service/DoraDataService.ts:86readLatestProductionDeploymentBefore fetches at most 50 candidates and filters to the first production deployment in application code. If more than 50 non-production deployments exist between the last production deploy and the window boundary, the method silently returns undefined. The caller cannot distinguish "no pre-window production deployment exists" from "the limit was exhausted."
    Remediation: Consider logging a warning when all N candidates are non-production and N equals the limit, so operators can tune the constant for their deployment patterns.

  • [naming-convention] workspaces/scorecard/plugins/scorecard-backend-module-dora/src/database/DatabaseDoraDeployments.ts:41 — The new store method readCandidatesBefore follows a different naming pattern than the existing readByEntityCollectorAndWindow. The established convention names read methods after their filter criteria (readBy + filter description). The difference is intentional (the method returns candidates for further filtering per its JSDoc), but worth noting for consistency.


Labels: PR adds a new feature to DORA metric scoring in the scorecard workspace.

},
): Promise<DbDoraDeployment | undefined> {
const candidates = await this.deploymentsDb.readCandidatesBefore(
catalogEntityRef,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] edge-case

readLatestProductionDeploymentBefore fetches at most 50 candidates and filters to the first production deployment in application code. If more than 50 non-production deployments exist between the last production deploy and the window boundary, the method silently returns undefined. The caller cannot distinguish no pre-window production deployment exists from the limit was exhausted.

Suggested fix: Consider logging a warning when all N candidates are non-production and N equals the limit, so operators can tune the constant for their deployment patterns.

): Promise<DbDoraDeployment[]>;
/**
* Newest deployments with `created_at` strictly before `before`, for the
* entity and collector identity. Callers filter to production.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] naming-convention

The new store method readCandidatesBefore follows a different naming pattern than the existing readByEntityCollectorAndWindow. The established convention names read methods after their filter criteria (readBy + filter description). The difference is intentional (the method returns candidates for further filtering per its JSDoc), but worth noting for consistency.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge enhancement New feature or request labels Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request ready-for-merge All reviewers approved — ready to merge workspace/scorecard

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant