fix(scorecard): Handle custom thresholds from scorecard backend

[dzemanov]

Hey, I just made a Pull Request!

• Handles custom thresholds from scorecard backend.
• Simplifies scorecard module providers as they no longer need to handle custom thresholds.
• Fixes unable to configure filecheck, openssf and dependabot thresholds.
• Updates openssf default thresholds to follow order of thresholds of all other modules (to start with success).
• Validates default thresholds from providers.

Fixes

Fixes https://redhat.atlassian.net/browse/RHDHBUGS-3203

How to test

• Confirm you can configure custom thresholds for all scorecard plugins. (Focus on filecheck, openssf, dependabot that are now configurable).
  For example:

    dependabot:
      alerts_critical:
        thresholds:
          rules:
            - key: low
              expression: '<5'
              icon: scorecardSuccessStatusIcon
              color: '#B4D3B2'
            - key: medium
              expression: '5-20'
              icon: scorecardWarningStatusIcon
              color: '#FAD5A5'
            - key: high
              expression: '>20'
              icon: scorecardErrorStatusIcon
              color: '#FAA0A0'

    openssf:
      code_review:
        thresholds:
          rules:
            - key: great
              expression: '>8'
              icon: scorecardSuccessStatusIcon
              color: '#B4D3B2'
            - key: good
              expression: '4-8'
              icon: scorecardWarningStatusIcon
              color: '#FFEA00'
            - key: critical
              expression: '<4'
              icon: scorecardErrorStatusIcon
              color: '#7F00FF'
    filecheck:
      thresholds:
        rules:
          - key: present
            expression: '==true'
            icon: scorecardSuccessStatusIcon
            color: 'success.main'
          - key: absent
            expression: '==false'
            icon: scorecardErrorStatusIcon
            color: 'error.main'

• Confirm errors are correctly handled for invalid threshold annotations - show in Entity Scorecards and Aggregation Scorecards, for example with scorecard.io/jira.open_issues.thresholds.rules.success: '<<32' in jira card.
• Confirm errors becuase of invalid threshold configuration in app-config.yaml appear on startup.

✔️ Checklist

• A changeset describing the change and affected packages. (more info)
• Added or Updated documentation
• Tests for new functionality and regression tests for bug fixes
• Screenshots attached (for UI changes)

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

[rhdh-gh-app]

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dependabot	workspaces/scorecard/plugins/scorecard-backend-module-dependabot	patch	v0.2.11
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-filecheck	workspaces/scorecard/plugins/scorecard-backend-module-filecheck	patch	v0.1.8
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-github	workspaces/scorecard/plugins/scorecard-backend-module-github	patch	v2.7.7
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-jira	workspaces/scorecard/plugins/scorecard-backend-module-jira	patch	v2.7.7
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-openssf	workspaces/scorecard/plugins/scorecard-backend-module-openssf	patch	v0.2.11
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-sonarqube	workspaces/scorecard/plugins/scorecard-backend-module-sonarqube	patch	v0.1.6
@red-hat-developer-hub/backstage-plugin-scorecard-backend	workspaces/scorecard/plugins/scorecard-backend	patch	v2.7.7
@red-hat-developer-hub/backstage-plugin-scorecard-node	workspaces/scorecard/plugins/scorecard-node	patch	v2.7.7

[rhdh-qodo-merge]

Review Summary by Qodo

Centralize custom threshold handling in scorecard backend

🐞 Bug fix ✨ Enhancement

Walkthroughs

Description

• Centralize custom threshold handling in scorecard backend module
• Remove threshold parameter from individual metric provider constructors
• Implement ThresholdResolver to manage provider and entity-level thresholds
• Reorder OpenSSF default thresholds to start with success status
• Enable threshold configuration for filecheck, openssf, and dependabot providers

Diagram

flowchart LR
  A["Metric Providers"] -->|"getMetricThresholds"| B["Provider Defaults"]
  C["app-config.yaml"] -->|"scorecard.plugins.*.thresholds"| D["ThresholdResolver"]
  E["Entity Annotations"] -->|"scorecard.io/*.thresholds"| D
  D -->|"resolveProviderThresholds"| F["Configured Thresholds"]
  D -->|"resolveEntityThresholds"| G["Merged Thresholds"]
  F --> H["CatalogMetricService"]
  G --> H

Loading

File Changes

1. workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts ✨ Enhancement +65/-0

New threshold resolution service for centralized management

workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts

---

2. workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts 🧪 Tests +160/-0

Comprehensive tests for threshold resolution logic

workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts

---

3. workspaces/scorecard/plugins/scorecard-backend/src/utils/mergeEntityAndProviderThresholds.ts ✨ Enhancement +2/-1

Add baseThresholds parameter for flexible threshold merging

workspaces/scorecard/plugins/scorecard-backend/src/utils/mergeEntityAndProviderThresholds.ts

---

View more (30)

4. workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProvider.ts 🐞 Bug fix +1/-4

Remove threshold parameter from constructor

workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProvider.ts

---

5. workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProvider.test.ts 🧪 Tests +6/-15

Update tests to remove custom threshold passing

workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProvider.test.ts

---

6. workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProviderFactory.ts 🐞 Bug fix +2/-5

Remove threshold parameter from factory functions

workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProviderFactory.ts

---

7. workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProviderFactory.test.ts 🧪 Tests +2/-23

Remove tests for custom threshold passing

workspaces/scorecard/plugins/scorecard-backend-module-dependabot/src/metricProviders/DependabotMetricProviderFactory.test.ts

---

8. workspaces/scorecard/plugins/scorecard-backend-module-filecheck/src/metricProviders/FilecheckMetricProvider.ts 🐞 Bug fix +2/-8

Remove threshold parameter from constructor

workspaces/scorecard/plugins/scorecard-backend-module-filecheck/src/metricProviders/FilecheckMetricProvider.ts

---

9. workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.ts 🐞 Bug fix +4/-15

Remove threshold configuration from provider

workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.ts

---

10. workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.test.ts 🧪 Tests +1/-46

Remove custom threshold configuration tests

workspaces/scorecard/plugins/scorecard-backend-module-github/src/metricProviders/GithubOpenPRsProvider.test.ts

---

11. workspaces/scorecard/plugins/scorecard-backend-module-jira/src/metricProviders/JiraOpenIssuesProvider.ts 🐞 Bug fix +5/-21

Remove threshold parameter from constructor

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/metricProviders/JiraOpenIssuesProvider.ts

---

12. workspaces/scorecard/plugins/scorecard-backend-module-jira/src/metricProviders/JiraOpenIssuesProvider.test.ts 🧪 Tests +1/-36

Remove custom threshold configuration tests

workspaces/scorecard/plugins/scorecard-backend-module-jira/src/metricProviders/JiraOpenIssuesProvider.test.ts

---

13. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFConfig.ts 🐞 Bug fix +2/-2

Reorder default thresholds to start with success

workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFConfig.ts

---

14. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.ts 🐞 Bug fix +3/-10

Remove threshold parameter from constructor

workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.ts

---

15. workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.test.ts 🧪 Tests +12/-48

Update tests to remove threshold parameter passing

workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFMetricProvider.test.ts

---

16. workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeBooleanMetricProvider.ts 🐞 Bug fix +6/-13

Remove threshold configuration from provider

workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeBooleanMetricProvider.ts

---

17. workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeBooleanMetricProvider.test.ts 🧪 Tests +1/-28

Remove custom threshold configuration tests

workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeBooleanMetricProvider.test.ts

---

18. workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeNumberMetricProvider.ts 🐞 Bug fix +6/-13

Remove threshold configuration from provider

workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeNumberMetricProvider.ts

---

19. workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeNumberMetricProvider.test.ts 🧪 Tests +1/-29

Remove custom threshold configuration tests

workspaces/scorecard/plugins/scorecard-backend-module-sonarqube/src/metricProviders/SonarQubeNumberMetricProvider.test.ts

---

20. workspaces/scorecard/plugins/scorecard-backend/src/plugin.ts ✨ Enhancement +8/-0

Integrate ThresholdResolver into plugin initialization

workspaces/scorecard/plugins/scorecard-backend/src/plugin.ts

---

21. workspaces/scorecard/plugins/scorecard-backend/src/scheduler/index.ts ✨ Enhancement +4/-0

Add ThresholdResolver to scheduler options

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/index.ts

---

22. workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts ✨ Enhancement +10/-6

Use ThresholdResolver instead of merge utility

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.ts

---

23. workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts ✨ Enhancement +8/-2

Use ThresholdResolver for threshold resolution

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.ts

---

24. workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts ✨ Enhancement +5/-2

Use ThresholdResolver for provider thresholds

workspaces/scorecard/plugins/scorecard-backend/src/service/router.ts

---

25. workspaces/scorecard/plugins/scorecard-backend/docs/thresholds.md 📝 Documentation +8/-44

Update documentation for centralized threshold handling

workspaces/scorecard/plugins/scorecard-backend/docs/thresholds.md

---

26. workspaces/scorecard/plugins/scorecard-backend-module-filecheck/README.md 📝 Documentation +19/-0

Add example threshold configuration

workspaces/scorecard/plugins/scorecard-backend-module-filecheck/README.md

---

27. workspaces/scorecard/.changeset/tired-hoops-happen.md 📝 Documentation +15/-0

Add changeset for threshold centralization

workspaces/scorecard/.changeset/tired-hoops-happen.md

---

28. workspaces/scorecard/plugins/scorecard-backend/__fixtures__/mockEntityBuilder.ts Miscellaneous +5/-0

Add withAnnotations helper method

workspaces/scorecard/plugins/scorecard-backend/fixtures/mockEntityBuilder.ts

---

29. workspaces/scorecard/plugins/scorecard-backend/src/scheduler/index.test.ts Additional files +5/-0

...

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/index.test.ts

---

30. workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.test.ts Additional files +19/-23

...

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.test.ts

---

31. workspaces/scorecard/plugins/scorecard-backend/src/scheduler/types.ts Additional files +2/-0

...

workspaces/scorecard/plugins/scorecard-backend/src/scheduler/types.ts

---

32. workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.test.ts Additional files +15/-20

...

workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.test.ts

---

33. workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts Additional files +26/-3

...

workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts

---

[fullsend-ai-review]

Review

Findings

Medium

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts:732 — The test builds expected values by calling provider.getMetricThresholds() directly, but the production code in router.ts (lines 183, 305) was changed to use thresholdResolver.resolveProviderThresholds(provider). Since the test has no custom thresholds configured, both paths return the same value, but the test does not exercise the new threshold resolution code path through the router.
  Remediation: Update the test expectation to use thresholdResolver.resolveProviderThresholds(provider) to match the production code path.

• [pattern-inconsistency] workspaces/scorecard/plugins/scorecard-backend/src/scheduler/tasks/PullMetricsByProviderTask.test.ts:26 — The PR changes import type { Config } to import { Config }, dropping the type qualifier. The codebase convention is to use import type for type-only imports. Config is only used as a type annotation in this file.
  Remediation: Keep import type { Config } from '@backstage/config'.

• [pattern-inconsistency] workspaces/scorecard/plugins/scorecard-backend/src/service/router.test.ts:16,53-54 — The PR drops the type qualifier from imports of Config, DatabaseMetricValues, and DbAggregatedMetric. These are used only as type annotations in this test file. The codebase consistently uses import type for type-only imports.
  Remediation: Keep import { ConfigReader, type Config }, import type { DatabaseMetricValues }, and import type { DbAggregatedMetric }.

• [pattern-inconsistency] workspaces/scorecard/plugins/scorecard-backend/src/service/CatalogMetricService.test.ts:53 — The PR changes import type { DatabaseMetricValues } to a value import. The symbol is used only as a type for mocking; sibling test files maintain the type import.
  Remediation: Keep import type { DatabaseMetricValues } from '../database/DatabaseMetricValues'.

Low

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — No test verifies resolveEntityThresholds for a batch provider where entity annotations target a specific metric ID (e.g., scorecard.io/filecheck.readme.thresholds.rules.success). Since getProviderId() for batch providers returns the prefix (e.g., filecheck), entity annotations scoped to individual metrics within a batch provider may not match. This edge case should be tested or documented.

• [scope-creep] workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFConfig.ts:147 — The reordering of OPENSSF_THRESHOLDS rules (success first, then warning, then error) is a behavioral change since threshold evaluation is order-dependent (first-match wins). The expressions are mutually exclusive so the reorder is safe in practice, but this is a behavioral modification bundled into a structural refactoring PR.

• [missing-doc] workspaces/scorecard/plugins/scorecard-backend-module-dependabot/README.md — The PR adds custom threshold configurability for dependabot, but the dependabot README has no threshold configuration section. Other provider READMEs (github, jira, filecheck) document threshold configuration.

• [stale-doc] workspaces/scorecard/plugins/scorecard-backend-module-github/README.md:70 — The section labeled "Default thresholds" shows an app-config.yaml example that is actually the custom override format. After this PR, defaults are hardcoded in code and the config shown is the override path.

• [stale-doc] workspaces/scorecard/plugins/scorecard-backend-module-jira/README.md:190 — Same as above: the "Default thresholds" section shows the custom override format, not the actual defaults.

Info

• [test-integrity] Tests for custom threshold handling were removed from all provider modules (dependabot, github, jira, sonarqube, openssf). This is consistent with the architectural change — custom threshold handling is centralized in ThresholdResolver. The new ThresholdResolver.test.ts provides equivalent coverage. No test weakening or split-payload pattern detected.

• [architectural-coherence] The ThresholdResolver class cleanly centralizes the three-layer threshold resolution (provider defaults → app-config overrides → entity annotation overrides) into a single service, placed alongside ThresholdEvaluator in the threshold/ directory. Well-motivated design improvement.

Previous run

Review

Findings

Medium

• [missing-test] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — ThresholdResolver tests only exercise single-metric MockNumberProvider instances. No test covers batch providers (e.g., filecheck with getProviderId()='filecheck' but multiple metric IDs). The config path structure differs for batch providers (scorecard.plugins.filecheck.thresholds vs scorecard.plugins.github.number_metric.thresholds), and this path is untested.
  Remediation: Add a test using MockBatchBooleanProvider (available in __fixtures__/mockProviders.ts) that configures thresholds at the batch provider's providerId path and verifies resolveProviderThresholds returns the configured thresholds.

• [error-handling-idiom] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts — getThresholdsFromConfig (in scorecard-node) throws a generic Error on invalid config, and ThresholdResolver propagates it as-is during construction. Meanwhile, MetricProvidersRegistry.register() wraps the same validation failures in ThresholdConfigFormatError for structured error handling. This inconsistency means callers catching ThresholdConfigFormatError would miss config-based threshold errors from ThresholdResolver.
  Remediation: Wrap the error thrown during config threshold loading in ThresholdConfigFormatError for consistency with MetricProvidersRegistry.register().

• [stale-doc] workspaces/scorecard/plugins/scorecard-backend-module-openssf/README.md:47 — Line 47 states "Not configurable" for OpenSSF thresholds, but this PR explicitly makes them configurable via the centralized ThresholdResolver. The changeset confirms: "Custom thresholds for filecheck, openssf, and dependabot are now configurable."
  Remediation: Update the OpenSSF README thresholds section to indicate thresholds are now configurable via scorecard.plugins.<providerId>.thresholds in app-config.yaml, with a link to the thresholds docs.

Low

• [missing-test] workspaces/scorecard/plugins/scorecard-backend/src/utils/mergeEntityAndProviderThresholds.ts:53 — The new optional baseThresholds parameter is only tested indirectly through ThresholdResolver.test.ts. The existing unit tests in mergeEntityAndProviderThresholds.test.ts do not exercise the explicit baseThresholds override path.

• [naming-convention] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — Typo in test fixture: 'github.other_netric' should be 'github.other_metric'. Appears in two test cases. Does not affect test validity but reads as an oversight.

• [scope-creep] workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFConfig.ts — The OPENSSF_THRESHOLDS rules were silently reordered from [error, warning, success] to [success, warning, error]. Since the expressions use non-overlapping ranges (<2, 2-7, >7), this does not change evaluation behavior. However, the change is not mentioned in the PR description or changeset.

Info

• [test-integrity] Tests for custom threshold configuration at the provider level were removed from multiple providers (GitHub, Jira, SonarQube, Dependabot). This is appropriate — threshold customization has been centralized in ThresholdResolver, and equivalent validation coverage exists in ThresholdResolver.test.ts and MetricProvidersRegistry.test.ts. No test weakening detected.

Previous run (2)

Review

Findings

Medium

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — The ThresholdResolver test suite does not cover batch providers (e.g., FilecheckMetricProvider). Batch providers have a getProviderId() that differs from individual metric IDs (e.g., filecheck vs filecheck.readme). The resolver uses getProviderId() to build config paths and cache thresholds, but there are no tests verifying this works correctly for batch providers — particularly that resolveEntityThresholds correctly merges annotation-based overrides against the batch-level resolved thresholds.

Low

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — Several provider-level tests verifying invalid threshold expressions (e.g., >!100 in GithubOpenPRsProvider) were removed. The centralized ThresholdResolver test for startup validation uses a minimal mock ({ getOptional: jest.fn() }) rather than the real ConfigReader pipeline. The underlying validation functions are tested in scorecard-node, but an integration-style test through ConfigReader would increase confidence.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts — The resolver constructs config paths like scorecard.plugins.${providerId}.thresholds. For providers with dotted IDs (e.g., dependabot.alerts_critical), Backstage's Config interprets dots as nested key separators. This is consistent with the prior behavior and Backstage conventions, but worth noting for documentation.

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts:102 — Test fixture uses 'github.other_netric' (typo for 'github.other_metric'). Does not affect correctness but could cause confusion during debugging.

Info

• [sub-agent-failure] N/A — The intent-coherence, style-conventions, and docs-currency sub-agents did not return findings due to model unavailability on the deployment. These are sonnet-tier dimensions; findings from these dimensions may be absent from this review.

Previous run (3)

Review

Findings

Low

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts:37 — resolveProviderThresholds returns cached Map entries by reference. If a future caller mutates the returned ThresholdConfig, the cached entry would be corrupted. Currently safe because mergeEntityAndProviderThresholds spreads rules via [...providerThresholds.rules], but fragile if new callers are added without the same care.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts:37 — If a provider were dynamically registered after ThresholdResolver construction, its app-config thresholds would be silently ignored (falls back to provider defaults). This cannot occur in practice under Backstage's initialization lifecycle (modules finalize before plugin init), but is worth noting for documentation.

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — All ThresholdResolver tests use MockNumberProvider. No integration-level test covers boolean-type providers through ThresholdResolver, though the underlying functions have their own boolean tests.

• [test-inadequate] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts:147 — Typo in test fixture: github.other_netric should be github.other_metric. Does not affect test correctness but could cause confusion when debugging.

Info

• [behavioral-change] workspaces/scorecard/plugins/scorecard-backend-module-openssf/src/metricProviders/OpenSSFConfig.ts — OpenSSF default threshold rule order changed from [error, warning, success] to [success, warning, error]. Since the three expressions (<2, 2-7, >7) partition the number line without overlap, this reorder has no behavioral impact on threshold evaluation.

• [test-integrity] — No test weakening detected. The removed per-provider custom threshold tests are properly replaced by centralized ThresholdResolver tests covering the same precedence logic.

• [sub-agent-failure] N/A — The style-conventions sub-agent did not return findings: model unavailable on deployment.

• [sub-agent-failure] N/A — The intent-coherence sub-agent did not return findings: model unavailable on deployment.

• [sub-agent-failure] N/A — The docs-currency sub-agent did not return findings: model unavailable on deployment.

Previous run (4)

Review

Findings

Low

• [test-adequacy] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.test.ts — The ThresholdResolver test suite does not cover the case where a provider not present in the constructor's provider list is passed to resolveProviderThresholds(). This scenario correctly falls back to provider.getMetricThresholds() but the fallback path for unknown providers is untested.

• [edge-case] workspaces/scorecard/plugins/scorecard-backend/src/threshold/ThresholdResolver.ts — The filecheck batch provider exposes multiple metric IDs (e.g., filecheck.readme, filecheck.license) but its provider ID is filecheck, so all metrics share a single config override path (scorecard.plugins.filecheck.thresholds). Per-metric thresholds are not possible for batch providers. This is a pre-existing design limitation, not introduced by this PR.

Info

• [sub-agent-failure] N/A — The style-conventions, intent-coherence, docs-currency, and cross-repo-contracts sub-agents did not return findings: model claude-sonnet-4-5@20250929 is not available on the deployment. Cross-repo contracts were manually verified: all module packages only export their Backstage module registration — no provider classes or factory functions are part of the public API, so the constructor signature changes are internal and non-breaking.

[codecov]

Codecov Report

❌ Patch coverage is 98.18182% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 53.92%. Comparing base (c1fbaa9) to head (83812b0).
⚠️ Report is 8 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3293   +/-   ##
=======================================
  Coverage   53.92%   53.92%           
=======================================
  Files        2379     2380    +1     
  Lines       86125    86134    +9     
  Branches    23915    23911    -4     
=======================================
+ Hits        46441    46449    +8     
- Misses      38175    38176    +1     
  Partials     1509     1509           

Flag	Coverage Δ		*Carryforward flag
adoption-insights	83.58% <ø> (ø)		Carriedforward from 842c1be
ai-integrations	70.03% <ø> (ø)		Carriedforward from 842c1be
app-defaults	69.60% <ø> (ø)		Carriedforward from 842c1be
augment	46.39% <ø> (ø)		Carriedforward from 842c1be
bulk-import	72.86% <ø> (ø)		Carriedforward from 842c1be
cost-management	16.49% <ø> (ø)		Carriedforward from 842c1be
dcm	59.64% <ø> (ø)		Carriedforward from 842c1be
extensions	61.79% <ø> (ø)		Carriedforward from 842c1be
global-floating-action-button	74.30% <ø> (ø)		Carriedforward from 842c1be
global-header	61.63% <ø> (ø)		Carriedforward from 842c1be
homepage	51.52% <ø> (ø)		Carriedforward from 842c1be
konflux	91.01% <ø> (ø)		Carriedforward from 842c1be
lightspeed	68.50% <ø> (ø)		Carriedforward from 842c1be
mcp-integrations	85.46% <ø> (ø)		Carriedforward from 842c1be
orchestrator	37.34% <ø> (ø)		Carriedforward from 842c1be
quickstart	62.09% <ø> (ø)		Carriedforward from 842c1be
sandbox	79.56% <ø> (ø)		Carriedforward from 842c1be
scorecard	83.86% <98.18%> (+0.01%)	⬆️
theme	64.54% <ø> (ø)		Carriedforward from 842c1be
translations	8.49% <ø> (ø)		Carriedforward from 842c1be
x2a	78.79% <ø> (ø)		Carriedforward from 842c1be

*This pull request uses carry forward flags. Click here to find out more.

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1fbaa9...83812b0. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[fullsend-ai-review]

[low] test-inadequate

Typo in test fixture: github.other_netric should be github.other_metric. Does not affect test correctness but could cause confusion.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud