[WIP] Jv use only nightlies for perf scale baseline#79890
Open
JoukoVirtanen wants to merge 3 commits into
Open
[WIP] Jv use only nightlies for perf scale baseline#79890JoukoVirtanen wants to merge 3 commits into
JoukoVirtanen wants to merge 3 commits into
Conversation
Contributor
|
Skipping CI for Draft Pull Request. |
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
Contributor
WalkthroughThis PR adds ORION_PR_ANALYSIS env vars to Orion step definitions, implements presubmit gating in the openshift-qe-orion commands script to pass --pr-analysis and input-vars, enables the variable in the StackRox perfscale chain, and adds a new perf-scale baseline manifest. ChangesOrion PR Analysis Feature
Sequence DiagramsequenceDiagram
participant CI_Job as CI Job (presubmit)
participant CmdScript as openshift-qe-orion-commands.sh
participant Orion as Orion CLI
participant Baseline as Periodic Baseline Store
CI_Job->>CmdScript: ORION_PR_ANALYSIS="true", PULL_NUMBER present
CmdScript->>Orion: run with --pr-analysis and --input-vars (jobtype,pull_number,organization,repository)
Orion->>Baseline: compare PR results to periodic baseline
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci-operator/step-registry/openshift-qe/orion/openshift-qe-orion-commands.sh`:
- Around line 192-197: The current EXTRA_FLAGS is built as a plain string so the
appended --input-vars includes literal surrounding single quotes and is passed
incorrectly; change EXTRA_FLAGS to a Bash array (e.g., declare -a EXTRA_FLAGS)
and replace string appends with array appends for flags (use
EXTRA_FLAGS+=(--pr-analysis) and add the JSON as a single separate array element
for --input-vars so the JSON is not wrapped in extra quotes), then update the
Orion invocation to use "${EXTRA_FLAGS[@]}" so each flag/argument is passed
correctly; locate and update the code that currently mutates EXTRA_FLAGS in the
conditional (the lines that append --pr-analysis and --input-vars) and the later
command that invokes Orion to use the array expansion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: c7a69877-b577-4d2f-beda-5422b732f60a
📒 Files selected for processing (6)
ci-operator/step-registry/openshift-qe/orion/cluster-density/openshift-qe-orion-cluster-density-ref.yamlci-operator/step-registry/openshift-qe/orion/crd-scale/openshift-qe-orion-crd-scale-ref.yamlci-operator/step-registry/openshift-qe/orion/node-density/openshift-qe-orion-node-density-ref.yamlci-operator/step-registry/openshift-qe/orion/openshift-qe-orion-commands.shci-operator/step-registry/openshift-qe/orion/openshift-qe-orion-ref.yamlci-operator/step-registry/stackrox/perfscale/stackrox-perfscale-chain.yaml
Comment on lines
+192
to
+197
| if [[ "${ORION_PR_ANALYSIS:-false}" == "true" ]] && [[ "${JOB_TYPE}" == "presubmit" ]] && [[ -n "${PULL_NUMBER:-}" ]]; then | ||
| job_type="pull" | ||
| pull_number="${PULL_NUMBER}" | ||
| EXTRA_FLAGS+=" --pr-analysis" | ||
| EXTRA_FLAGS+=" --input-vars '{\"jobtype\":\"pull\",\"pull_number\":\"${PULL_NUMBER}\",\"organization\":\"${REPO_OWNER}\",\"repository\":\"${REPO_NAME}\"}'" | ||
| elif [[ "${JOB_TYPE}" == "periodic" ]]; then |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
current=' --pr-analysis --input-vars '\''{"jobtype":"pull","pull_number":"123","organization":"org","repository":"repo"}'\'''
echo "Current tokenization:"
for t in $current; do printf '[%s]\n' "$t"; done
echo
echo "Array tokenization:"
arr=(--pr-analysis --input-vars '{"jobtype":"pull","pull_number":"123","organization":"org","repository":"repo"}')
for t in "${arr[@]}"; do printf '[%s]\n' "$t"; doneRepository: openshift/release
Length of output: 332
Fix --input-vars quoting so Orion receives raw JSON (not literal '{...}')
EXTRA_FLAGSis a plain string and is later expanded unquoted, so the--input-varsargument passed to Orion includes literal surrounding single quotes ('{...}') rather than raw JSON, which can break PR-analysis input-vars parsing.- Build
EXTRA_FLAGSas a Bash array and append--input-varswith a single JSON argument, then invoke Orion with"${EXTRA_FLAGS[@]}".
Proposed fix (use argv array for safe flag passing)
-EXTRA_FLAGS="${ORION_EXTRA_FLAGS:-} --lookback ${LOOKBACK}d --hunter-analyze"
+EXTRA_FLAGS=()
+if [[ -n "${ORION_EXTRA_FLAGS:-}" ]]; then
+ # Split user-provided extra flags intentionally.
+ # shellcheck disable=SC2206
+ EXTRA_FLAGS+=(${ORION_EXTRA_FLAGS})
+fi
+EXTRA_FLAGS+=(--lookback "${LOOKBACK}d" --hunter-analyze)
@@
- EXTRA_FLAGS+=" --pr-analysis"
- EXTRA_FLAGS+=" --input-vars '{\"jobtype\":\"pull\",\"pull_number\":\"${PULL_NUMBER}\",\"organization\":\"${REPO_OWNER}\",\"repository\":\"${REPO_NAME}\"}'"
+ EXTRA_FLAGS+=(--pr-analysis)
+ EXTRA_FLAGS+=(--input-vars "{\"jobtype\":\"pull\",\"pull_number\":\"${PULL_NUMBER}\",\"organization\":\"${REPO_OWNER}\",\"repository\":\"${REPO_NAME}\"}")
@@
-orion --node-count ${IGNORE_JOB_ITERATIONS} --config ${ORION_CONFIG} ${EXTRA_FLAGS} --viz | tee ${ARTIFACT_DIR}/orion-output.txt
+orion --node-count "${IGNORE_JOB_ITERATIONS}" --config "${ORION_CONFIG}" "${EXTRA_FLAGS[@]}" --viz | tee "${ARTIFACT_DIR}/orion-output.txt"🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 196-196: Quotes/backslashes will be treated literally. Use an array.
(SC2089)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci-operator/step-registry/openshift-qe/orion/openshift-qe-orion-commands.sh`
around lines 192 - 197, The current EXTRA_FLAGS is built as a plain string so
the appended --input-vars includes literal surrounding single quotes and is
passed incorrectly; change EXTRA_FLAGS to a Bash array (e.g., declare -a
EXTRA_FLAGS) and replace string appends with array appends for flags (use
EXTRA_FLAGS+=(--pr-analysis) and add the JSON as a single separate array element
for --input-vars so the JSON is not wrapped in extra quotes), then update the
Orion invocation to use "${EXTRA_FLAGS[@]}" so each flag/argument is passed
correctly; locate and update the code that currently mutates EXTRA_FLAGS in the
conditional (the lines that append --pr-analysis and --input-vars) and the later
command that invokes Orion to use the array expansion.
JoukoVirtanen
changed the title
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: JoukoVirtanen The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
|
[REHEARSALNOTIFIER]
A total of 276 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs. A full list of affected jobs can be found here Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml (1)
16-20: ⚡ Quick winConsider adding resource limits.
The configuration sets resource requests but no limits. While this may be intentional for performance scale testing to avoid throttling, the coding guidelines recommend defining resource limits (cpu, memory) on every container.
Consider either:
- Adding explicit
limitsalongsiderequests, or- Documenting why limits are intentionally omitted for this performance baseline test
📊 Example with limits added
resources: '*': requests: cpu: 2000m memory: 4000Mi + limits: + cpu: 4000m + memory: 8000Mi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml` around lines 16 - 20, The resources block currently only sets requests (resources -> '*' -> requests -> cpu/memory); add matching limits to that same resources entry (resources -> '*' -> limits -> cpu and memory) to conform with guidelines, or else add a clear comment or top-level note in this YAML explaining why limits are intentionally omitted for the perf-scale baseline; update the resources '*' entry so limits mirror appropriate upper bounds (e.g., cpu and memory values) or include the documentation comment adjacent to the requests stanza.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml`:
- Around line 11-15: The releases.latest.release.channel/version fields are
correctly pinned to stable/4.20.8 and should not be changed to a nightly stream;
instead, ensure the nightlies behavior is driven by zz_generated_metadata.branch
and the presubmit StackRox branch selection. Update the file to keep
releases.latest.release.channel: stable and version: 4.20.8 intact, and add or
adjust a clarifying comment near releases.latest.release and
zz_generated_metadata.branch (or ensure the presubmit/job which reads
zz_generated_metadata.branch uses the StackRox branch like "nightlies" or
"jv-use-only-nightlies-for-perf-scale-baseline") so future readers know
nightlies are controlled by the branch metadata rather than the OCP release
channel.
---
Nitpick comments:
In
`@ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml`:
- Around line 16-20: The resources block currently only sets requests (resources
-> '*' -> requests -> cpu/memory); add matching limits to that same resources
entry (resources -> '*' -> limits -> cpu and memory) to conform with guidelines,
or else add a clear comment or top-level note in this YAML explaining why limits
are intentionally omitted for the perf-scale baseline; update the resources '*'
entry so limits mirror appropriate upper bounds (e.g., cpu and memory values) or
include the documentation comment adjacent to the requests stanza.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 94ddd0e9-cdb5-4694-acc7-b96044c6191e
⛔ Files ignored due to path filters (1)
ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yamlis excluded by!ci-operator/jobs/**
📒 Files selected for processing (1)
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml
Comment on lines
+11
to
+15
| releases: | ||
| latest: | ||
| release: | ||
| channel: stable | ||
| version: 4.20.8 |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Verify OCP 4.20.8 exists and check nightly vs stable alignment
# Check if version 4.20.8 exists in stable channel
echo "Checking for OCP 4.20.8 in stable channel..."
oc adm release info registry.ci.openshift.org/ocp/release:4.20.8 2>&1 | head -20
# List recent 4.20 stable releases
echo -e "\nRecent 4.20 stable releases:"
curl -s https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.20 | jq -r '.nodes[].version' | sort -V | tail -10Repository: openshift/release
Length of output: 257
🌐 Web query:
OpenShift CI operator nightly release configuration vs stable channel
💡 Result:
In ci-operator test-release configuration, “nightly” vs “stable” is primarily a matter of which release “channel/stream” (or Cincinnati channel) you point the job at. 1) How ci-operator selects OpenShift payloads (release configuration) - The ci-operator config supports a releases map where you define named release targets such as latest, previous, candidate, etc. [1] - For the “stable” channel specifically, the docs show latest.release configured with channel: stable and version: ..., where channel: stable means ci-operator will query Red Hat’s Cincinnati upgrade-service graph using stable-<major>.<minor> (example described in the docs). [1] - The same doc describes other examples: - previous.candidate.stream: nightly for referencing a candidate release stream named nightly. [1] - previous.candidate also has a relative: 1 selector to pick the Nth latest payload in that stream. [1] 2) What “nightly” means in ci-operator vs what “stable” means in channels - “Stable” (in ci-operator terms) is wired to Cincinnati via channel: stable. The stable channel delays promotion compared to “fast/candidate” style channels; the delay exists to collect/analyze regression signals before adding a release to stable. [2] - “Nightly” (in ci-operator terms) is used as a candidate release “stream” (stream: nightly) to select from a continually updated stream of candidate payloads. In other words, nightly is not the same mechanism as channel: stable; it’s a different selector path (stream-based candidate vs Cincinnati channel-based stable). [1] 3) Concrete example of where “stable” vs “nightly” shows up in OpenShift release CI job naming/config - In the OpenShift release CI job context, you can see jobs whose names include ...-main-stable-4.y... and others include ...-main-nightly-... (e.g., an upgrade e2e periodic job and a metal/ipi nightly periodic-ci job tag/name). This is consistent with the distinction between a stable-channel job vs a nightly-stream job. [3] So, the practical difference is: - Stable channel: releases.*.release.channel: stable (Cincinnati-backed; promoted after analysis delay). [1][2] - Nightly stream: releases.*.candidate.stream: nightly (select from a nightly candidate stream, often with relative to pick a specific nightly payload). [1]
Citations:
- 1: https://docs.ci.openshift.org/docs/architecture/ci-operator/
- 2: https://docs.redhat.com/en/documentation/openshift_container_platform/4.10/html/updating_clusters/understanding-upgrade-channels-releases
- 3: https://openshift-release.apps.ci.l2s4.p1.openshiftapps.com/releasetag/4.16.58?from=4.16.57
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check whether 4.20.8 exists in the stable-4.20 upgrade graph
echo "Checking stable-4.20 graph for 4.20.8..."
curl -s "https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.20" \
| jq -r '
[.nodes[].version] | unique | map(select(.=="4.20.8")) | length
'
# If present, print surrounding versions for context
echo "Printing nearest stable versions to 4.20.8 (if any)..."
curl -s "https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.20" \
| jq -r '
[.nodes[].version] | unique | sort_by(.) |
( [ .[] ] | . )
' | awk '
BEGIN{target="4.20.8"}
{versions[NR]=$0}
END{
for(i=1;i<=NR;i++){
if(versions[i]==target){print "FOUND_AT="i; found=1}
}
if(!found){
print "NOT_FOUND";
}
}'
# Find other stackrox perf-scale baseline configs to compare "nightlies-only" vs "stable"
echo "Searching for similar perf-scale baseline stackrox configs..."
fd -a "stackrox-stackrox-.*perf-scale.*\.ya?ml" . 2>/dev/null | head -50 || trueRepository: openshift/release
Length of output: 701
🏁 Script executed:
#!/bin/bash
set -euo pipefail
target="ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
echo "=== Target file ($target) ==="
nl -ba "$target" | sed -n '1,80p'
echo
echo "=== Related configs ==="
for f in \
"ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-master__perf-scale.yaml"
do
if [ -f "$f" ]; then
echo
echo "--- $f ---"
# Print only the releases section if present; fallback to first 120 lines
if rg -n "^\s*releases\s*:" "$f" >/dev/null 2>&1; then
start=$(rg -n "^\s*releases\s*:" "$f" | head -1 | cut -d: -f1)
end=$((start+80))
nl -ba "$f" | sed -n "${start},${end}p"
else
nl -ba "$f" | head -n 120
fi
else
echo
echo "--- $f (missing) ---"
fi
doneRepository: openshift/release
Length of output: 262
🏁 Script executed:
#!/bin/bash
set -euo pipefail
show_releases_block () {
local f="$1"
echo
echo "=== $f (releases block) ==="
if [ ! -f "$f" ]; then
echo "(missing)"
return
fi
# Find the first "releases:" line number
local start
start=$(rg -n "^[[:space:]]*releases:[[:space:]]*$" "$f" | head -1 | cut -d: -f1 || true)
if [ -z "${start}" ]; then
echo "(no releases: key found)"
return
fi
# Print from releases: through next ~80 lines or until EOF
local end=$((start+90))
awk -v s="$start" -v e="$end" 'NR>=s && NR<=e {printf "%5d\t%s\n", NR, $0}' "$f" | sed -n '1,90p'
}
target="ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
echo "=== Target file: $target ==="
show_releases_block "$target"
echo
echo "=== Related configs ==="
show_releases_block "ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml"
show_releases_block "ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml"
show_releases_block "ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
show_releases_block "ci-operator/config/stackrox/stackrox/stackrox-stackrox-master__perf-scale.yaml"
# Also quickly show any occurrences of "nightlies" vs "stable" around releases.*.release.channel/stream
for f in "$target" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
do
echo
echo "=== Channel/stream selectors in $f ==="
rg -n "(channel:\s*stable|stream:\s*nightly|channel:\s*nightly|releases:|latest:|candidate:|previous:|release:|version:)" "$f" || true
doneRepository: openshift/release
Length of output: 10070
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== OCP stable-4.20 graph: exact match for 4.20.8 ==="
curl -s "https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.20" \
| jq -r '.nodes[].version' \
| sort -u \
| grep -Fxq "4.20.8" && echo "FOUND" || echo "NOT_FOUND"
echo
echo "=== Neighbor versions around 4.20.8 (from stable graph) ==="
curl -s "https://api.openshift.com/api/upgrades_info/v1/graph?channel=stable-4.20" \
| jq -r '.nodes[].version' \
| sort -V -u \
| awk '
BEGIN{t="4.20.8"; seen=0}
{v[NR]=$0}
END{
# print last 5 <= t and first 5 >= t
for(i=1;i<=NR;i++){
if(v[i]==t){seen=i;break}
}
if(seen==0){ # not found
# find insertion point: first > t
ins=NR+1
for(i=1;i<=NR;i++){
if(v[i]>t){ins=i;break}
}
start=ins-5; if(start<1)start=1
end=ins+4; if(end>NR)end=NR
for(i=start;i<=end;i++)print v[i]
} else {
start=seen-5; if(start<1)start=1
end=seen+4; if(end>NR)end=NR
for(i=start;i<=end;i++)print v[i]
}
}'
echo
echo "=== Search for nightly stream selectors in stackrox perf-scale configs ==="
rg -n "stream:\s*nightly|candidate:\s*\n|candidate\.stream:\s*nightly" ci-operator/config/stackrox/stackrox \
|| trueRepository: openshift/release
Length of output: 542
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Occurrences of 'nightly' in target and related configs ==="
rg -n "nightly" \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-master__perf-scale.yaml \
ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml
echo
echo "=== Search for OCP payload selectors in stackrox perf-scale configs ==="
rg -n "releases:|channel:|stream:|candidate:|candidate\.stream:" ci-operator/config/stackrox/stackrox --glob '*perf-scale*.yaml' || true
echo
echo "=== Inspect releases+metadata sections for the target file (around lines ~1-80 and metadata tail) ==="
f="ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
wc -l "$f"
# first 120 lines
sed -n '1,120p' "$f"
# last 60 lines
echo "----- last 60 lines -----"
sed -n "$(( $(wc -l < "$f") - 60 + 1 )),$p" "$f"
echo
echo "=== Search for how zz_generated_metadata.branch 'nightlies' is consumed ==="
rg -n "zz_generated_metadata|branch:\s*nightlies|jv-use-only-nightlies-for-perf-scale-baseline" ci-operator --glob '*.yaml' || trueRepository: openshift/release
Length of output: 124
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Occurrences of 'nightly' in target and related configs ==="
rg -n "nightly" \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml \
ci-operator/config/stackrox/stackrox/stackrox-stackrox-master__perf-scale.yaml \
ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml
echo
echo "=== Search for OCP payload selectors in stackrox perf-scale configs ==="
rg -n "releases:|channel:|stream:|candidate:|candidate\.stream:" ci-operator/config/stackrox/stackrox --glob '*perf-scale*.yaml' || true
echo
echo "=== Inspect releases+metadata sections for the target file ==="
f="ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
wc -l "$f"
sed -n '1,140p' "$f"
echo "----- last 60 lines -----"
sed -n "$(( $(wc -l < "$f") - 60 + 1 )),$p" "$f"
echo
echo "=== Search for how zz_generated_metadata.branch 'nightlies' is used ==="
rg -n "zz_generated_metadata|branch:\s*nightlies|jv-use-only-nightlies-for-perf-scale-baseline" ci-operator --glob '*.yaml' || trueRepository: openshift/release
Length of output: 124
🏁 Script executed:
#!/bin/bash
set -euo pipefail
targets=(
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml"
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml"
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-master__perf-scale.yaml"
"ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml"
)
echo "=== Search for 'nightlies' (plural) in target-related files ==="
for f in "${targets[@]}"; do
echo "--- $f ---"
rg -n "nightlies" "$f" || true
done
echo
echo "=== Inspect top of presubmits YAML (to find how it selects payloads) ==="
p="ci-operator/jobs/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline-presubmits.yaml"
if [ -f "$p" ]; then
sed -n '1,200p' "$p"
echo
echo "=== Search in presubmits YAML for release/payload selectors ==="
rg -n "releases:|channel:|stream:|candidate:|version:|zz_generated_metadata|branch:" "$p" || true
else
echo "Missing: $p"
fi
echo
echo "=== Inspect zz_generated_metadata in perf-scale configs (tail areas) ==="
for f in \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml" \
"ci-operator/config/stackrox/stackrox/stackrox-stackrox-nightlies__perf-scale.yaml"
do
echo "--- $f (tail 60 lines) ---"
tail -n 80 "$f"
doneRepository: openshift/release
Length of output: 8705
OCP stable pin to 4.20.8 is consistent; “nightlies-only” is implemented via StackRox branch, not OCP payload channel.
releases.latest.release.channel: stable/version: 4.20.8points to an existing OpenShift 4.20 stable payload.- The “nightlies” perf-scale configs (
...nightlies__perf-scale.yamland...jv-use-only-nightlies...__perf-scale.yaml) also keepstable/4.20.8; the “nightlies” behavior is driven byzz_generated_metadata.branch(e.g.,branch: nightlies/branch: jv-use-only-nightlies-for-perf-scale-baseline) and the presubmit’s StackRox branch selection, not byreleases.*.release.channelor any nightly stream.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@ci-operator/config/stackrox/stackrox/stackrox-stackrox-jv-use-only-nightlies-for-perf-scale-baseline__perf-scale.yaml`
around lines 11 - 15, The releases.latest.release.channel/version fields are
correctly pinned to stable/4.20.8 and should not be changed to a nightly stream;
instead, ensure the nightlies behavior is driven by zz_generated_metadata.branch
and the presubmit StackRox branch selection. Update the file to keep
releases.latest.release.channel: stable and version: 4.20.8 intact, and add or
adjust a clarifying comment near releases.latest.release and
zz_generated_metadata.branch (or ensure the presubmit/job which reads
zz_generated_metadata.branch uses the StackRox branch like "nightlies" or
"jv-use-only-nightlies-for-perf-scale-baseline") so future readers know
nightlies are controlled by the branch metadata rather than the OCP release
channel.
Contributor
|
@JoukoVirtanen: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
This PR adds an opt-in mode that makes Orion use only periodic (nightly) runs as baselines when analyzing pull requests, preventing presubmit runs from polluting or being compared against other PR runs.
What changed (practical effect)
Affected areas
Impact