Skip to content

сhore(ci): improve e2e nightly report#2245

Open
universal-itengineer wants to merge 13 commits intomainfrom
chore/ci/improve-e2e-nightly-report
Open

сhore(ci): improve e2e nightly report#2245
universal-itengineer wants to merge 13 commits intomainfrom
chore/ci/improve-e2e-nightly-report

Conversation

@universal-itengineer
Copy link
Copy Markdown
Member

@universal-itengineer universal-itengineer commented Apr 20, 2026

Description

Refactor nested E2E reporting in GitHub Actions by replacing inline shell-based report aggregation with reusable Node.js scripts and JSON-based intermediate reports.

The change introduces dedicated report builders for:

  • parsing JUnit XML test results into structured per-cluster JSON reports;
  • detecting failed pipeline stages and generating fallback reports when test artifacts are missing;
  • rendering a consolidated nested-cluster E2E summary message;
  • publishing the summary and failed-test thread to Loop via API.

The workflows were updated to install Node.js dependencies, execute the new scripts through actions/github-script, and pass structured outputs and artifacts between reusable and matrix workflows.

Additional tests, linting configuration, JSDoc annotations, and reporting fixes were added for the new logic, including skipped-test handling and a stricter single-file lookup for report discovery.

Why do we need it, and what problem does it solve?

The previous implementation built E2E summaries with large inline bash scripts in workflows. That approach was hard to maintain, hard to test, and fragile when artifacts were missing or when failures happened before the E2E stage.

This change moves the reporting flow into tested JavaScript modules and uses structured JSON reports as the contract between workflow stages. That makes report generation easier to validate, easier to evolve, and less dependent on brittle shell parsing.

It also improves failure handling by preserving stage-level metadata, distinguishing between test failures and cluster setup failures, supporting missing-artifact scenarios explicitly, and correctly accounting for skipped tests.

As a result, nested-cluster E2E reporting becomes more reliable and maintainable for both successful and failed runs.

What is the expected result?

  1. Run the nested E2E workflows for configured storage types.
  2. The reusable pipeline produces a structured e2e_report_<storage>.json report based on JUnit XML data or fallback stage metadata.
  3. The matrix workflow downloads the generated report artifacts and renders a consolidated summary.
  4. The summary message contains:
    • a test results table for clusters with test reports;
    • a separate cluster-failures section for setup-stage or artifact-related failures;
    • a failed-tests thread when testcase-level failures are available.
  5. If Loop credentials are configured, both the main message and the thread reply are sent through the Loop API.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: ci
type: fix
summary: "Refactor nested E2E report generation into tested Node.js scripts with JSON reports and improve Loop reporting for missing artifacts, skipped tests, and stage failures."
impact_level: low

@universal-itengineer universal-itengineer changed the title Chore/ci/improve e2e nightly report сhore(ci): improve e2e nightly report Apr 20, 2026
@universal-itengineer universal-itengineer force-pushed the chore/ci/improve-e2e-nightly-report branch 2 times, most recently from 5497087 to 32d79af Compare April 24, 2026 12:21
@universal-itengineer universal-itengineer added this to the v1.9.0 milestone Apr 24, 2026
@universal-itengineer universal-itengineer marked this pull request as ready for review April 24, 2026 12:28
"artifact-missing": "TEST REPORTS NOT FOUND",
};

const preE2EStages = new Set([
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why Set?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const reportKind = preE2EStages.has(stageName) ? "stage-failure" : "tests";

Comment thread .github/scripts/js/e2e/report/cluster-report.js Outdated
Comment thread .github/scripts/js/e2e/report/cluster-report.js Outdated
};
}

function hasOwnProperty(object, key) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe move to utils?

reportSource: parsedReport.source,
};

fs.writeFileSync(config.reportFile, `${JSON.stringify(report, null, 2)}\n`);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handle errors

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
@universal-itengineer universal-itengineer force-pushed the chore/ci/improve-e2e-nightly-report branch 2 times, most recently from 06ba221 to f8b5531 Compare April 28, 2026 09:28
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
@universal-itengineer universal-itengineer force-pushed the chore/ci/improve-e2e-nightly-report branch from f8b5531 to 35d8060 Compare April 28, 2026 11:22
@universal-itengineer universal-itengineer force-pushed the chore/ci/improve-e2e-nightly-report branch 3 times, most recently from 27f6952 to 35d8060 Compare April 30, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants