Skip to content

Add CI/CD GitHub Actions workflows, dependabot config and runner image - #19

Open
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/1787265031-cicd-workflows
Open

Add CI/CD GitHub Actions workflows, dependabot config and runner image#19
devin-ai-integration[bot] wants to merge 4 commits into
mainfrom
devin/1787265031-cicd-workflows

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

Adds ci.yml (compile check, headless TestNG matrix, dependency scanning, nightly cron), a fully mocked AWS delivery pipeline in cd.yml, dependabot.yml, a container runner image, and README docs. Two small framework changes were required to make CI feasible; everything else is workflow/config.

Framework changes (BaseTest)

The suite hardcoded a Chrome driver created via WebDriverManager, so a browser matrix and offline-safe driver resolution were impossible:

setup() {
  browser  = System.getProperty("browser", "chrome");   // chrome | firefox
  headless = System.getProperty("headless", "true");    // --headless=new / -headless
  // prefer a driver already on the box, fall back to WebDriverManager:
  if (!resolveDriverBinary("webdriver.chrome.driver", "CHROMEWEBDRIVER", "chromedriver"))
      WebDriverManager.chromedriver().setup();
}

resolveDriverBinary honours webdriver.*.driver and the CHROMEWEBDRIVER/GECKOWEBDRIVER directories that GitHub runners (and the new Dockerfile) expose, so no driver download is needed at run time. wrapUp now calls only driver.quit() — the previous close() + quit() pair throws NoSuchSessionException on Firefox, where close() already ends the session (reproduced locally in a container with real Firefox 154).

pom.xml gains a suiteXmlFile property so Surefire's suite is selectable with -DsuiteXmlFile=.... Compiler source/target stay at 1.8; the JDK in CI is Temurin 17 because Selenium 4.25 ships Java 11 class files and cannot be loaded by a JDK 8 toolchain.

ci.yml

push/PR + nightly cron (02:00 UTC) + workflow_dispatch (suite, browser chrome/firefox/both, thread_count — applied by rewriting thread-count="N" into a copy of the suite before the run). Jobs: compile (mvn -ntp -B clean test-compile), matrix-setup (resolves the browser matrix from the dispatch input), test (chrome/firefox matrix, headless, Extent HTML + logfile.log + Surefire/TestNG artifacts, job-summary table rendered by .github/scripts/testng_summary.py), dependency-scan (Trivy SARIF + dependency tree) and dependency-review on PRs. Concurrency groups cancel superseded PR runs only; permissions are read-only except where SARIF/PR comments need more.

The suite job is intentionally non-gating (continue-on-error + -Dmaven.test.failure.ignore=true): the demo tests drive google.co.in and facebook.com, and FaceBookLoginTest asserts false by design, so an outage or that intentional failure would otherwise red-flag every run. The job summary distinguishes "no results produced" (suite never started) from real failures. The gating signal is the compile job.

cd.yml — mocked, cannot touch real infra

workflow_dispatch only, and DRY_RUN unless dry_run=false and repository variable ENABLE_REAL_AWS_DEPLOY == 'true'. In DRY_RUN the runner image is built but no AWS credentials are configured — the aws ecr/aws ecs run-task/aws s3 cp commands are only echoed into the job summary. Real-run steps use AWS OIDC (permissions: id-token: write) with a placeholder role ARN. All identifiers are commented placeholders: 123456789012.dkr.ecr.us-east-1.amazonaws.com, repo/cluster demo-selenium-testng, task def demo-selenium-testng-runner, reports bucket demo-selenium-testng-reports. Deploy stages run through GitHub environments staging then production (add required reviewers to production for the approval gate).

Verified locally

  • mvn -ntp clean test-compile and headless single-class run: pass.
  • Full suite exactly as CI invokes it (suite copy with rewritten thread-count, CHROMEWEBDRIVER set, -Dmaven.test.failure.ignore=true): Tests run: 2, Failures: 1 (the intentional FaceBookLoginTest) with BUILD SUCCESS, and the summary script renders that table.
  • docker build + docker run of the new image: suite runs headlessly inside the container against Chrome for Testing.
  • Firefox could not be validated on a GitHub-like runner here (this VM's base images only ship the snap stub); in a container with a real Firefox the session starts and navigates, and Google's consent page then fails the assertion — another reason the suite job is non-gating.

Link to Devin session: https://partner-workshops.devinenterprise.com/sessions/2f4377267f124bd29acb1ea2c36eb3de
Requested by: @mbatchelor81

@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration

Copy link
Copy Markdown
Author

End-to-end verification of the CI/CD changes (headless, local runner)

Executed the pipeline's real commands on a Linux box (Chrome 137.0.7118.2 / chromedriver 137.0.7151.119). Firefox leg skipped locally — not installed on the test machine — but the UI suite (firefox) job passes on GitHub Actions.

Check Result
mvn -ntp -B clean test-compile BUILD SUCCESS
CI invocation: sed thread-count→3, -Dheadless=true -Dmaven.test.failure.ignore=true Tests run: 2, Failures: 1 + BUILD SUCCESS (exit 0)
Driver taken from CHROMEWEBDRIVER, no WebDriverManager download 0 download log lines; ~/.cache/selenium unchanged
BROWSER=chrome python3 .github/scripts/testng_summary.py | 2 | 1 | 1 | 0 | 0 | 12.6 | + failed-test list
ExtentReports HTML with embedded base64 screenshots renders (below)
docker build + docker run smoke suite (GoogleSearchTest only) Tests run: 1, Failures: 0, BUILD SUCCESS
actionlint 1.7.7 + shellcheck 0.10.0 on ci.yml/cd.yml 0 errors
CD guard: dry_run=false only for input false and ENABLE_REAL_AWS_DEPLOY=true verified for all 4 combos; every aws step gated
ExtentReports output from the headless run

ExtentReports: 1 pass / 1 fail
Embedded base64 screenshot renders

Notes
  • FaceBookLoginTest fails with NoSuchElementException on #email before reaching its intentional assertTrue(false) — still failing by design; the summary script wording was corrected accordingly.
  • The passing GoogleSearchTest screenshot shows Google's "unusual traffic" captcha page — the title assertion passes anyway, so a green result there isn't proof the search executed (pre-existing test weakness).
  • run-suite in cd.yml now checks out the repo, so the real-run S3 publish step is no longer missing a working directory.

Tested by Devin — session: https://partner-workshops.devinenterprise.com/sessions/2f4377267f124bd29acb1ea2c36eb3de

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant