Add Agent Playbook section for automated updates

README.md

@@ -49,6 +49,76 @@ sbt "runMain com.codacy.pmd.DocGenerator"
 We use the [codacy-plugins-test](https://github.com/codacy/codacy-plugins-test) to test our external tools integration.
 You can follow the instructions there to make sure your tool is working as expected.
 
+## Agent Playbook: Updating This Repository End-to-End
+
+This section is written for an AI coding agent (or a human) tasked with updating this repo — most commonly bumping the wrapped [PMD](https://pmd.github.io/) version, but also base image / sbt / CircleCI orb bumps. Follow it top to bottom; it tells you what to change, how to regenerate derived files, how to test locally, and how to interpret CI so you can iterate on failures without guessing.
+
+### 1. What this repository is
+
+This is a **Codacy engine**: a Scala (2.13, sbt build) wrapper around [PMD](https://pmd.github.io/), built on `codacy-engine-scala-seed`, packaged as a Docker image that Codacy's platform runs against a customer's source code. PMD itself is pulled in as a set of ordinary sbt/Maven library dependencies (`pmd-core`, `pmd-java`, `pmd-apex`, `pmd-jsp`, `pmd-javascript`, `pmd-plsql`, `pmd-vm`, `pmd-xml`, `pmd-visualforce`, plus a forced `rhino` version) — there is no separate git clone of the PMD repo at build time.
+
+The `docs/` directory (`src/main/resources/docs/`) is machine-consumed configuration, same family as other Codacy engines:
+
+- `docs/patterns.json` — the full list of PMD rules ("patterns"), their parameters/defaults, and which are enabled by default. **Generated file, do not hand-edit.**
+- `docs/description/description.json` + `docs/description/*.md` — human-readable titles/descriptions per pattern, used in the Codacy UI. **Generated file, do not hand-edit.**
+- `docs/tests/*` and `docs/multiple-tests/*` — fixtures used by `codacy-plugins-test` to validate the engine against real code samples.
+- `docs/tool-description.md` — short blurb about the tool, hand-maintained.
+
+All the generated artifacts above come from **`DocGenerator`** (`src/main/scala/com/codacy/pmd/DocGenerator.scala`, run via `sbt "runMain com.codacy.pmd.DocGenerator"`). Unlike some sibling Codacy engines, it does **not** clone an external GitHub repo — it reads the ruleset XML files bundled as classpath resources inside the `pmd-*` jars that were just resolved for the new `toolVersionKey`, via `ResourceHelper`/`Languages`/`RuleSets`. It also regenerates `src/main/scala/com.codacy/pmd/RuleSets.scala` (marked `AUTOGENERATED: DO NOT CHANGE HERE`). This means an sbt dependency resolution (network access to Maven Central) is enough — no separate git clone or pandoc is needed here.
+
+### 2. Files that encode versions — check all of these on every update
+
+| File | What it controls | What to check |
+|---|---|---|
+| `build.sbt` → `toolVersionKey` | The PMD version bundled and used to generate `docs/patterns.json` | Bump to the target PMD release. Confirm all `pmd-*` artifacts (`pmd-core`, `pmd-java`, `pmd-apex`, `pmd-jsp`, `pmd-javascript`, `pmd-plsql`, `pmd-vm`, `pmd-xml`, `pmd-visualforce`) still publish that version on Maven Central — PMD has occasionally dropped/renamed a module between majors (e.g. `pmd-vm` had no PMD 7 release at the time of this repo's PMD-7 attempt and had to be commented out). |
+| `build.sbt` → `org.mozilla % rhino` forced version | Workaround for [pmd/pmd#2081](https://github.com/pmd/pmd/issues/2081) | Only touch if instructed or if the PMD bump reintroduces that bug; otherwise leave as-is. |
+| `build.sbt` → `codacy-engine-scala-seed` dependency | Codacy's engine SDK/base library | Check Maven Central if asked to bump it; not tied to PMD bumps. |
+| `build.sbt` → `dockerBaseImage` | JRE the packaged app runs on (currently `amazoncorretto:11-alpine3.18`) | Only bump if the new PMD version raises its minimum JDK requirement (check PMD's release notes) or if asked explicitly. |
+| `build.sbt` → `scalaVersion` | Scala compiler version | Rarely tied to a PMD bump; bump only if asked or if the build fails to resolve. |
+| `project/build.properties` | sbt version | Sometimes bumped alongside major tool bumps; check only if the build itself fails to load. |
+| `.circleci/config.yml` → `codacy/base` orb | Shared CircleCI steps (checkout, versioning, sbt build, docker publish, tagging) | Check the latest published version, or use `git log -p .circleci/config.yml` for prior bump history as a fallback reference. |
+| `.circleci/config.yml` → `codacy/plugins-test` orb | Runs `codacy-plugins-test` in CI after the image is built | Same as above. |
+
+Look at recent bump commits for the shape of a typical diff: `git log --oneline --all | grep -iE "bump|update pmd"`, then `git show <hash>`. Routine PMD point-releases (e.g. commit `feb32e8`, "bump pmd to latest version 6.55.0") touch only `build.sbt` plus the regenerated `docs/description/*.md`, `docs/description/description.json`, and `docs/patterns.json`. **Major PMD version bumps are riskier**: commit `98efa78` ("Bump to PMD 7") also touched `.circleci/config.yml` (orb bumps), `project/build.properties`, `dockerBaseImage`, added/removed `pmd-*` module dependencies, and even touched a Java helper class (`CodacyInMemoryRenderer.java`) — and it was subsequently **reverted** in commit `8b17d55` ("Revert tool (#128)": *"Revert PMD to version 7. Version 7 will be launched as a different tool"*) because of a real regression (duplicated violations on the same file/line). Treat a major-version PMD bump as substantially higher risk than a point-release bump, and be prepared for it to need dedicated engine-code changes, not just a version string change.
+
+### 3. Step-by-step update procedure
+
+1. **Bump the version(s)** in `build.sbt` (`toolVersionKey`, and `dockerBaseImage`/`scalaVersion`/`project/build.properties` only if actually required) and `.circleci/config.yml` orbs, as scoped by the task.
+2. **Regenerate the docs.** Requires network access to resolve the new PMD jars from Maven Central (no separate git clone or pandoc needed):
+   ```bash
+   sbt "runMain com.codacy.pmd.DocGenerator"
+   ```
+   This rewrites `docs/patterns.json`, `docs/description/description.json`, `docs/description/*.md`, and `src/main/scala/com.codacy/pmd/RuleSets.scala`. Review the diff for new/removed/renamed rules and stale fixtures under `docs/tests/`/`docs/multiple-tests/`. Watch the console output for `Ruleset ... is missing from rulesets.properties` warnings.
+3. **Format and compile:**
+   ```bash
+   sbt "scalafmt::test; test:scalafmt::test; sbt:scalafmt::test"
+   sbt "set Docker / version := \"latest\"; Docker / publishLocal"
+   ```
+   (This is the same sequence CI's `publish_docker_local` job runs.)
+4. **Run `codacy-plugins-test` locally** before pushing. CI runs it with `run_multiple_tests: true`, meaning both the single-pattern/json tests (`docs/tests/`) and the multiple-tests suite (`docs/multiple-tests/`) — clone https://github.com/codacy/codacy-plugins-test and run the corresponding DockerTest commands against your local `codacy-pmd:latest` image.
+5. **Iterate on failures**, re-running only the relevant test command after each fix.
+6. **Commit** the version bump(s) together with the regenerated `docs/` and `RuleSets.scala` files in one change.
+7. **Push and open a PR.** CI (CircleCI only — the GitHub Actions workflows in `.github/workflows/` are unrelated Jira-ticket-sync automation, not build/test) runs `codacy/checkout_and_version` -> `publish_docker_local` (runs `DocGenerator` again + scalafmt + docker build) -> `plugins_test` -> `codacy/publish_docker` (master only) -> `codacy/tag_version`.
+8. **Poll the PR's real CI checks until they all pass — local validation is NOT the finish line.** After every push, run `gh pr checks <pr-url>` and keep re-polling (short sleep while any check is `pending`) until all checks finish. If a check fails, fetch its actual log (the CircleCI job log — don't guess), find the true root cause, fix it, push again (never `--no-verify`, never force-push), and re-poll. Repeat until every check is green. This repo has direct history of a version bump that looked fine in isolation but caused a real regression only caught downstream (the PMD 7 bump merged, then had to be reverted for producing duplicated violations) — treat any test or behavior change surfaced by CI as a real signal to investigate, not noise to suppress. Only stop iterating when every check passes, or you hit a genuine product/infra decision that needs a human — in which case explain it in the PR rather than guessing.
+
+### 4. Common failure modes and fixes
+
+| Symptom | Likely cause | Fix |
+|---|---|---|
+| `scalafmt::test` fails in CI/locally | Generated or hand-edited Scala/Java file not formatted | Run `sbt scalafmt` then re-run the check command |
+| sbt fails to resolve a `pmd-*` module for the new version | Module renamed/removed/not yet released for that PMD line (seen with `pmd-vm` on the PMD 7 attempt) | Check PMD's Maven Central listing for that release; comment out or adjust the dependency as needed, matching the shape of commit `98efa78` |
+| `pattern`/`json` DockerTest fails | Rule renamed/removed/added upstream between PMD versions | Re-run `DocGenerator`; confirm the change matches PMD's release notes |
+| `multiple` DockerTest fails, especially around duplicate/overlapping violations | Renderer/engine behavior regression on major PMD bumps (this is exactly what caused the PMD 7 revert) | Do not just patch the fixture — verify the underlying violation output is actually correct before updating expectations |
+| CI `codacy/publish_docker`/`tag_version` don't run on your branch | Expected — gated to default branch (`master`) only | Nothing to fix |
+
+### 5. Definition of done
+
+- Version bump(s) reflected in all files that encode them (`build.sbt`, `.circleci/config.yml`, and `project/build.properties`/`dockerBaseImage` if actually needed).
+- `docs/patterns.json`, `docs/description/*`, and `RuleSets.scala` regenerated via `DocGenerator` and committed, with fixture inconsistencies under `docs/tests/`/`docs/multiple-tests/` resolved.
+- Local `scalafmt`/compile/docker build commands pass.
+- `codacy-plugins-test` (single and multiple-tests) pass locally against the freshly built image.
+- **After pushing and opening/updating the PR, every CI check on it is green.** Poll `gh pr checks <pr-url>` and iterate on any failure (fetch the real CI log, fix, push, re-poll) until all pass — a passing local build is not sufficient, and this repo has a concrete precedent (the PMD 7 revert) of a merged bump turning out to be wrong in practice.
+
 ## What is Codacy?
 
 [Codacy](https://www.codacy.com/) is an Automated Code Review Tool that monitors your technical debt, helps you improve your code quality, teaches best practices to your developers, and helps you save time in Code Reviews.