Skip to content

fix(helm): upgrade keycloak to 26.6 - #679

Merged
vincentchalamon merged 5 commits into
4.3from
fix/keycloak-upgrade
Jul 30, 2026
Merged

fix(helm): upgrade keycloak to 26.6#679
vincentchalamon merged 5 commits into
4.3from
fix/keycloak-upgrade

Conversation

@vincentchalamon

Copy link
Copy Markdown
Contributor

Split out of #676 so it can land quickly, and carrying the dependency updates already reviewed in #678. The wildcard redirectUris removal is now #676 alone.

Summary

Keycloak 26.4 is inside the vulnerable range of CVE-2026-7504 / GHSA-rp95-xpg9-c2cq, fixed upstream in 26.6.2. This bumps the pinned image to 26.7 (latest stable) and adorsys/keycloak-config-cli to 6.5.1-26.

This does not remediate GHSA-2rmv-4743-38c6 on its own. Measured on 26.7.0 with the current wildcard realm freshly imported: api-platform-pwa and api-platform-swagger still answer 200 to redirect_uri=https://attacker.evil.example/callback, while the properly scoped built-in account client answers 400 Invalid parameter: redirect_uri`. Removing the wildcards is what closes it — that is #676.

Two upstream behaviour changes had to be handled

Without them the bump simply does not work.

  1. Since 26.6, JavaScript-based policies require the scripts feature. Otherwise the realm import aborts with Couldn't find policy provider with type [script-owner-policy.js] and Keycloak never starts. Enabled at build time for start --optimized, and through KC_FEATURES for start-dev, which re-runs the augmentation and drops build-time options.

  2. Since 26.6.2 the Protection API forces the owner of a resource to the resource server itself (keycloak/keycloak#49910, introduced by Enforce owner checks when calling the resource set service keycloak/keycloak#49121 while fixing CVE-2025-14778). Confirmed intended and permanent by the maintainer:

    The only possible way for a resource server to create user-owned resources is through the Admin API now.

    This broke the Owner policy: reviews ended up owned by the api-platform-api client instead of their author, so resource.owner == identity.id never matched and the three front-office review E2E tests failed. Resource creation, lookup and deletion now go through the Admin API, and the service account is granted the single manage-authorization role that requires. The client's internal id comes from configuration, so no view-clients role is needed.

    Verified against 26.6.4 and 26.7.0, with owner sent as a string and as an object: the Protection API always returned the resource server. Isolating it further, the same realm on 26.4 kept every test green.

Dependency updates

The commits of #678 ride along unchanged — ops images, Helm chart, Composer, Symfony 8.1, the PWA block and Playwright, one commit per step. Their rationale and the deliberate exclusions (ESLint 10, TypeScript 7, PostgreSQL, Symfony recipes) are in #678.

Verification

Composed stacks, not just unit level.

  • Resource ownership restored on 26.7: fixtures now register owner: {"name": "john.doe@example.com"} instead of the client.
  • E2E (compose.yaml:compose.prod.yaml:compose.e2e.yaml, images rebuilt, realm reimported from scratch, fixtures reloaded): @write 8 passed / 3 flaky / 0 failed — the flaky set is the pre-existing login-page timing, green on retry.
  • Backend: PHPUnit 126 tests / 576 assertions OK · PHPStan no errors · rector --dry-run clean · doctrine:schema:validate in sync · api:openapi:export | redocly lint valid · php-cs-fixer 0 of 76 files · hadolint clean.
  • Helm: helm lint plus helm template on demo.api-platform.com and pr-42-demo.api-platform.com.

Follow-up

Production only deploys on tag push (cd.yml), so demo.api-platform.com needs a tag once this lands.


🤖 Generated with Claude Code

vincentchalamon and others added 2 commits July 30, 2026 15:46
Keycloak 26.4 is within the vulnerable range of CVE-2026-7504 /
GHSA-rp95-xpg9-c2cq, fixed upstream in 26.6.2. keycloak-config-cli follows to
6.5.1-26.

Two upstream behaviour changes had to be handled to make the bump work:

- Since 26.6, JavaScript-based policies require the `scripts` feature, otherwise
  importing the realm fails with "Couldn't find policy provider with type
  [script-owner-policy.js]". It is enabled at build time for `start --optimized`
  and through KC_FEATURES for `start-dev`, which re-runs the augmentation and
  drops build-time options.
- Since 26.6.2 the Protection API forces the owner of a resource to the resource
  server itself (keycloak/keycloak#49910, confirmed intended and permanent by the
  maintainer), which broke the "Owner" policy: reviews became owned by the
  `api-platform-api` client instead of their author, and the front-office review
  permissions were denied. Resource creation, lookup and deletion now go through
  the Admin API as recommended upstream, and the service account is granted the
  single `manage-authorization` role this requires. The client's internal id comes
  from configuration, so no `view-clients` role is needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore(ops): update docker base images

FrankenPHP 1.12.4 -> 1.12.6, and the E2E mock server follows the node:lts tag
already used by the PWA image instead of pinning node 22.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(helm): update external-dns chart to 1.21.1

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(api): update composer dependencies

api-platform 4.3.14 -> 4.3.17, doctrine/doctrine-bundle 3.2.4 -> 3.3.1,
phpunit 13.2.1 -> 13.2.6, rector 2.4.6 -> 2.5.8, phpstan 2.2.2 -> 2.2.7,
webonyx/graphql-php 15.33.1 -> 15.37.1, zenstruck/foundry 2.10.1 -> 2.11.0.

Rector 2.5.8 adds NegatedAndsToPositiveOrsRector, which rewrites two test mocks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(api): upgrade symfony to 8.1

The lock was already half on 8.1 through transitive Dependabot bumps (#655, #656),
while composer.json still pinned 8.0.*.

Symfony 8.1 reports a more precise denormalization error for backed enums, so the
BookTest expectations are derived from BookCondition::cases() instead of the old
"must belong to a backed enumeration" wording.

Recipe updates are available for several packages, some predating this change, and
are deliberately left out to keep the upgrade reviewable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(pwa): update minor and patch dependencies

react-admin and the ra-* packages to 5.15.x, next 16.2.12, better-auth 1.6.25,
react 19.2.8, tailwindcss 4.3.3, react-hook-form 7.83.0, pg 8.22.0 and the
remaining patches.

packageManager follows pnpm 11.18.0: 11.8.0 and 11.18.0 disagree on the importer
specifier recorded for a direct dependency that an override also covers, so the
postcss range is aligned with its override to keep the lockfile stable whichever
version writes it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(pwa): upgrade typescript to 6

TypeScript 6 rejects the deprecated `target: es5` (TS5107), which the project no
longer needs: `moduleResolution` follows `bundler` as Next expects, and spreading
the `matchAll` iterator no longer requires downlevel iteration, so the
`@ts-expect-error` that covered it is dropped.

TypeScript 7 is not reachable yet: eslint-config-next bundles typescript-eslint 8,
whose peer range is `>=4.8.4 <6.1.0`, so it would break `pnpm lint`.

ESLint 10 is left out for the same class of reason: eslint-plugin-react 7.37.5,
the latest release, peers at `^9.7` and crashes under ESLint 10 with
"contextOrFilename.getFilename is not a function".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(pwa): upgrade mui to 9

MUI has no v7 -> v8 step, 9.2.0 is the next major. react-admin 5.15.1 and
ra-ui-materialui already peer on `^5.16.12 || ^6 || ^7 || ^9`.

Three breaking changes had to be followed:

- Box no longer accepts system props, they move to `sx`.
- Menu replaces `MenuListProps` with `slotProps.list`.
- The icon named `PersonOutline` is now `PersonOutlined`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(pwa): align @types/node with the node lts runtime

26.1.2 is the latest published, but the types must describe the runtime: both
images build on `node:lts`, currently Node 24. The previous `^25` matched no
version of Node in use.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(pwa): remove unused @babel/core and @popperjs/core

Neither is used by the project: there is no babel config in the repository and
Next 16 compiles through SWC, while @popperjs/core is referenced by no source
file. Both remain in the tree as transitive dependencies, so declaring them
brought nothing but a spurious `@babel/core` 7 -> 8 major to consider.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(e2e): upgrade playwright to 1.62

The CI runner image tag follows, otherwise the container would keep installing
1.58 while package.json asks for 1.62. @types/node tracks the Node LTS the
images run on, like the PWA.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The container limit was 500Mi with a 400Mi request, which leaves no headroom:
importing the demo realm, with its authorization services and JavaScript policy,
leaves the JVM just under the limit, so the pod is OOMKilled during rollout and
the post-upgrade keycloak-config-cli hook then times out waiting for a Keycloak
that never becomes ready.

Measured locally with the demo realm imported and `start --optimized`:

| version | limit 500Mi | limit 768Mi |
| --- | --- | --- |
| 26.4.7 | OOMKilled | 671Mi (87%) |
| 26.7.0 | 496Mi (99%) | 553Mi (72%), 590Mi under admin API load |

This is not a consequence of the version bump: 26.7 is in fact leaner than 26.4
here. The limit was simply too small for both, and any rollout was one allocation
away from being killed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vincentchalamon

Copy link
Copy Markdown
Contributor Author

Deploy failure: Keycloak is OOMKilled, not a chart-content problem

Both attempts of the feature deploy have the same root cause. Attempt 1 surfaced as helm upgrade --wait hitting its 5-minute deadline (client rate limiter Wait returned an error: context deadline exceeded while polling the external-dns Service). Attempt 2 got further and named it:

Error: UPGRADE FAILED: post-upgrade hooks failed: timed out waiting for the condition
pr-679-api-platform-keycloak-realm: Jobs active: 0, jobs failed: 5, jobs succeeded: 0

The kubectl get events output of that attempt shows why:

Warning  OOMKilling  node/gke-...-pool-n1-custom-...  Memory cgroup out of memory:
  Killed process (java) total-vm:1839544kB, anon-rss:508548kB ... UID:1000
Warning  OOMKilling  node/gke-...-pool-n1-custom-...  Killed process (kc.sh)
Warning  OOMKilling  node/gke-...-pool-n1-custom-...  Killed process (C2 CompilerThread)

anon-rss508 MB against a container limit of 500Mi. Keycloak is killed, never becomes ready, and the post-upgrade keycloak-config-cli hook retries five times and times out.

I was wrong about the cause

My first read was that 26.7 needs more memory than 26.4. Measured, that is not true — 26.7 is the leaner of the two. Same host, demo realm imported, start --optimized:

version limit 500Mi limit 768Mi
26.4.7 OOMKilled 671Mi (87%)
26.7.0 496Mi (99.3%) 553Mi (72%)

At 768Mi, 26.7.0 stays at 590Mi (77%) after 75 admin API calls of the kind the config-cli job makes.

So the 500Mi limit was already without headroom on 4.3 today, for either version. This rollout did not create the problem, it just landed on it. Worth knowing: the limit is a latent issue independent of this PR, so you may prefer to cherry-pick fix(helm): give keycloak enough memory to survive a rollout on its own.

Fix pushed

requests.memory 400Mi → 512Mi, limits.memory 500Mi → 768Mi, as its own commit so it can be reviewed or split out.

Two things left to your judgement, since cluster sizing is yours:

  • 768Mi leaves 26.4.7 at 87%. Fine for 26.7, tighter if you ever roll back. 1Gi would be the conservative choice.
  • No JVM ceiling is set anywhere (JAVA_OPTS_KC_HEAP unset), so the heap is sized from the cgroup limit and the visible CPU count. Pinning it would make the footprint deterministic across node shapes instead of depending on ergonomics — I left it out to keep the change minimal, happy to add it.

Ready for another deploy attempt whenever you want.

realm-demo.json is a full realm export, so it carries Keycloak's own built-in
clients. keycloak-config-cli therefore tries to update `realm-management`, which
Keycloak 26.7 forbids:

    Update client 'realm-management' in realm 'demo'
    Cannot update client 'realm-management' in realm 'demo': HTTP 403 Forbidden

The post-upgrade hook then fails on every retry and the release never completes.
26.4 allowed the update, which is why this only shows up now.

Keycloak recreates `realm-management` and its roles itself when the realm is
created, so removing them from the export changes nothing functionally: the
service account still resolves its `manage-authorization` role against the
auto-created client. The other built-in clients are left in place — only
`realm-management` is protected, and config-cli updates the rest without
complaint.

Verified against 26.7.0 with keycloak-config-cli 6.5.1-26: import succeeds on a
fresh realm and is idempotent on the second pass, which is what the
post-install/post-upgrade hook needs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vincentchalamon

Copy link
Copy Markdown
Contributor Author

Second blocker, and it is not the memory

The memory fix worked — no new OOM in the third attempt (the OOMKilling events in that log are 42–55 min old, from the earlier attempts, and still show anon-rss: 508544kB against the old 500Mi limit). The deploy got further and then failed on the post-upgrade hook again, for a different reason.

I reproduced it locally. The exact error, from keycloak-config-cli with debug logging:

DEBUG d.a.k.c.service.ClientImportService : Update client 'realm-management' in realm 'demo'
ERROR d.a.k.config.KeycloakConfigRunner   : Error during Keycloak import:
      Cannot update client 'realm-management' in realm 'demo': HTTP 403 Forbidden

realm-demo.json is a full realm export, so it carries Keycloak's own built-in clients. keycloak-config-cli, managing every client in the file, tries to update realm-management — and Keycloak 26.7 forbids modifying that client. 26.4 allowed it, which is why it surfaces only now.

Isolation

Same host, same config-cli invocation, realm imported from scratch:

Keycloak config-cli realm result
26.4.7 6.5.0-26 4.3 import OK (14.5s)
26.7.0 6.5.0-26 this PR 403
26.7.0 6.5.1-26 4.3 403
26.7.0 6.5.1-26 this PR 403

So it is neither the config-cli bump nor the manage-authorization role I added — it is Keycloak 26.7 alone.

Fix pushed

Drop the realm-management client and its roles from the export. Keycloak recreates both itself when the realm is created, so nothing changes functionally — verified after import:

  • all 9 clients present, account still has redirectUris: ["/realms/demo/account/*"] exactly as the export declared;
  • the service account still resolves manage-authorization against the auto-created client;
  • the deleted clientScopeMappings entry (account-consolemanage-account, view-groups) is a Keycloak default and comes back on its own.

I kept the removal minimal: only realm-management is protected in 26.7, and config-cli updates account, account-console, admin-cli, broker and security-admin-console without complaint, so those stay in the file.

Import against 26.7.0 now succeeds on a fresh realm (8.8s) and is idempotent on a second pass (0.8s) — which is exactly what the post-install,post-upgrade hook and the nightly cronjob need.

Ready for another deploy attempt.

26.7.0 refuses to manage any realm role literally named `admin`, in any realm:

    // RolePermissions.canManage(RoleModel), 26.7.0
    if (role.getContainer() instanceof RealmModel) {
        return root.realm().canManageRealm() && !isRealmAdminRole(role);
    }

    private boolean isRealmAdminRole(RoleModel role) {
        return role.getContainer() instanceof RealmModel
            && List.of(AdminRoles.ADMIN, AdminRoles.CREATE_REALM).contains(role.getName());
    }

The check is purely name-based and ignores which realm the role belongs to, so the
demo realm's own `admin` role becomes unmanageable: `PUT /admin/realms/demo/roles/admin`
answers 403 even to a full master-realm admin, while `roles/user` and a freshly
created `admin2` update fine. keycloak-config-cli hits it as soon as that role has
drifted from the file, so the post-upgrade hook fails on every retry.

`isRealmAdminRole` exists only in 26.7.0 — it is absent from 26.6.4, 26.5.7 and
26.4.7 — and the advisory only requires >= 26.6.2. 26.6 therefore closes
CVE-2026-7504 without the regression, and keeps every other adjustment relevant:
26.6 already requires the `scripts` feature, and its Protection API already forces
the resource owner to the resource server.

Verified on 26.6.4: realm imported at startup, keycloak-config-cli succeeds and is
idempotent on a second pass, review resources are owned by their author again, and
the E2E @Write suite passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vincentchalamon vincentchalamon changed the title fix(helm): upgrade keycloak to 26.7 fix(helm): upgrade keycloak to 26.6 Jul 30, 2026
@vincentchalamon

Copy link
Copy Markdown
Contributor Author

Third blocker found, with cluster access — and it settles the version choice

Thanks for the gcloud auth login. The cluster told the rest of the story.

The memory fix worked. The Keycloak pod has been 2/2 Running for 94 minutes with 0 restarts, and its spec carries limits: {memory: 768Mi} / requests: {memory: 512Mi}. The OOMKilling events in the last deploy log are 42–55 min old, from the earlier attempts, still showing anon-rss: 508544kB against the old limit.

The realm-management fix worked too. The deployed ConfigMap no longer contains it.

But the hook still failed, on a third, different cause.

What the pod logs showed

DEBUG RoleImportService : Update realm-level role 'uma_authorization' in realm 'demo'
DEBUG RoleImportService : Update realm-level role 'offline_access' in realm 'demo'
DEBUG RoleImportService : Update realm-level role 'admin' in realm 'demo'
ERROR KeycloakConfigRunner : Error during Keycloak import: HTTP 403 Forbidden
      at RoleImportService.updateRoleIfNeeded(RoleImportService.java:223)

Reproduced directly against the cluster's Keycloak, with a full master-realm admin token:

request result
GET /admin/realms/demo/roles/admin 200
PUT /admin/realms/demo/roles/admin (same body) 403
PUT /admin/realms/demo/roles/user 204
PUT /admin/realms/demo/roles/offline_access 204
create admin2, then PUT .../roles/admin2 204

So it is not the name pattern, not the composites, and not admin permissions v2 (adminPermissionsEnabled: false). It is this, in 26.7.0:

// RolePermissions.canManage(RoleModel)
if (role.getContainer() instanceof RealmModel) {
    return root.realm().canManageRealm() && !isRealmAdminRole(role);
}

private boolean isRealmAdminRole(RoleModel role) {
    return role.getContainer() instanceof RealmModel
        && List.of(AdminRoles.ADMIN, AdminRoles.CREATE_REALM).contains(role.getName());
}

The check is purely name-based and ignores which realm the role belongs to. The demo realm defines its own realm role named admin, so it became unmanageable — admin2 is fine, admin is not. keycloak-config-cli trips on it as soon as that role has drifted from the file, which is why it reproduced in the cluster and not on a locally recreated realm.

Fix: pin 26.6, not 26.7

isRealmAdminRole exists only in 26.7.0 — absent from 26.6.4, 26.5.7 and 26.4.7 — and the advisory only requires >= 26.6.2. So 26.6 closes CVE-2026-7504 without the regression, and keeps every other adjustment in this PR relevant: 26.6 already requires the scripts feature, and its Protection API already forces the resource owner to the resource server.

The PR title is updated accordingly.

Verified on 26.6.4 locally: realm imported at startup, Preview features enabled: scripts:v1, keycloak-config-cli succeeds (9.7s) and is idempotent on a second pass (0.8s), review resources are owned by john.doe@example.com again, memory sits at 584Mi / 768Mi (76%), and E2E @write is 8 passed / 3 flaky / 0 failed.

Worth reporting upstream

isRealmAdminRole looks like a genuine bug: the intent is clearly to protect the master realm's admin role, but the check applies to every realm, so any realm that happens to define a role named admin or create-realm can no longer be managed through the Admin API. Happy to open an issue on keycloak/keycloak if you want — say the word and I will, with the reproduction above.

I left the cluster clean: the debug pods and jobs I created in pr-679 are deleted.

@vincentchalamon

Copy link
Copy Markdown
Contributor Author

Deploy is green

Deploy / Deploy passed in 6m30s, and the post-deploy Check (k6) too.

Cluster state on pr-679:

NAME                                 STATUS     COMPLETIONS   DURATION
pr-679-api-platform-keycloak-realm   Complete   1/1           48s
pr-679-api-platform-fixtures         Complete   1/1           82s

Keycloak reports 26.6.4 on JVM with Preview features enabled: scripts:v1, and the pod carries limits: {memory: 768Mi} / requests: {cpu: 50m, memory: 512Mi}.

The deployed environment answers:

path status
/ 200
/books 200
/docs 200
/oidc/realms/demo/.well-known/openid-configuration 200

And, as expected on this PR, the advisory is still open here:

$ curl -o /dev/null -w '%{http_code}' \
    'https://pr-679-demo.api-platform.com/oidc/realms/demo/protocol/openid-connect/auth?client_id=api-platform-pwa&redirect_uri=https://attacker.evil.example/callback&response_type=code&scope=openid'
200

A live confirmation that the version bump alone does not remediate GHSA-2rmv-4743-38c6 — #676 is what closes it.

@vincentchalamon
vincentchalamon merged commit a8a513a into 4.3 Jul 30, 2026
7 checks passed
@vincentchalamon
vincentchalamon deleted the fix/keycloak-upgrade branch July 30, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deploy Deploys Pull Request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant