Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
name: Integration tests (${{ inputs.java-version }}, ${{ inputs.kube-version }}, ${{ inputs.http-client }})
runs-on: ubuntu-latest
continue-on-error: ${{ inputs.experimental }}
timeout-minutes: 40
timeout-minutes: 120
steps:
- uses: actions/checkout@v7
with:
Expand Down
37 changes: 26 additions & 11 deletions docs/content/en/blog/news/read-after-write-consistency.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> contex
}
```

In addition to that, the framework will automatically filter events for your own updates,
so they don't trigger the reconciliation again.
In addition to that, the framework will provide facilities to filter out
events for own updates so they don't trigger the reconciliation again.

{{% alert color=success %}}
**This should significantly simplify controller development, and will make reconciliation
Expand Down Expand Up @@ -180,12 +180,6 @@ From this point the idea of the algorithm is very simple:
the one in the TRC. If yes, evict the resource from the TRC.
3. When the controller reads a resource from cache, it checks the TRC first, then falls back to the Informer's cache.

The actual filtering of events for our own writes is more nuanced than a simple
"evict on RV ≥ TRC version" rule — it is driven by a per-resource state machine
that tracks in-flight writes and the events received around them. See
[Filtering events for our own updates](#filtering-events-for-our-own-updates) below.


```mermaid
sequenceDiagram
box rgba(50,108,229,0.1)
Expand Down Expand Up @@ -226,10 +220,31 @@ sequenceDiagram
When we update a resource, the informer will eventually propagate an event that would trigger a reconciliation.
In most cases, however, this is not desirable. Since we already have the up-to-date resource at that point,
we want to be notified only when the change originates outside our reconciler.
Therefore, in addition to caching the resource, we filter out events caused by our own updates.
Therefore, in addition to caching the resource, provide utilities to so you can optimize the update/patch
operations to filter out those events.

See [ResourceOperations](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ResourceOperations.java#L49)
for details.

```java
public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> context) {

ConfigMap managedConfigMap = prepareConfigMap(webPage);

// resource operation in this case will resource only if
// it does not match the actual, and will filter our the related event
context.resourceOperations().serverSideApply(managedConfigMap);

// UpdateControl.patchStatus would only cache the resource to
// filter out events too you have to use resourceOperations.
context.resourceOperations().serverSideApplyPrimaryStatus(alterStatusObject(webPage));

return UpdateControl.noUpdate();
}
```

Note that the implementation of this is relatively complex: while performing the update, we record all the
events received in the meantime and decide whether to propagate them further once the update request completes.
Note that the implementation of this is relatively complex and has some caveats:
while performing the update, we record all the events received in the meantime and decide whether to propagate them further once the update request completes.

This way, we significantly reduce the number of reconciliations, making the whole process much more efficient.

Expand Down
16 changes: 16 additions & 0 deletions docs/content/en/docs/documentation/error-handling-retries.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,22 @@ Retry can be skipped in cases of unrecoverable errors:
ErrorStatusUpdateControl.patchStatus(customResource).withNoRetry();
```

When `updateErrorStatus` returns any `ErrorStatusUpdateControl` other than
`ErrorStatusUpdateControl.defaultErrorProcessing()`, the framework considers the error handled by
the reconciler and, while retry attempts remain, no longer logs the "Uncaught error during event
processing" warning; the error is logged on `DEBUG` level instead. This lets a reconciler keep
retrying an expected, recoverable condition without producing a continuous stream of `WARN`
messages, while it remains free to log the error at whatever level it deems appropriate inside
`updateErrorStatus`.

This log downgrade applies only when retry is actually taking place, i.e. a retry is configured and
the returned control does not disable it. Controls that explicitly disable retry — `withNoRetry()`
and `rescheduleAfter()` — cancel the native retry and its `@GradualRetry` backoff, so nothing is
retried in those cases. Likewise, on the last retry attempt (or when no retry is configured) the
failure is final, so the framework keeps its higher-severity logging to avoid hiding a
non-recoverable error. Returning `ErrorStatusUpdateControl.defaultErrorProcessing()` preserves the
default behavior, including the warning.

### Correctness and Automatic Retries

While it is possible to deactivate automatic retries, this is not desirable unless there is a particular reason.
Expand Down
50 changes: 46 additions & 4 deletions docs/content/en/docs/documentation/reconciler.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,42 @@ supports stronger guarantees, both for primary and secondary resources. If this
they would otherwise look like own echoes, since the relist may have
hidden events.

#### Requesting event filtering and its correctness requirements

Own-event filtering is only safe if the framework can tell an own write apart from a concurrent
third-party write. This requires **either**:

- a *matcher* to be provided (so a filtered event can be confirmed to reflect the desired state we
just wrote), **or**
- the update to be done using *optimistic locking* (a resource version set on the written resource),
so a conflicting concurrent change is rejected by the API server rather than silently swallowed.

`ResourceOperations` methods accept an `Options` argument to select the behavior; each maps to a
`Mode`:

- `Options.matchAndFilter(matcher)` / `Options.matchAndFilterWithDefaultMatcher(updateType)` — compare
the desired state to the actual (cached) state; if they already match, skip the write entirely,
otherwise write and filter the own event. This is the **default** for the server-side apply and
patch (JSON Patch / JSON Merge Patch) methods, and generally the most efficient option: it filters
the own event *and* avoids a request to the API server when nothing changed. Default matchers are
provided for every operation type, but they are heuristics — a workflow relying on them should be
tested against the concrete resources it manages, or a custom matcher supplied.
- `Options.filterWithOptimisticLocking()` — filter the own event; the write must use optimistic
locking (a resource version set on the written resource), otherwise an `IllegalArgumentException`
is thrown. Requiring optimistic locking guarantees a concurrent third-party change is rejected by
the API server rather than being silently filtered out. The overloads taking a matcher /
`updateType` additionally skip the write when the desired state already matches. This match +
optimistic locking combination is the **default** for the PUT `update` / `updatePrimary` /
`updatePrimaryStatus` methods (a full PUT should not clobber a concurrent change).
- `Options.cacheOnly()` — only cache the response (read-cache-after-write consistency), no own-event
filtering.
- `Options.forceFilterEvents()` — always filter, regardless of optimistic locking. Only safe when
correctness is otherwise guaranteed (mostly for internal usage); a concurrent external update in
the filter window may otherwise be missed until the next resync.

See the [`ResourceOperations`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ResourceOperations.java)
and `ResourceOperations.Options` documentation for details.


In order to benefit from these stronger guarantees, use [`ResourceOperations`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ResourceOperations.java)
from the context of the reconciliation:
Expand All @@ -208,20 +244,26 @@ public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> contex
var upToDateResource = context.getSecondaryResource(ConfigMap.class);

makeStatusChanges(webPage);

// built in update methods by default use this feature
// patches the status and caches the response (does not filter the own event by default, see below)
return UpdateControl.patchStatus(webPage);
}
```

`UpdateControl` and `ErrorStatusUpdateControl` by default use this functionality, but you can also update your primary resource at any time during the reconciliation using `ResourceOperations`:
{{% alert title="UpdateControl and event filtering" %}}
Since v5.5, returning an `UpdateControl` (or `ErrorStatusUpdateControl`) updates the resource and
keeps the cache read-after-write consistent, but by default it **no longer filters the own event** —
the resulting update may an additional reconciliation (which should be idempotent). To also filter the own
event, perform the update through `ResourceOperations` instead and return `UpdateControl.noUpdate()`.
{{% /alert %}}

You can update your primary resource at any time during the reconciliation using `ResourceOperations`:

```java

public UpdateControl<WebPage> reconcile(WebPage webPage, Context<WebPage> context) {

makeStatusChanges(webPage);
// this is equivalent to UpdateControl.patchStatus(webpage)
// updates the status, filters the own event and skips the write if nothing changed
context.resourceOperations().serverSideApplyPrimaryStatus(webPage);
return UpdateControl.noUpdate();
}
Expand Down
69 changes: 69 additions & 0 deletions docs/content/en/docs/migration/v5-5-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: Migrating from v5.4 to v5.5
description: Migrating from v5.4 to v5.5
---

## No breaking API changes

v5.5 does **not** contain breaking API changes: existing code compiles and continues to work without
modification. There is, however, one **behavioral** change around own-event filtering that you should
be aware of (see below).

## `UpdateControl` no longer filters own events by default

In previous versions, updating the primary resource (or its status) by returning an `UpdateControl`
from the reconciler would filter out the event caused by that own update, so the write did not
trigger an additional reconciliation. Unfortunately, in some edge cases this could lead an
incorrect behavior, thus filtering out events which should be propagated.

More precisely in case where for example the spec part of the primary resource was updated, while
controller patched the status, but that patch status was a no-op operation. Note that
these no-op operations are causing issue, which should not be done in first place.
From 5.5 we provide methods in `ResourceOperations` which allow only operations which are
correct.

From v5.5, `UpdateControl` **no longer filters these own events by default**. Returning an
`UpdateControl` still updates the resource and keeps the cache read-after-write consistent, but the
resulting update event is now delivered like any other event, which may cause an additional
reconciliation.

This is safe (reconciliations are expected to be idempotent), but if you relied on the previous
filtering — for example to avoid an extra reconciliation after a status update — use
[`ResourceOperations`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ResourceOperations.java)
directly, which does filter own events.

### Using `ResourceOperations` instead

`ResourceOperations` is available from the reconciliation `Context` via
`context.resourceOperations()`. Instead of returning an `UpdateControl`, perform the update through
it and return `UpdateControl.noUpdate()`:

```java
// before (v5.4): the own status update event was filtered
@Override
public UpdateControl<MyResource> reconcile(MyResource resource, Context<MyResource> context) {
resource.setStatus(new MyStatus().setReady(true));
return UpdateControl.patchStatus(resource);
}
```

```java
// after (v5.5): filter the own event explicitly via ResourceOperations
@Override
public UpdateControl<MyResource> reconcile(MyResource resource, Context<MyResource> context) {
resource.setStatus(new MyStatus().setReady(true));
context.resourceOperations().serverSideApplyPrimaryStatus(resource);
return UpdateControl.noUpdate();
}
```

`ResourceOperations` covers every update/patch strategy (server-side apply, update, JSON Patch, JSON
Merge Patch) for both the whole resource and the status subresource, as well as their primary
variants. By default these operations match the desired state against the actual (cached) state
before writing and filter the own event, so they only issue a request to the Kubernetes API server
when something actually changed.

> **Note**: Safe own-event filtering requires either a matcher (used by default) or the update to be
> done with optimistic locking. See the `ResourceOperations` and `ResourceOperations.Options`
> documentation for the available modes, the correctness requirements, and the caveats of the
> default matchers.
Loading
Loading