CAMEL-24082: camel-aws-cloudtrail - fix consumer event loss (static cursor, no pagination, window skip)

[oscerd]

Description

Fixes CAMEL-24082.

The camel-aws-cloudtrail consumer could silently drop CloudTrail events:

1. Shared static cursor — lastTime was private static Instant, shared by every cloudtrail consumer in the JVM, so two routes (different eventSource/region) clobbered each other's position, and the value leaked across restarts.
2. No pagination — poll() issued a single lookupEvents(...) call and never followed response.nextToken(). With the default maxResults = 1, at most one event per poll was retrieved and the rest of the window was discarded.
3. Time-window skip — the cursor advanced to the newest event time and the next request used startTime(lastTime.plusMillis(1000)), so events within that 1s window (or sharing the newest timestamp) were skipped forever, with no event-id de-duplication.

Fix

• lastTime is now a per-consumer instance field, initialized in doStart() to the consumer start time so it tails from startup instead of replaying history.
• poll() paginates through every page (nextToken) before delivering.
• The cursor advances using the newest event time as an inclusive startTime, and boundary events are de-duplicated by eventId (bounded to the newest-timestamp instant) — replacing the lossy +1000ms skip.
• Default maxResults raised 1 → 50 (the AWS maximum). With pagination this is a page-size hint, not a per-poll cap.

Tests

The module previously shipped no unit tests. Adds CloudtrailConsumerTest (with a test-scoped mockito-junit-jupiter dependency, consistent with the sibling aws2-* modules):

• pollDrainsEveryPage — two pages, asserts all events delivered and lookupEvents called per page.
• pollDoesNotRedeliverBoundaryEventsAcrossPolls — asserts the inclusive-startTime boundary event is not re-delivered on the next poll.

Docs

Behavior + default change documented in the 4.22 upgrade guide. Regenerated component/catalog/DSL metadata included.

Backport

Same defect on camel-4.18.x and camel-4.14.x; will be backported after merge (fixVersions 4.22.0 / 4.18.4 / 4.14.9). Per project policy the upgrade-guide entry stays on main only.

---

Claude Code on behalf of oscerd

[gnodet]

Thorough and well-implemented fix for three genuine, long-standing bugs. The diagnosis is accurate, the deduplication logic is sound, and the tests cover the critical scenarios.

What works well:

• Static cursor → instance field: correct fix, verified against the sibling CloudTrailReloadTriggerTask pattern.
• Pagination loop: properly drains all pages via nextToken before processing.
• Event-id dedup: naturally bounded to events sharing the newest timestamp. The inclusive startTime + id-based filtering is a much better approach than the lossy +1000ms skip.
• doStart() guard: if (lastTime == null) correctly means first-start tails from "now", but stop/restart resumes from the prior position — no event gap.

Suggestions (non-blocking):

1. The processBatch implementation (unchanged by this PR) doesn't set BATCH_INDEX/BATCH_SIZE/BATCH_COMPLETE or call isBatchAllowed(), which is inconsistent with the ScheduledBatchPollingConsumer contract (cf. Sqs2Consumer). This is a pre-existing issue, not introduced here, but could be a follow-up improvement.
2. Consider adding an @AfterEach teardown in the test to stop the context/component/consumer — prevents leaked threads if more tests are added later.
3. An empty-poll edge case test would round out the coverage nicely.

Checklist:

• Tests included (pagination drain + boundary dedup)
• Upgrade guide entry documents both behavior changes (maxResults default, startup tail-from-now)
• Generated files regenerated (catalog, DSL builders)
• Backward compat — old default of maxResults=1 was functionally broken; change to 50 is documented
• No security concerns
• CI not yet reported

Reviewed with Claude Code on behalf of gnodet. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying.

[davsclaus]

Solid fix for three real bugs in the CloudTrail consumer (static cursor, missing pagination, lossy time-window skip). The fix is correct and well-tested:

• Per-consumer instance cursor with doStart() initialization
• Pagination via nextToken drains all pages
• Bounded dedup by event ID at the boundary timestamp
• Two focused unit tests covering pagination drain and cross-poll boundary dedup
• Behavior changes documented in the 4.22 upgrade guide
• Generated files regenerated

No blocking issues found.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

[apupier]

conflict to resolve with upgrade guide