Implement lock-free versioned (revisionId) and Set-based Redis caching for /tasks

[eyebrowsoffire]

To eliminate read bottlenecks against the Firestore /tasks (kTaskCollectionId = 'tasks') collection without distributed locking (tryLock), this change introduces an optimistic, multi-tier caching layer across Cocoon (Task, FirestoreQueries, and CacheService):

1. Lock-Free Versioned (revisionId) Payload Caching (tasks subcache):

   • Every Task document contains a monotonically increasing revisionId integer.
   • CacheService.insertVersioned(subcacheName, entries) executes an atomic check-and-set via Redis Lua script (EVAL) verifying entry.revisionId > cachedRevisionId before updating payloads.
   • Chunked batching (batchSize = 20) guarantees <0.1ms Lua execution without starving concurrent readers (MGET).

2. Native Set-Based Commit Task Indexing (tasks_by_commit_ids):

   • Replaces monolithic JSON array strings with native Redis Sets (SMEMBERS, SADD) via getSet, updateSet, and addToSetIfExists.
   • Leverages two domain invariants:
     1. Immutable commitSha: Task mutations (updateCacheForTaskMutations) never change a commit's task list membership and do not touch or invalidate tasks_by_commit_ids.
     2. Monotonically Increasing Set: Task attempts only grow over time, so unioning IDs via SADD (updateCacheForCreatedTasks) safely converges without lock coordination.

3. Partial Cache Recovery (_queryTasksByCommitCached):

   • When reading commit tasks, _queryTasksByCommitCached retrieves cached task IDs (SMEMBERS) and performs a batch lookup (MGET).
   • If individual payloads are expired or missing (missingDocIds), _queryTasksByCommitCached selectively queries getDocument only for missing entries and merges them with foundTasks, eliminating redundant full-commit queries.

4. Modular & Explicit Cache Handlers (FirestoreQueries):

   • _cacheTaskDocuments(tasks): Reusable helper for versioned Task payload insertions.
   • _fetchAndCacheCommitTasks(commitSha): Reusable slow-path query helper shared across read-miss and task-creation-miss paths.
   • updateCacheForCreatedTasks(tasks) and updateCacheForTaskMutations(writes): Explicit domain handlers replacing generic ad-hoc cache invalidations.
   • _subcacheRecentTasksIds and _queryRecentTasksByNameCached removed in favor of _cacheTaskDocuments warming on query execution (~65 lines simplified).

[flutter-dashboard]

This pull request is not mergeable in its current state, likely because of a merge conflict. Pre-submit CI jobs were not triggered. Pushing a new commit to this branch that resolves the issue will result in pre-submit jobs being scheduled.

[gemini-code-assist]

Code Review

This pull request introduces a lock-free, versioned, and set-based caching strategy for Firestore tasks in Cocoon to resolve read bottlenecks. It adds optimistic concurrency tracking via a revisionId on tasks, introduces batch and set operations to the CacheService (with Redis and in-memory implementations), and integrates caching into FirestoreQueries. The review feedback highlights critical issues: a memory leak in Redis where revision IDs are stored in a non-expiring shared hash, incorrect caching behavior when tasks are deleted, and a performance bottleneck caused by fetching missing tasks sequentially in a loop instead of in parallel.