Problem Description
I'm using EntityFrameworkCore.Sqlite.Concurrency in an application with mixed write workloads. For example, a background service is performing a large BulkInsertOptimizedAsync (low priority, can tolerate latency), while user-facing API requests need to persist critical data with minimal delay (high priority).
Currently, the SqliteWriteQueue uses a single FIFO Channel [1]. All writes — regardless of their urgency — are serialized in the order they arrive. This means a backlog of low-priority bulk writes can stall latency-sensitive writes, causing poor user experience. There is no way to say "drain this write before those other ones."
The library already has a WriteQueueCapacity option for back-pressure [2] and a SqliteConcurrencyOptions configuration model [2], making a priority extension architecturally natural.
Proposed Solution
Introduce priority levels (e.g., Low, Normal, High) on the write queue, so that high-priority writes are always dequeued and executed before normal and low-priority writes.
API sketch:
Low =0,
Normal =1, // default — preserves backward compatibility
High =2
// Every write method gets an optional WritePriority parameter:
// SaveChangesAsync
await db.SaveChangesAsync(priority: WritePriority.High);
// SaveChangesSerializedAsync
await db.SaveChangesSerializedAsync(
ct => db.SaveChangesAsync(ct),
priority: WritePriority.High);
// BulkInsertOptimizedAsync
await db.BulkInsertOptimizedAsync(records, priority: WritePriority.Low);
// ExecuteWithRetryAsync
await db.ExecuteWithRetryAsync(async ctx => { ... }, priority: WritePriority.High);
// ThreadSafeSqliteContext.ExecuteWriteAsync
await context.ExecuteWriteAsync(async ctx => { ... },
priority: WritePriority.High);
Implementation approach:
- Internally, use multiple
Channel instances (one per priority level) or a single Channel paired with a priority-aware dequeue loop that drains High → Normal → Low in order [1]. The existing single background writer loop in SqliteWriteQueue [2] would be extended.
Alternatives Considered
- Separate database files: Split high-priority and low-priority data into different SQLite files. This works but adds significant complexity — cross-database joins, multiple connection strings, and manual coordination. Not practical for most use cases.
- Use a separate
IDbContextFactory per priority: Register multiple factories with different SqliteConcurrencyOptions. However, the write queue is per-connection-string (i.e., per database file), so this wouldn't actually create separate queues — it would need the library to support multiple queues per database, which is essentially this feature request.
- Custom
Channel implementation outside the library: Write my own priority queue wrapper that calls into the library's APIs. This defeats the purpose of using the library for concurrency and would likely conflict with the interceptor/write queue internals.
- Use
System.Threading.Channels with a custom IComparer: Unfortunately, Channel does not natively support priority ordering, so this requires the library to implement it internally.
Additional Context
No response
Problem Description
I'm using
EntityFrameworkCore.Sqlite.Concurrencyin an application with mixed write workloads. For example, a background service is performing a largeBulkInsertOptimizedAsync(low priority, can tolerate latency), while user-facing API requests need to persist critical data with minimal delay (high priority).Currently, the
SqliteWriteQueueuses a single FIFOChannel[1]. All writes — regardless of their urgency — are serialized in the order they arrive. This means a backlog of low-priority bulk writes can stall latency-sensitive writes, causing poor user experience. There is no way to say "drain this write before those other ones."The library already has a
WriteQueueCapacityoption for back-pressure [2] and aSqliteConcurrencyOptionsconfiguration model [2], making a priority extension architecturally natural.Proposed Solution
Introduce priority levels (e.g.,
Low,Normal,High) on the write queue, so that high-priority writes are always dequeued and executed before normal and low-priority writes.API sketch:
Implementation approach:
Channelinstances (one per priority level) or a singleChannelpaired with a priority-aware dequeue loop that drainsHigh→Normal→Lowin order [1]. The existing single background writer loop inSqliteWriteQueue[2] would be extended.Alternatives Considered
IDbContextFactoryper priority: Register multiple factories with differentSqliteConcurrencyOptions. However, the write queue is per-connection-string (i.e., per database file), so this wouldn't actually create separate queues — it would need the library to support multiple queues per database, which is essentially this feature request.Channelimplementation outside the library: Write my own priority queue wrapper that calls into the library's APIs. This defeats the purpose of using the library for concurrency and would likely conflict with the interceptor/write queue internals.System.Threading.Channelswith a customIComparer: Unfortunately,Channeldoes not natively support priority ordering, so this requires the library to implement it internally.Additional Context
No response