Skip to content

Serialize runtime SQLite writes#16

Open
mjc wants to merge 4 commits intomainfrom
t3code/serialize-sqlite-writes
Open

Serialize runtime SQLite writes#16
mjc wants to merge 4 commits intomainfrom
t3code/serialize-sqlite-writes

Conversation

@mjc
Copy link
Copy Markdown
Owner

@mjc mjc commented Apr 17, 2026

Summary

  • add a single in-process DbWriter to serialize runtime SQLite mutations
  • route runtime write-heavy paths through the writer and simplify webhook processing
  • reduce pool size and busy timeout defaults, and add focused DbWriter tests

Validation

  • nix develop -c mix compile
  • nix develop -c mix test test/reencodarr/db_writer_test.exs test/reencodarr/media/video_upsert_test.exs test/reencodarr/core/retry_test.exs

Copilot AI review requested due to automatic review settings April 17, 2026 18:01
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a global in-process database writer to serialize SQLite write operations at runtime, reducing lock contention and simplifying previously write-serialization logic in webhook processing.

Changes:

  • Add Reencodarr.DbWriter (GenServer) to serialize runtime SQLite mutations and apply retry behavior centrally.
  • Route multiple write-heavy code paths (Media, VideoUpsert, VideoStateMachine, Services, VideoFailure, webhooks) through DbWriter.
  • Adjust SQLite/pool tuning (smaller repo pool, reduced busy timeout) and add focused DbWriter tests.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
lib/reencodarr/db_writer.ex New GenServer-based writer that serializes write execution and wraps writes with retry logic.
test/reencodarr/db_writer_test.exs Adds tests covering inline behavior in test env, nesting, enqueue, and transaction semantics.
lib/reencodarr_web/webhook_processor.ex Simplifies webhook processing to enqueue work onto the global DbWriter.
lib/reencodarr/media.ex Routes many write paths through DbWriter and adds labeled wrappers for writes/transactions.
lib/reencodarr/media/video_upsert.ex Wraps upsert/batch_upsert in DbWriter and removes per-transaction retry helper usage.
lib/reencodarr/media/video_state_machine.ex Routes Repo updates through DbWriter instead of direct retry helper.
lib/reencodarr/media/video_queries.ex Wraps “claim for analysis” state transition in DbWriter transaction.
lib/reencodarr/media/video_failure.ex Routes failure insert/update through DbWriter.
lib/reencodarr/services.ex Routes config create/update/delete through DbWriter.
lib/reencodarr/sync.ex Removes explicit Repo transactions around some upsert flows (now handled elsewhere).
lib/reencodarr/core/retry.ex Tweaks transient SQLite retry log message format.
lib/reencodarr/application.ex Adds Reencodarr.DbWriter to supervision tree.
config/config.exs Reduces SQLite busy timeout default now that writes are serialized.
config/dev.exs / config/runtime.exs Reduces repo pool size defaults to reflect serialized writes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/reencodarr/db_writer.ex Outdated
Comment thread lib/reencodarr/db_writer.ex
Comment thread lib/reencodarr/media/video_upsert.ex Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (1)

lib/reencodarr_web/webhook_processor.ex:12

  • WebhookProcessor is still started under supervision as a GenServer, but it no longer queues/handles any messages (it just delegates to DbWriter.enqueue/2). This leaves an extra always-idle process and can be misleading to future maintainers. Consider converting this to a plain module (or keep the process only if you need it for compatibility/health checks) and update the supervision tree accordingly.
  use GenServer

  alias Reencodarr.DbWriter

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/reencodarr/db_writer_test.exs Outdated
Comment thread test/reencodarr/db_writer_test.exs Outdated
Comment thread lib/reencodarr/media/video_upsert.ex Outdated
Comment thread lib/reencodarr/media/video_upsert.ex Outdated
Comment thread lib/reencodarr/sync.ex
Comment thread lib/reencodarr/media/video_queries.ex Outdated
@mjc mjc force-pushed the t3code/serialize-sqlite-writes branch from 0215401 to 1a4b86e Compare April 18, 2026 02:37
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

lib/reencodarr_web/webhook_processor.ex:12

  • WebhookProcessor no longer handles any GenServer messages (it just forwards to DbWriter.enqueue/2), but it is still started as a supervised GenServer and keeps an idle process around. Consider converting this to a plain module (remove use GenServer, start_link/1, and the child entry) or, if it must remain a child for compatibility, document why it needs to run and/or add minimal callbacks to justify the process.
  use GenServer

  alias Reencodarr.DbWriter

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +60
test "run uses writer_timeout without consuming Repo timeout options" do
previous_env = Application.get_env(:reencodarr, :env)
Application.put_env(:reencodarr, :env, :dev)
on_exit(fn -> Application.put_env(:reencodarr, :env, previous_env) end)
test_pid = self()
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Application.put_env(:reencodarr, :env, :dev) mutates global application config and can leak into other tests running concurrently (this repo has many async: true tests). This can make the suite flaky. Prefer a test-only switch that forces non-inline mode without touching global env (e.g., an option like DbWriter.run(..., inline?: false) / mode: :writer) or isolate the behavior behind dedicated test config instead of runtime mutation.

Copilot uses AI. Check for mistakes.
Comment on lines +93 to +96
test "enqueue logs async failures with the writer label" do
previous_env = Application.get_env(:reencodarr, :env)
Application.put_env(:reencodarr, :env, :dev)
on_exit(fn -> Application.put_env(:reencodarr, :env, previous_env) end)
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test also changes :reencodarr, :env globally. Even with async: false for this module, ExUnit can still run other async tests in parallel, so this can cause cross-test interference. Consider avoiding Application.put_env/3 here by injecting a writer/inline mode override into DbWriter.enqueue/2 for tests.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants