Replica: Make drop_table idempotent so concurrent retention runs don't crash (#3595) - #82
Open
lucaforni wants to merge 16 commits into
Open
Replica: Make drop_table idempotent so concurrent retention runs don't crash (#3595)#82lucaforni wants to merge 16 commits into
lucaforni wants to merge 16 commits into
Conversation
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The app-wide CSP already blocks inline script execution, but the HTML preview iframe for a stored email was same-origin and un-sandboxed, and the html_raw response had no per-action hardening. Add a sandbox on the iframe and tighten the CSP on html_raw to script-src 'none' with nosniff and no-referrer so the preview has defence in depth against a future CSP bypass or regression. Relates to GHSA-f6g9-8555-cw28.
The /img/<server>/<message> endpoint accepted a src=<url> query parameter and proxied the body of that URL back to the caller. Nothing in the codebase ever produces a src= parameter — the parser only inserts a plain tracking pixel and rewrites href links — so this branch is dead code inherited from the original AppMail import. Drop the src branch: requests with src now return 400. The no-src path that serves the tracking pixel and records loads is unchanged, and a spec covers both the pixel-serving path and the removed branch.
The endpoint and domain option helpers interpolated model attributes straight into an HTML string before marking the whole buffer html_safe. Wrap the interpolations in h() so untrusted attributes can't break out of the surrounding tag. Also stop the helpers glob in rails_helper from eagerly requiring _spec.rb files so helper specs can live under spec/helpers/, and add a small application helper spec covering the escape behaviour.
url_with_return_to only checked that return_to started with a forward slash, which also allowed protocol-relative values like //host and /\host. Rails 7.1 already refuses to follow those via redirect_to, so the user just saw a 500. Reject the same shapes in the helper instead so we fall back to the default URL cleanly. Adds a sessions request spec covering the rejected shapes plus the happy-path relative redirect.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…rfpg-3xr5) The Legacy API message lookup endpoints parsed the request body as JSON and passed the `id` parameter straight through to the message database. A JSON object supplied for `id` arrived as a Ruby Hash and was used as a raw set of SQL `WHERE` conditions. `hash_to_sql` interpolated each Hash key directly inside backtick identifier quoting while escaping only the value, so a key containing a backtick could break out of the identifier and inject arbitrary SQL into the SELECT (blind, time-based) against the message database. Fixes: - Escape all identifiers (columns, tables, database names) through a new `escape_identifier` helper that wraps in backticks and doubles embedded backticks. Applied across hash_to_sql, select, insert, insert_multi, update and delete so no caller can inject via an identifier. - Validate the Legacy API `id` parameter at the controller boundary: reject any non-scalar value before it reaches the database and coerce it to an integer. Internal Hash-based lookups (e.g. tracking middleware) are unaffected. Adds regression tests at the unit (hash_to_sql / escape_identifier) and request (legacy messages/deliveries) levels.
Webhook and HTTP message endpoint deliveries both flow through Postal::HTTP, which parsed the user-supplied URL and connected to its host with no address validation. An authenticated user could point a webhook or endpoint at a private, loopback or link-local address (e.g. 127.0.0.1, 169.254.169.254 cloud metadata, RFC1918 hosts) and make the server issue requests into its own internal network. Add Postal::HTTP::AddressGuard, which resolves the destination host and rejects private/loopback/link-local/reserved/multicast IPv4 and IPv6 addresses, then pins the connection to the validated address so it cannot be redirected via a DNS-rebinding race. Administrators can permit specific destinations via the new postal.allowed_request_destinations config option (hostnames or IP/CIDR ranges). Address selection only uses families this server can actually reach so we do not pin to an IPv6 address on a host without IPv6 connectivity; IPv4 is preferred for predictability. HTTPEndpoint now validates that its URL is a well-formed HTTP(S) URL with a host.
The spec relied on the test machine having real IPv6 connectivity, which GitHub Actions runners do not have.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ProcessMessageRetentionScheduledTask's lock is stealable after a few minutes, so two workers can call Provisioner#drop_table for the same table. The bare DROP TABLE then raises Mysql2::Error (Unknown table) on the second worker and crashes the task (postalserver#3595). Use DROP TABLE IF EXISTS, mirroring the concurrent-run tolerance create_raw_table already has via its rescue. Adds a spec asserting drop_table does not raise when the table is gone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Questa PR replica la PR originale: postalserver#3616
Autore originale: @youdie006
Branch originale:
fix/drop-table-if-existsRepository originale: youdie006/postal
Problem
ProcessMessageRetentionScheduledTaskcrashes withMysql2::Error: Unknown tablewhen two workers run retention concurrently (postalserver#3595). The task's lock is stealable after a few minutes, so two workers can callProvisioner#drop_tablefor the sameraw-<date>table; the second one runs a bareDROP TABLEon an already-dropped table and raises.In
lib/postal/message_db/provisioner.rb,drop_tablewas the one un-guarded destructive query. Its siblingcreate_raw_tablealready tolerates the same concurrent-run race withrescue Mysql2::Error ... raise unless ... already exists.Fix
Use
DROP TABLE IF EXISTS, so dropping an already-removed table is a no-op instead of an error. This mirrors the concurrent-run tolerancecreate_raw_tablehas, but at the SQL level so it doesn't swallow any unrelatedMysql2::Error.Adds a spec asserting
drop_tabledoes not raise when the table no longer exists (it fails on the old bareDROP TABLEand passes withIF EXISTS).Scope
This is the safe, idempotent guard for the crash itself. @willpower232 noted on the issue that the underlying lock-renewal behaviour (two workers holding the retention lock at once) may also be worth revisiting — that's a separate change, and
IF EXISTSstops the crash regardless of whether the lock overlap is later tightened.AI disclosure: prepared with AI assistance (Claude Code); the fix was traced against the code and syntax-checked with
ruby -cbefore submitting. The message_db specs run against MySQL, so the added spec is exercised by CI.