Famedly release/v1.155#273
Open
FrenchGithubUser wants to merge 28 commits into
Open
Conversation
…from 1.34.0 to 1.43.0 in /complement (#19673) Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
…atches group (#19803) Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
Some attempts to debug element-hq/synapse#19795. --------- Co-authored-by: Eric Eastwood <madlittlemods@gmail.com> Co-authored-by: Eric Eastwood <erice@element.io>
Bumps [hashicorp/vault-action](https://github.com/hashicorp/vault-action) from 3.4.0 to 4.0.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/vault-action/releases">hashicorp/vault-action's releases</a>.</em></p> <blockquote> <h2>v4.0.0</h2> <h2>4.0.0 (May 12, 2026)</h2> <p>Improvements:</p> <ul> <li>Bump node runtime from node20 to node24 <a href="https://redirect.github.com/hashicorp/vault-action/pull/604">GH-604</a></li> <li>Fix leading slash in secret paths causing HTTP 400 errors (e.g. <code>/cubbyhole/test</code> → <code>v1/cubbyhole/test</code> instead of <code>v1//cubbyhole/test</code>)</li> <li>bump jsrsasign from 11.1.0 to 11.1.3</li> <li>bump body-parser from 1.20.3 to 1.20.5</li> <li>bump qs from 6.13.0 to 6.15.1</li> <li>bump http-errors from 2.0.0 to 2.0.1</li> <li>bump minimatch from 3.1.2 to 3.1.5</li> <li>bump underscore from 1.13.4 to 1.13.8</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/hashicorp/vault-action/blob/main/CHANGELOG.md">hashicorp/vault-action's changelog</a>.</em></p> <blockquote> <h2>4.0.0 (May 12, 2026)</h2> <p>Improvements:</p> <ul> <li>Bump node runtime from node20 to node24 <a href="https://redirect.github.com/hashicorp/vault-action/pull/604">GH-604</a></li> <li>Fix leading slash in secret paths causing HTTP 400 errors (e.g. <code>/cubbyhole/test</code> → <code>v1/cubbyhole/test</code> instead of <code>v1//cubbyhole/test</code>)</li> <li>bump jsrsasign from 11.1.0 to 11.1.3</li> <li>bump body-parser from 1.20.3 to 1.20.5</li> <li>bump qs from 6.13.0 to 6.15.1</li> <li>bump http-errors from 2.0.0 to 2.0.1</li> <li>bump minimatch from 3.1.2 to 3.1.5</li> <li>bump underscore from 1.13.4 to 1.13.8</li> </ul> <h2>3.4.0 (June 13, 2025)</h2> <p>Bugs:</p> <ul> <li>replace all dot chars during normalization (<a href="https://redirect.github.com/hashicorp/vault-action/pull/580">hashicorp/vault-action#580</a>)</li> </ul> <p>Improvements:</p> <ul> <li>Prevent possible DoS via polynomial regex (<a href="https://redirect.github.com/hashicorp/vault-action/pull/583">hashicorp/vault-action#583</a>)</li> </ul> <h2>3.3.0 (March 3, 2025)</h2> <p>Features:</p> <ul> <li>Wildcard secret imports can use <code>**</code> to retain case of exported env keys <a href="https://redirect.github.com/hashicorp/vault-action/pull/545">GH-545</a></li> </ul> <h2>3.2.0 (March 3, 2025)</h2> <p>Improvements:</p> <ul> <li>Add retry for jwt auth login to fix intermittent login failures <a href="https://redirect.github.com/hashicorp/vault-action/pull/574">GH-574</a></li> </ul> <h2>3.1.0 (January 9, 2025)</h2> <p>Improvements:</p> <ul> <li>fix wildcard handling when field contains dot <a href="https://redirect.github.com/hashicorp/vault-action/pull/542">GH-542</a></li> <li>bump body-parser from 1.20.0 to 1.20.3</li> <li>bump braces from 3.0.2 to 3.0.3</li> <li>bump cross-spawn from 7.0.3 to 7.0.6</li> <li>bump micromatch from 4.0.5 to 4.0.8</li> </ul> <p>Features:</p> <ul> <li><code>secretId</code> is no longer required for approle to support advanced use cases like machine login when <code>bind_secret_id</code> is false. <a href="https://redirect.github.com/hashicorp/vault-action/pull/522">GH-522</a></li> <li>Use <code>pki</code> configuration to generate certificates from Vault <a href="https://redirect.github.com/hashicorp/vault-action/pull/564">GH-564</a></li> </ul> <h2>3.0.0 (February 15, 2024)</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/hashicorp/vault-action/commit/892a26828f195e65540a40b4768ae4571f51ebfc"><code>892a268</code></a> Update copywrite headers for v.4.0.0 release (<a href="https://redirect.github.com/hashicorp/vault-action/issues/607">#607</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/a7ffa26e2e6ede175ca2e4f7dec54e78425d6936"><code>a7ffa26</code></a> Prepare for release v4.0.0 (<a href="https://redirect.github.com/hashicorp/vault-action/issues/606">#606</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/a049f0183861f1dbbd996f64b48335487cc968db"><code>a049f01</code></a> [COMPLIANCE] Add/Update Copyright Headers (<a href="https://redirect.github.com/hashicorp/vault-action/issues/605">#605</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/95977a3e2387e93244aaae1232de66fc47b379a3"><code>95977a3</code></a> Adding team-vault-consumption as CODEOWNERS (<a href="https://redirect.github.com/hashicorp/vault-action/issues/600">#600</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/7e48e563b6a9b4b0ba8b028c5ee89c41a8ae2671"><code>7e48e56</code></a> Upgrade Node.js to 24 and update dependencies (<a href="https://redirect.github.com/hashicorp/vault-action/issues/604">#604</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/79632e33d6953d190b940ffa440bf97821cabd80"><code>79632e3</code></a> [COMPLIANCE] Add Copyright and License Headers (Batch 1 of 1) (<a href="https://redirect.github.com/hashicorp/vault-action/issues/589">#589</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/734c523c4fbdb289cdf26dd2dc177f3627d1e140"><code>734c523</code></a> README.md: Removing jwtGithubAudience default (<a href="https://redirect.github.com/hashicorp/vault-action/issues/590">#590</a>)</li> <li><a href="https://github.com/hashicorp/vault-action/commit/2c5827061f1ad91ca97897d6257ebe638e033699"><code>2c58270</code></a> [Compliance] - PR Template Changes Required (<a href="https://redirect.github.com/hashicorp/vault-action/issues/586">#586</a>)</li> <li>See full diff in <a href="https://github.com/hashicorp/vault-action/compare/4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b...892a26828f195e65540a40b4768ae4571f51ebfc">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [idna](https://github.com/kjd/idna) from 3.11 to 3.15. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's changelog</a>.</em></p> <blockquote> <h2>3.15 (2026-05-12)</h2> <ul> <li>Enforce DNS-length cap on individual labels early in <code>check_label</code>, short-circuiting contextual-rule processing for oversized input while staying compatible with UTS 46 usage.</li> <li>Tidy core helpers: hoist bidi category sets to module-level frozensets (avoiding per-codepoint list construction), simplify length checks, and reuse the shared <code>_unicode_dots_re</code> from <code>idna.core</code> in the codec module.</li> <li>Use <code>raise ... from err</code> for proper exception chaining and switch internal string formatting to f-strings.</li> <li>Allow <code>flit_core</code> 4.x in the build backend.</li> <li>Expand the ruff lint set (flake8-bugbear, flake8-simplify, pyupgrade, perflint) and apply the surfaced fixes; pin lint CI to Python 3.14.</li> <li>Add Dependabot configuration for GitHub Actions.</li> <li>Convert README and HISTORY from reStructuredText to Markdown.</li> <li>Reference CVE-2026-45409 for the 3.14 advisory in place of the initial GHSA identifier.</li> </ul> <p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for contributions to this release.</p> <h2>3.14 (2026-05-10)</h2> <ul> <li>Removed opportunity to process long inputs into quadratic time by rejecting oversize inputs up-front. Closes a bypass of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li> </ul> <p>Thanks to Stan Ulbrych for reporting the issue.</p> <h2>3.13 (2026-04-22)</h2> <ul> <li>Correct classification error for codepoint U+A7F1</li> </ul> <h2>3.12 (2026-04-21)</h2> <ul> <li>Update to Unicode 17.0.0.</li> <li>Issue a deprecation warning for the transitional argument.</li> <li>Added lazy-loading to provide some performance improvements.</li> <li>Removed vestiges of code related to Python 2 support, including segmentation of data structures specific to Jython.</li> </ul> <p>Thanks to Rodrigo Nogueira for contributions to this release.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a> Release 3.15</li> <li><a href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a> Pre-release 3.15rc0</li> <li><a href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from kjd/convert-docs-to-markdown</li> <li><a href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a> Convert README and HISTORY from reStructuredText to Markdown</li> <li><a href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from kjd/dependabot/github_actions/actions-f3e34333ea</li> <li><a href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a> Merge branch 'master' into dependabot/github_actions/actions-f3e34333ea</li> <li><a href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from StanFromIreland/patch-1</li> <li><a href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a> Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions group</li> <li><a href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a> Merge branch 'master' into patch-1</li> <li><a href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a> Merge pull request <a href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from StanFromIreland/patch-2</li> <li>Additional commits viewable in <a href="https://github.com/kjd/idna/compare/v3.11...v3.15">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/element-hq/synapse/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Ports the event class to Rust. The main difference here are: 1. There is now a single event class 2. We now validate a lot more at event construction time than we previously did (we basically checked nothing before). This required some changes to the tests, including matrix-org/sytest#1423 Reviewable commit-by-commit. ### Overview of Event Rust structure The format of the event struct in Rust is quite different than that in Python. The top-level looks like: ```rust pub struct Event { /// The parsed event JSON. fields: FormattedEvent, /// The event ID. For format v1 this is read directly from the JSON; /// for v2+ it is computed from the canonical-JSON hash at /// construction time and cached here. event_id: Arc<str>, /// Synapse-internal per-event state that lives outside the federated /// JSON (e.g. outlier flag, soft-failure, stream positions). #[pyo3(get)] internal_metadata: EventInternalMetadata, /// The room version this event was parsed for. #[pyo3(get)] room_version: &'static RoomVersion, /// `None` for accepted events; otherwise a short reason set by auth /// when the event was rejected. rejected_reason: Option<Box<str>>, } ``` which includes the actual parsed event in `FormattedEvent`, plus the rest of the event metadata. ```rust pub struct FormattedEvent<E = Arc<EventFormatEnum>> { #[serde(default)] pub signatures: Signatures, #[serde(default)] pub unsigned: Unsigned, #[serde(flatten)] pub specific_fields: E, #[serde(flatten)] pub common_fields: Arc<EventCommonFields>, } ``` The struct is further split into the common fields, format specific fields, plus the signatures and unsigned. We split out the signature and unsigned fields as they are mutable, so when we clone the event we can still share the common and specific fields and only copy signature and unsigned. The `specific_fields` are the fields that depend on the format version. They can either be a specific format (e.g. `E = EventFormatV1`) or a type-erased enum `EventFormatEnum` that is across all room versions: ```rust pub enum EventFormatEnum { V1(EventFormatV1), V2V3(EventFormatV2V3), V4(EventFormatV4), VMSC4242(EventFormatVMSC4242), } ``` For example: ```rust /// Shared flat-list encoding of `auth_events` and `prev_events`, reused /// by every format from v2/v3 onwards. #[derive(Serialize, Deserialize)] pub struct SimpleAuthPrevEvents { pub auth_events: Vec<String>, pub prev_events: Vec<String>, } /// Version-specific fields for room versions 3-10. #[derive(Serialize, Deserialize)] pub struct EventFormatV2V3 { pub room_id: Box<str>, #[serde(flatten)] pub auth_prev_events: SimpleAuthPrevEvents, } ``` ### Dev notes As discussed in [`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$3gTjDO440GbAz57cXcCawwiyFLiD0crrarvS1uhzKOY?via=jki.re&via=element.io&via=matrix.org) --------- Co-authored-by: Eric Eastwood <erice@element.io>
Follow on from #19701. Some Synapse servers may have events in their database that don't pass the canonical JSON checks. This is bad, but we still want to be able to load them nonetheless.
This is based on element-hq/synapse#18416, which got reverted (#19614) due to it incorrectly rejecting to-device messages to users with many devices (and thus breaking message sending). Fix element-hq/synapse#17035 A to-device message content looks like: ```jsonc { "@user:domain": {"device1": {...}, "device2": {...}}, ... } ``` The previous PR would split up into multiple EDUs, each with a subset of the users. However, if one user's entry was too large it would not further split it up and then error out. The main change in this PR is to allow splitting up a single user into multiple EDUs. Other changes: 1. Rename to `SOFT_MAX_EDU_SIZE` to indicate that we sometimes send EDUs with larger size than that, and its more a target than a hard limit. 2. Check early if any to-device message (to a specific device) is too large to send, even if we're not going to send it over federation. This ensures that we catch issues where clients try to send too large to-device. This still means that if a client send a large individual to-device message it will fail, but I don't believe we ever send such large to-device messages (normally they're in the range of a few KB). --- I ended up changing the implementation a bunch to make it easy to reuse the code to split up dictionaries. Instead of repeatedly splitting up the EDU until each bit fits into the size, we instead record the size of each entry in the dict and instead split up based on cumulative size. This means we call `encode_canonical_json` on each entry rather than once on the entire struct, but its not significantly slower to do so. -- cc @MatMaul @MadLittleMods --------- Co-authored-by: Mathieu Velten <matmaul@gmail.com> Co-authored-by: mcalinghee <mcalinghee.dev@gmail.com> Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Follow on from element-hq/synapse#19701. Unfortunately serde has a bug when using `#[serde(flatten)]` with `arbitrary-precision` feature when handling integers that fit in a i128 when doing `serde_json::from_value`. See serde-rs/serde#2230. The `depythonize` hits the same issue. To fix this we make it so we only parse events from strings and not values.
Follow on from #19701 This (more or less) matches what we had before. Otherwise we just get a default `<builtins.Event at 0x...>` --------- Co-authored-by: Eric Eastwood <erice@element.io>
…all` (#19818) Knowledge from @reivilibre in [`#element-backend-internal:matrix.org`](https://matrix.to/#/!SGNQGPGUwtcPBUotTL:matrix.org/$2WUfCdA02wZw-6-jhw3QbLQ44BmKJrqLuZ6wjz2r7hk?via=jki.re&via=element.io&via=matrix.org) on 2025-12-15. Spawning from me making Rust changes but nothing useful was printed until I added `-v`, ```shell $ poetry install --extras all Installing dependencies from lock file Package operations: 0 installs, 1 update, 0 removals - Updating pyjwt (2.11.0 -> 2.12.0) Installing the current project: matrix-synapse (1.154.0rc1) Failed to install /home/eric/Documents/github/element/synapse ``` I also see `poetry run maturin develop` suggested but I'd prefer not to need to install `maturin` as yet another system tool to manage myself.
Fix #2860 Also cleans up comments around plans to define `ALLOWED_SCHEMES` as we can rely on Bleach's `ALLOWED_PROTOCOLS` defaults (`http`, `https` and `mailto`).
Followup to element-hq/synapse#19801: I only meant for this logging to happen on the instance that is doing the persisting.
…SM-only versions, is included in our support policy. (#19823) The reason for querying this support was wanting support for SQLite's JSON operators, which are currently not present in the SQLite version found in Ubuntu's oldest supported LTS. The JSON operators were used in some of the sticky events work (related: #19452). Our ruling was that we should support Ubuntu oldest LTS equally to Debian oldstable, so support the oldest of the two versions from those. That makes some kind of sense as it would be difficult to do otherwise without dropping support for that version of Ubuntu altogether, given if we kept publishing packages intended for use with Postgres, there's a risk that an innocent sysadmin would update their SQLite deployment without realising that it is no longer supported. This was [discussed months ago (private)](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.fcdvoc88dy5s) and at [private](https://docs.google.com/document/d/12RZKPk3a4__JUSH9wYHODo9rRyKzsHg6BSCAcmqmbOU/edit?tab=t.0#bookmark=id.u48ivjge4qpt). --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
…s changed (#19792) Reintroduces #19714, after being reverted in #19784. Fix element-hq/synapse#18844 Fix element-hq/synapse#19783 Fix element-hq/synapse#18880 This PR also adds a fix so that we don't always return immediately when using the e2ee extension. --------- Co-authored-by: Benjamin Bouvier <benjamin@bouvier.cc> Co-authored-by: Eric Eastwood <madlittlemods@gmail.com> Co-authored-by: Eric Eastwood <erice@element.io>
This is in prep for converting the event serialization to Rust. This is a fairly mechanical port, except that we store the appservice ID rather than the appservice object. This avoids us having to store a `Py<..>` (or port the appservice object over).
…orkers when MSC4452: Preview URL capabilities API is enabled. (#19839) Fixes: #19825 Introduced in: #19715 Always populate `url_preview_enabled` so `/capabilities` can expose it Needed so this line can be happy: https://github.com/element-hq/synapse/blob/106ed3623d434891fe1ac50aacc851e9804404fe/synapse/rest/client/capabilities.py#L82 --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
…f them fails. (#19842) When building v1.155.0rc1, flaky deb builds combined with fail-fast behaviour meant that I had to press the retry button 5 times to get a full set. Without fail-fast, I suspect 1 or 2 retries would have done the job. --------- Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org> Co-authored-by: Eric Eastwood <erice@element.io>
FrenchGithubUser
force-pushed
the
famedly-release/v1.155
branch
2 times, most recently
from
5d8feb8 to
6db25b4
Compare
upstream v1.155 ported the Event class to Rust, removing freeze() and unfreeze() from the Python API. Our workaround used a pattern solely to guarantee event.unfreeze() always ran. That pattern is not necessary anymore. Revert to upstream's state.
FrenchGithubUser
force-pushed
the
famedly-release/v1.155
branch
from
6db25b4 to
0b8abb8
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #273 +/- ##
==========================================
- Coverage 80.42% 80.42% -0.01%
==========================================
Files 502 502
Lines 72430 72208 -222
Branches 10917 10888 -29
==========================================
- Hits 58254 58071 -183
+ Misses 10896 10868 -28
+ Partials 3280 3269 -11
... and 9 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Member
Author
|
Waiting for upstream to look into the failing |
Member
|
Although it may not strictly be a blocker, the Test failure snipThis may just be a failure in the testing itself and not a major functional problem. Passed it to upstream to see what they wish to do. Apparently, this particular test step is getting skipped and they will be fixing that in the future so it does appropriately run when relevant changes are introduced |
FrenchGithubUser
force-pushed
the
famedly-release/v1.155
branch
2 times, most recently
from
2811085 to
d87ed0c
Compare
Member
This did end up being a fake test failure and the multiple fixes are being included in the next release. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Synapse to the v1.155.0 release line (including Famedly’s v1.155.0_1 additions), bringing in a batch of upstream changes across federation/to-device handling, Sliding Sync long-poll behavior, logging, and the continuing Rust port of core types.
Changes:
- Add to-device EDU sizing/splitting logic and related constants/tests to avoid oversized federation transactions.
- Update Sliding Sync long-polling to return immediately when request config effectively expands (e.g. required_state changes), plus refine “new data” detection semantics.
- Continue Rust migration work (Requester, event parsing/unsigned/signatures), add GCP structured logging formatter, and apply assorted fixes/tests/docs/version bumps.
Reviewed changes
Copilot reviewed 106 out of 110 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/util/test_split_dict.py | New unit tests for dict splitting helper used for size-bounded payloads. |
| tests/test_utils/event_builders.py | Adjust create-event defaults for v11+ derived room IDs in tests. |
| tests/test_event_auth.py | Extend join rules test builder to carry rejected_reason. |
| tests/synapse_rust/test_unsigned.py | Update Unsigned tests for new JSON-string constructor. |
| tests/storage/test_stream.py | Switch test event creation to make_event_from_dict helper. |
| tests/storage/test_redaction.py | Rebuild events via make_event_from_dict to set event_id safely. |
| tests/storage/test_msc4242_state_dag.py | Use a mocked EventBase for stable event_id in MSC4242 DAG tests. |
| tests/state/test_v2.py | Remove outdated FrozenEvent return doc in test helper. |
| tests/rest/client/test_transactions.py | Update requester mock to app_service_id field. |
| tests/rest/client/test_third_party_rules.py | Document/ignore immutability assignment in test callback. |
| tests/rest/client/test_sendtodevice.py | Add tests covering to-device size limits, EDU splitting, and transaction splitting. |
| tests/rest/client/sliding_sync/test_sliding_sync.py | Add timeout/await_result support and query encoding in sync helper. |
| tests/rest/client/sliding_sync/test_rooms_required_state.py | Add test ensuring required_state change wakes long-poll immediately. |
| tests/rest/client/sliding_sync/test_room_subscriptions.py | Add tests for room subscription required_state expansions waking long-polls. |
| tests/rest/client/sliding_sync/test_extension_e2ee.py | Add tests ensuring OTK/fallback-key fields don’t trigger early return. |
| tests/replication/storage/test_events.py | Ensure hashes key exists in built event dict for replication tests. |
| tests/push/test_email.py | Add tests ensuring email link schemes are sanitized. |
| tests/module_api/test_api.py | Add typing ignores for deprecated Event APIs in tests. |
| tests/logging/test_terse_json.py | Add tests for new GcpJsonFormatter output and severity mapping. |
| tests/handlers/test_room_policy.py | Update signature handling to mutate Signatures rather than replace. |
| tests/handlers/test_room_member.py | Switch to make_event_from_dict for test events. |
| tests/handlers/test_device.py | Switch to make_event_from_dict and adjust return types. |
| tests/handlers/test_appservice.py | Use new device inbox API for local messages. |
| tests/events/test_validator.py | Simplify mentions validation test (remove freeze/unfreeze path). |
| tests/events/test_py_protocol.py | Adjust protocol tests after EventProtocol runtime/type-checking changes. |
| tests/events/test_event_parsing.py | New tests for large integer handling and redaction across parsing paths. |
| tests/api/test_ratelimiting.py | Ensure appservice is cached for new AS lookup by id in ratelimiter. |
| tests/api/test_auth.py | Ensure mocked appservices have id; update requester creation fields. |
| synapse/util/init.py | Add split_dict_to_fit_to_size helper for size-bounded JSON dict batching. |
| synapse/types/handlers/sliding_sync.py | Refine bool semantics and document “return immediately” behavior (esp. E2EE). |
| synapse/types/init.py | Switch Requester type to Rust implementation; adjust create_requester to pass app_service_id. |
| synapse/synapse_rust/types.pyi | New stub for Rust Requester API. |
| synapse/synapse_rust/events.pyi | Update stubs for Rust Event/Unsigned APIs and related types. |
| synapse/storage/util/id_generators.py | Add debug logging for to-device stream persisted-position investigation. |
| synapse/storage/databases/main/events.py | Add explicit typing for auth chain map in chain-cover calculation. |
| synapse/storage/databases/main/events_worker.py | Add redact_behaviour.as_is handling; harden DB event parsing with error logging. |
| synapse/storage/databases/main/events_bg_updates.py | Use Rust redaction for signature verification pruning. |
| synapse/storage/databases/main/deviceinbox.py | Split local vs remote to-device inbox enqueue APIs; add msgid-aware logging. |
| synapse/storage/databases/main/censor_events.py | Use Rust redaction for pruned JSON stored during censorship/expiry. |
| synapse/storage/controllers/persist_events.py | Broaden some internal typing from list to Sequence for event_contexts. |
| synapse/rest/media/create_resource.py | Resolve AS object by app_service_id for rate-limit behavior. |
| synapse/rest/client/transactions.py | Use requester.app_service_id when forming transaction idempotency keys. |
| synapse/rest/client/room.py | Check app_service_id for AS-only timestamp behavior. |
| synapse/rest/client/login.py | Resolve AS by id for appservice login flow. |
| synapse/rest/client/keys.py | Update cross-signing UIA exemption checks for app_service_id. |
| synapse/rest/client/directory.py | Resolve AS by id for directory delete/edit operations; pass AS id explicitly. |
| synapse/rest/client/devices.py | Update AS exemptions to app_service_id checks. |
| synapse/rest/client/appservice_ping.py | Resolve AS by id for ping authorization and URL validation. |
| synapse/rest/client/account.py | Update AS exemption checks to app_service_id. |
| synapse/rest/admin/rooms.py | Typing improvements for power levels content. |
| synapse/replication/tcp/resource.py | Refactor POSITION command emission and improve replication logs. |
| synapse/replication/http/send_events.py | Switch Requester.deserialize to Rust signature (no store parameter). |
| synapse/replication/http/membership.py | Switch Requester.deserialize to Rust signature (no store parameter). |
| synapse/push/mailer.py | Remove unreleased bleach protocol-schemes comments from safe_markup. |
| synapse/module_api/callbacks/third_party_event_rules_callbacks.py | Stop freezing/unfreezing events; simplify callback return behavior. |
| synapse/logging/_terse_json.py | Add GcpJsonFormatter producing GCL-compatible structured JSON. |
| synapse/logging/init.py | Export GcpJsonFormatter in synapse.logging public API. |
| synapse/handlers/sliding_sync/room_lists.py | Treat effective config expansion (incl. required_state) as “must send room”. |
| synapse/handlers/sliding_sync/init.py | Compute result before waiting; wait from now_token for efficiency and config changes. |
| synapse/handlers/room_member.py | Use app_service_id for linearizer key; add warning/logging for missing events in event_map. |
| synapse/handlers/message.py | Resolve AS object by id for membership visibility and privacy-policy exemptions. |
| synapse/handlers/directory.py | Resolve AS object by id for alias namespace authorization. |
| synapse/handlers/devicemessage.py | Enforce per-message size; split remote to-device messages into multiple EDUs; new helpers/constants usage. |
| synapse/handlers/admin.py | Switch Requester.deserialize to Rust signature (no store parameter). |
| synapse/federation/sender/per_destination_queue.py | Use shared MAX_EDUS_PER_TRANSACTION and reserved EDU constant. |
| synapse/federation/federation_client.py | Use Event.deep_copy for signed_state copies instead of copy.copy. |
| synapse/events/utils.py | Use Rust redact_event for prune_event; use deep_copy for clone_event; update serialization logic for AS detection. |
| synapse/events/py_protocol.py | Rework EventProtocol for runtime vs type-checking, and update MSC4242 typing. |
| synapse/event_auth.py | Typing improvement for power_levels_content. |
| synapse/crypto/keyring.py | Use Rust redaction for deferred redacted JSON creation. |
| synapse/crypto/event_signing.py | Use Rust redact_event_dict for signature computation. |
| synapse/config/repository.py | Set url_preview_enabled regardless of worker role to avoid /capabilities 500s. |
| synapse/api/ratelimiting.py | Resolve AS by id for rate-limit exemptions. |
| synapse/api/constants.py | Add EDU/to-device sizing constants and reserved EDU count for transactions. |
| synapse/api/auth/msc3861_delegated.py | Update tracing tag emission to use app_service_id. |
| synapse/api/auth/mas.py | Update tracing tag emission to use app_service_id. |
| synapse/api/auth/internal.py | Update tracing tag emission to use app_service_id. |
| synapse/api/auth/base.py | Update IP recording logic around app_service_id and dummy-device behavior. |
| synapse/api/auth_blocking.py | Update MAU blocking logic to app_service_id checks. |
| schema/synapse-config.schema.yaml | Bump schema $id version and normalize whitespace. |
| rust/src/types/mod.rs | Add Rust Requester implementation with serialize/deserialize and tests. |
| rust/src/lib.rs | Register new Rust json/types modules. |
| rust/src/json.rs | Add AllowMissing wrapper type with serde helpers and tests. |
| rust/src/events/unsigned.rs | Change Unsigned constructor to accept JSON string; add deep_copy. |
| rust/src/events/signatures.rs | Add deep_copy and Default for Signatures. |
| rust/src/events/json_object.rs | Add get_field helper to JsonObject. |
| rust/src/events/internal_metadata.rs | Expose methods publicly and add deep_copy naming. |
| rust/src/events/formats/vmsc4242.rs | Add MSC4242 event format support (prev_state_events, auth derivation). |
| rust/src/events/formats/v4.rs | Add v4 format support with optional room_id derivation and auth derivation. |
| rust/src/events/formats/v2v3.rs | Add v2/v3 event format support (no explicit event_id). |
| rust/src/events/formats/v1.rs | Add v1 format support (explicit event_id, paired prev/auth events). |
| rust/src/events/formats/mod.rs | Add generic formatted-event container and format enum, with deep-copy semantics. |
| rust/src/events/constants.rs | Add Rust constants for event types/fields. |
| rust/src/duration.rs | Improve SynapseDuration (milliseconds storage, ordering, const ctors, IntoPyObject). |
| pyproject.toml | Bump Synapse version to 1.155.0. |
| poetry.lock | Update Python dependency lock (e.g. idna). |
| docs/usage/configuration/config_documentation.md | Normalize whitespace in generated config docs. |
| docs/structured_logging.md | Document GcpJsonFormatter usage for GKE/GCL. |
| docs/development/contributing_guide.md | Expand Rust build guidance and troubleshooting. |
| docs/deprecation_policy.md | Clarify SQLite support policy includes Ubuntu LTS (non-ESM). |
| debian/changelog | Add 1.155.0 / rc1 Debian changelog entries. |
| complement/go.sum | Update Go dependency checksums for complement tooling. |
| complement/go.mod | Bump Go version and update module requirements. |
| CHANGES.md | Add 1.155.0 and 1.155.0rc1 changelog entries (incl. Famedly additions). |
| Cargo.lock | Update Rust dependency lock (e.g. rand). |
| .github/workflows/release-artifacts.yml | Disable matrix fail-fast for Debian artifact builds. |
| .github/workflows/docker.yml | Bump pinned GitHub Action SHAs (vault-action, cosign-installer). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+187
to
+193
| def split_dict_to_fit_to_size( | ||
| original_dict: dict[str, Any], | ||
| *, | ||
| soft_max_size: int, | ||
| wrapping_object_size: int = 2, | ||
| ) -> Iterator[tuple[dict[str, JsonDict], int]]: | ||
| """Splits a dict up into a list of dicts, each of which is small enough to |
Comment on lines
+782
to
+786
| else: | ||
| # We (should) have covered all possible values of | ||
| # redact_behaviour, so this is unreachable. | ||
| assert_never(redact_behaviour) | ||
| raise ValueError(f"Unknown redact_behaviour {redact_behaviour}") |
Comment on lines
+101
to
+102
| timeout_ms: Optional timeout in milliseconds to use for the request. | ||
| await_result: Whether to block and wait for the result before returning. |
|
|
||
| for user_id, messages_by_device in messages_by_user_then_device.items(): | ||
| messages_json_for_user = {} | ||
| # Mesages to send to this specific user. A map |
Comment on lines
+1131
to
+1132
| """Serialiize a to-device message, ready to add to the device_inbox table. | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Famedly additions for v1.155.0_1
SYN-83
depends on: famedly/complement#19