Conversation
Two gaps, both closed offline. The first is that nothing exercised sending and saving *together*. Each half was tested against fixtures of its own making, so the two could disagree about the url, its `d` fragment, the key or the size and both still pass. MockNetwork grows the part of a file server that matters here: upload_file encrypts as the routers do and keeps the bytes, and download serves them back by the id in the url. A message sent with a file can then be saved back through the same object, and a disagreement between the halves shows up as bytes that do not match. upload_file joins download in being virtual -- a method meant not to be overridden would be private; these were merely never needed. The second is that legacy decryption had never run through the download plumbing. It has known-answer vectors, but those test the decryptor, not the path that feeds it -- picking the scheme from the url, checking the key length, holding the whole ciphertext because the MAC covers all of it, trimming to the sender's size. The blob is hard-coded rather than generated: libsession has no legacy encryptor, deliberately, so producing one here would mean writing the thing we chose not to have and then testing it against itself. Both were checked by breaking what they cover: generating the url without its fragment fails the round trip, and ignoring the sender's size fails the legacy save. An older test asserted attachments were *not* uploaded, which was only true because nothing could upload them. With a file server present that state is unreachable, so it now asserts what it was really about -- that the recorded size is the file's own, not the padded ciphertext's.
…d field A ProProof is version 0 by its type, not by a value it carries. Scope ProProofVersion (enum class : uint8_t) and rename ProProof -> ProProof_v0 with `using ProProof = ProProof_v0` (a note explains the alias becomes a variant or virtual base when a v1 arrives). The version field is dropped from the C++ and C proof structs and from operator==, and config no longer sets it. fill_proof also stops reading a JSON `version` off the generate_pro_proof response: the endpoint fixes the format and the proof's version is bound into its signature via the personalisation, so there is nothing for the client to read or gate on -- a future format is a new endpoint returning a new ProProof_vN, not a version bump on this response. The wire version survives only where it must: the protobuf-embedded proof a peer decodes with no endpoint context, where it selects the layout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…the message A received Pro proof whose wire version we don't recognize used to fail the parse, discarding the entire message -- the recipient silently never saw it. That's backwards for a forward-compat field: a newer proof format should cost the sender their Pro affordances on old clients, not the whole message. parse_pro_message now treats an unknown (or missing) version as a graceful degrade: it flags the proof ProStatus::UnsupportedVersion (C SESSION_PROTOCOL_PRO_STATUS_UNSUPPORTED_VERSION) and returns a non-pro message, so the caller skips signature evaluation and delivers it as an ordinary message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
session::json::require<T>/maybe<T> now accept a scoped enum T, validated as its underlying integer (nlohmann converts through the underlying type in extract), so an enum-typed field can be requested directly rather than cast at the call site. No current caller needs it -- it is a zero-cost `if constexpr` branch that only instantiates when asked -- but it rounds out the generic helper. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inner connection's UDP payload size was derived from the tunnel's suggested_mtu, which cannot be made to mean what it needs to. That suggestion is the outer *endpoint's* configured max_udp_payload minus a fixed overhead -- a policy knob rather than a measurement, unset in the default configuration, so the subtraction does not even run there. Where it does run, on a client that pinned the outer, it produces an inner MTU derived from someone's cap rather than from the path. The two values that are real are no better suited. A connection's max_datagram_size is deliberately the size reachable *by splitting a packet in two* -- get_max_datagram_size() is literally get_max_datagram_piece() * (splitting ? 2 : 1) -- so sizing the inner from it guarantees every inner packet splits. The per-piece size that would avoid splitting belongs to a connection this layer does not hold, and knowing it once would not be enough anyway: it moves when the first hop changes, and nothing here would hear about that. So the inner is pinned to the one size QUIC guarantees every path carries. libquic splits datagrams that do not fit, so this is a throughput question rather than a correctness one: a larger value would win when the outer path is roomy and cost a split per datagram when it is not, and we cannot tell which we have. The outer connection still discovers its own path MTU, which is where the gain actually is, and an application whose discovery misbehaves can still pin it through opt::quic_max_udp_payload.
An application that can save an attachment then wants to show that it did: without it, "save" is offered forever and saving twice writes two copies. Every client faces that, so it belongs in the data model rather than in each of them -- and Client is the only thing that knows a save completed, since save_attachment is where it happens. One nullable timestamp, not a path. Where a file went is the application's business: it chose the location and can move the file afterwards, so a path recorded here would be wrong as soon as it did. What is actually being asked is whether offering "save" again is pointless, and a timestamp answers that. It records an event rather than describing the filesystem, so "this was saved, then" stays true after the file is deleted. The column means the same thing in both directions: when the *recipient* of the message last saved this attachment. For one we received that is us; for one we sent it is them, from their notification. Nothing has to consult `outgoing` to read it, which is the trap the content-hash split just cost us an afternoon of. That second half needed the receive side of DataExtractionNotification, which did not exist -- _on_message_received returned early on anything without a DataMessage. It resolves the message by (timestamp, msgId), both required: the timestamp alone cannot identify a message sent in the same millisecond as another, and msgId alone is far too small. It also requires the message to be one of ours in the conversation with the sender, so a peer cannot make claims about a message they were not sent. attIndex names one attachment, -1 means all of them saved together, and absence means neither -- never "the first one". The saved time is the notification's own timestamp, not msgTimestamp, which identifies the message being talked about and is generally older; the test fails if the two are confused, which is how the mistake was caught while writing it. Unset means "not known to have been saved" and never "not saved": the outgoing half depends entirely on the other end volunteering a notification, which no released client does in a form we can read.
A node that does not participate in session routing has no relay contact, so nothing can be carried to it. Until now the request that discovered that simply failed: _fail_tunnel reported it to the caller, and the node was struck out of the pool only *afterwards*. So each such node cost one dead request -- a send surfaced as a failed message, a poll as a round that quietly returned nothing. The swarm is not in question when this happens, only one member of it, so recovery is to keep the swarm and move to the next member. That is the opposite of the 421 path, which fires when the swarm information itself is wrong and recovers by throwing it away and re-resolving; the two rhyme but must not be shared. Hence a set of failed nodes rather than another counter: "once per node" cannot be written as a number, since choosing the next one has to know which are already spent. Selection walks the swarm in order and stops when nothing is left, reporting the original failure rather than one of its own. Request::retry_count becomes retry_421_count. It only ever counted 421 redirects -- bounded by a config knob actually named redirect_retry_count -- and a generically named field with one specific meaning is how the next person makes the mistake this commit nearly made. Timeouts come down to match: 10s per attempt where it was 20s, with a 60s budget across the whole operation. A node that has not answered in ten seconds is better abandoned than waited on, which is the same conclusion session-ios reaches from the other direction -- it spends its budget on many short attempts rather than one patient one. Each attempt gets the per-request timeout or whatever is left of the overall budget, whichever is shorter, so walking a swarm of unreachable members cannot outlive what the caller asked for; below a two second remainder it gives up rather than start something that cannot finish. The upload path also gets the tunnel callbacks the download path already had. Without them an upload took the local port and sent into a mapping whose session might not be up, so the handshake was dropped and retried on QUIC's own timer, and an unreachable relay was discovered by waiting out the request rather than being told.
Network's own logic sat above routing and below MockNetwork, which overrides send_request wholesale -- so nothing exercised it. Neither the node walk nor _handle_421_retry, which has never had a test at all. Two seams open it up, both following the pattern already used for Core: TestHelper becomes a friend of Network and SnodePool. Substituting the router lets a test script how each node answers; seeding the swarm cache lets get_swarm resolve at all, which it otherwise cannot in a test -- a pool with no seed nodes has no swarms, which is what those "Cannot refresh cache" lines in every run have been saying. FakeRouter is otherwise trivial: eight no-ops and one method that answers from a table. Writing it corrected the implementation. The comment claimed the swarm comes back in a fixed order, so the walk would try members in the same order every time; get_swarm actually shuffles before partitioning by strike count, so what it really gives is least-struck-first in a random order among equals. That is the better preference anyway, and the walk only needs a member never to be chosen twice -- but the comment asserted something untrue and the tests now assert what is. Covered: reaching a working member without spending any twice, trying every member exactly once before giving up, reporting the original failure rather than an invented one, not walking on a failure that is the request's fault rather than the node's, not walking a request that names no swarm, and the budget shortening retries and refusing to start one it cannot finish. The order-sensitive parts are asserted as distinctness rather than sequence, and the suite was run five times over, since a shuffle makes a flaky test easy to write by accident. Verified by breaking the selection to ignore failed nodes, which fails the two tests that exist to catch exactly that.
The user configs carry two different kinds of state and the schema already had the seam for it: what an account *is* goes on accounts, and what a conversation with it is like goes on conversations. contact_info mixes both, and base_group_info carries the conversation half again for groups and communities -- the same fields meaning the same things -- so those become one set of columns here rather than three. Contact-ness becomes a row in `contacts` rather than a column on accounts. A flag would restate something the database already knows with nothing keeping the two in agreement; existence cannot disagree with itself. It also puts each field where it means something: a name and a picture belong to the account, because we learn those for people who are not contacts at all, while a nickname and the approval pair need a relationship before they mean anything. A contact removed on another device is then a row that goes away while the account stays, which is what we want -- we have still seen that person. The Pro proof lands on accounts even though ConvoInfoVolatile carries it per conversation. It describes the peer rather than the conversation, and session-ios already resolves it into its per-account Profile record at both ends -- nobody wants it per conversation, that is just where the sync mechanism puts it. Per-account also generalises where per-conversation cannot: convo::group has no pro_base, so a group member's proof has nowhere to go in the config at all, and can still be cached locally. Time units are per column now, matching whatever the config carries, which is not uniform: ConvoInfoVolatile's last_read is milliseconds while contact_info's created, profile_updated and mute_until are seconds. One rule produces both -- a value compared against a message timestamp is milliseconds, everything else is seconds. last_read qualifies because messages.timestamp is the other side of that comparison, and at second resolution reading a message stamped mid-second forces a choice between leaving it unread and marking the rest of that second read. Nothing else needs to know the sub-second moment a conversation was created or a mute expires, so conversations.created moves to seconds. Every mirrored column then round-trips with no conversion at all, so none of them can dirty a config by being written back slightly differently. The migrations go with it. No released version has a client schema -- v1.8.0 has no src/client/schema at all -- so all six existed to upgrade databases that cannot exist, which is why src/core/schema has been full_schema alone all along. Their comments were checked before deleting: everything in 001-004 and 006 is already reproduced in the column and table comments, and what is only in 005 argues about a column that no longer exists. That leaves the history check nothing to walk, since a schema change without a migration is exactly the break it looks for. SCHEMA_FLOOR names the last revision behind that break; everything after it is still walked, so consecutive commits must still upgrade cleanly and only the cutoff itself is exempt. Not done here: the conversation list still reads accounts.name rather than preferring a nickname, because nothing populates contacts until the config reconciliation lands. Communities and blinded identities wait on the same work -- both need servers split from rooms, which is a change to tables that have no rows yet.
Core gains a Configs component holding the five config objects, and the poll gains the four namespaces they live in. Nothing yet interprets what they contain -- that is Client's work -- but a config now arrives, merges against what we hold, and survives a restart. Core owns them because everything mechanical about a config is Core's work already: retrieving from a namespace, merging, keeping the dump, pushing what changed. None of it needs to know what a contact is. That does mean core.hpp names config types, so core links config publicly rather than privately now. The dumps table is keyed by encryption_domain() rather than by storage namespace. The two coincide for four of the five configs and diverge exactly where it matters: Local has no namespace of its own and reports UserProfile's as a stand-in, so a namespace key would collide. A domain is already required to be unique per config type and can never change, since changing it would break decryption of everything written under it. for_namespace() maps by hand for the same reason -- searching over storage_namespace() would find two configs for namespace 2 and answer according to whatever order it looked in. The configs are built on first use rather than in init(), because they are encrypted to the account key and a Core opened with defer_account has none yet. Nothing can reach them before an account exists in any case: a network cannot be attached without one, so neither polling nor pushing runs. Batch is the other half of the dumping story. A config is dumped when it changes, which is right when nobody knows any better -- but a caller working through a batch of messages does know better, and dumping between them writes intermediate states nobody reads. One poll carrying four config namespaces now dumps once rather than four times. It nests, so merge() can hold one itself without defeating a caller that already has. It also fixes what a debounce timer can only approximate: debouncing is a guess at where a batch ended, and the guess is only needed where nothing knows. Local is left out of the pushable set even though it already reports needs_push() as false unconditionally. That override is not what should be keeping it off a swarm: the push path asks which configs have a destination, and answering by trusting each config to decline makes "has nowhere to go" indistinguishable from "has nothing to say". The poll tests were positional -- three namespaces, Devices at index 1 -- so they broke on contact with a longer list and would break again on the next one. They now find a subrequest by the namespace it asks for, and the mock response is built from the request so that it lines up by construction, which is what the storage server does anyway. Not here: pushing. needs_push() reports it and nothing acts on it, so a local change is held and dumped but never leaves. The per-pubkey debounce and the sequence request are the next piece.
A dirty config now reaches the swarm. Everything owed goes in a single sequence: a store per config message, then one delete naming every hash those stores obsolete. The delete goes last because a sequence stops at its first failure, so nothing is removed before what replaces it has landed -- which is the whole reason this is a sequence and not a batch. Only this account's configs go in that request. A group's live under a different pubkey and are pushed separately even when the swarm turns out to be the same one: a request carrying both would tell that swarm the account and the group belong to the same person, and swarms are assigned per pubkey, so the collision is ordinary rather than unlikely. The component is per-account, so its timer is already per-pubkey and a group would get its own instance rather than a shared map. Confirmation is per config and all-or-nothing across the messages it split into. Confirming a partial push would drop the parts that did land from the obsolete list while leaving the config believing it is clean, so the part that failed would never be sent again. The debounce is the one from the design discussion: no sooner than two seconds after the last change, no later than ten after the first. Both bounds are recomputed from timestamps on each firing rather than tracked, so a re-arm cannot drift past the deadline the first change set. A steady trickle of changes therefore cannot defer a push indefinitely. What arms it is releasing the outermost Batch, unconditionally rather than only after a merge -- so the batch a poll already holds doubles as a sweep, and a config changed locally without one is noticed within a poll interval instead of sitting unpushed forever. Deferred work is handed to the event loop, which outlives this component and cannot be told to forget a call already scheduled. Callbacks hold a weak reference and do nothing once it has expired, which is what stops a pending push firing into a destroyed Core. swarm_request.hpp is the namespace rules, the timeouts, and the signed values, moved out of core.cpp because more than one part of Core now needs them and transcribing them twice is how one copy ends up disagreeing with the storage server. delete is the odd one out and gains its own helper: it signs the hashes it names and carries neither a namespace nor a timestamp. The debounce is tested by driving the decision rather than waiting out real intervals -- the timer fires on the event loop while a test reads from its own thread, so a test that waited would be both slow and racy. Verified by removing the cap and the in-flight guard, each of which fails exactly the test written for it.
Merging updated the configs and dumped them, and nothing above Core was told, so nothing could reconcile. configs_changed now fires with the namespaces a merge altered. It carries namespaces rather than a diff. A config diff describes a transition between config *states*, and the reconciling layer's currency is not a config state and is not recorded anywhere: a merge routinely lands several updates forward at once, `<` only reaches back config_lags, and a crash between merging and reconciling leaves the database behind by an amount nothing measured. Comparing against its own stored rows is what makes reconciliation self-correcting, and a diff would silently skip whatever those cases left behind -- permanently, since nothing would revisit it. Namespace is an unambiguous identifier *here* specifically because only merging configs can appear, and Local never merges -- so the collision with UserProfile's namespace, which forced for_namespace() to map by hand, cannot arise on this path. Firing is once per batch, after the dumps, so a handler never reads state that is not yet on disk and never sees a poll half-applied. Only merges are reported: a config the application changed itself is not news to it, and reporting it would invite reconciling its own write back over itself. ConfigBase gains a seqno() accessor, which is what makes "did that merge do anything?" answerable. The hash set merge() returns does not answer it -- a hash counts as parsed whether or not it turned out to be useful, so a stale config that changed nothing still comes back in it. The seqno moves on a local change, on adopting a higher config, and on resolving a conflict, and stays put when the merge was a no-op. Verified by reporting unconditionally, which fails the case that merges the same message twice.
The watermarks decide when the producer thread -- the loop reading and encrypting the file -- is paused and resumed against the stream's unsent buffer. Resuming at 128KB left very little to send while that thread was waking up and doing its next read, and that read is blocking file I/O, so the stream could run dry waiting on the disk rather than on the network. Raising the pair to 1MB/512KB leaves half a megabyte still queued at the moment the producer is asked for more. The comment on the loop records why that matters, since `next()` looking like a plain iterator call is exactly what makes the blocking easy to miss. Also drops an <fstream> include nothing used.
configs_changed decides "did this merge change anything" by comparing the seqno either side of it, which is only sound if a merge cannot alter the data while leaving the seqno alone. That was argued rather than checked, and the argument had a hole in it. The worry is a conflict at our *own* seqno: if resolving two same-numbered configs could adopt one of them in place, the seqno would sit still while the contents changed, and the comparison would miss it silently and permanently -- nothing revisits a config it decided was unchanged. Four combinations, because two things vary independently. Our own config is either Dirty, meaning the change has not been serialised into a message and nothing else can have built on it, or Waiting, meaning a message exists and has gone to the swarm where another device may have seen it -- and _merge takes a different path for each, since it passes `_state == ConfigState::Dirty` to make_config_message. Crossed with the incoming config either diverging from ours or containing ours outright, the latter being where "adopt the superset" would be tempting if the superset were chosen on data rather than on the diff chain. It is not: two distinct messages at one seqno are a conflict whatever their contents, and a conflict resolves to one past the highest. All four report. ConfigBase::seqno() being public is what makes the precondition assertable at all -- the test checks the two configs really are at the same seqno before merging, so it cannot pass by failing to set up the case it claims to cover.
Two devices making the same change from the same starting point is the case where a naive merge could leave us dirty with nothing to say, and push an update carrying no changes. It does not: the result settles clean against the other device's message, and owes no push. That is what the test pins down. It deliberately does not pin the seqno, which advances despite the data not changing -- merging while dirty builds a MutableConfigMessage, and that constructor increments unconditionally by design, while _merge's unwind for the spurious increment only applies when the winning config is our own. Here it is the incoming one, deliberately: a hash-less local copy is placed last so that an identical incoming message wins and we adopt its hash rather than having none. So we end clean at a seqno the swarm has never seen, and that is a real cost rather than a curiosity. The "within N" rule drops any config more than config_lags behind the highest seqno, and that window is counted in seqno values rather than in actual updates -- so a seqno consumed without publishing anything shortens it in real terms, and a device holding an unpushed change has it discarded sooner than it should be. Two devices making the same change is ordinary enough (marking the same conversation read, approving the same contact) for this to add up. Not asserted, then, because it is a wart to be fixed rather than a contract: pinning it would make the fix look like a regression. It also does not assert whether this reported to the application. It currently does, since the seqno moved, so the reconciling layer walks and finds nothing to do: idempotent, and the safe direction to be wrong in. Were the merge taught to recognise an identical config, this would stop reporting and nothing here would need to change. The direction that must never happen -- data changing without a report -- is covered separately.
The spurious increment from merging an identical config does not stop at one wasted number. Two devices agree at seqno 1; the identical merge leaves us clean at 2 while the swarm's newest is still 1. The other device, correctly at 1, then makes an ordinary change of its own -- which lands on 2, the value we already consumed. What should have been a clean adoption is a conflict against our phantom, so it resolves to 3 and leaves us dirty, owing a push we would otherwise not have made. One real change, two seqnos, and an extra round trip. It settles once the other device adopts ours, so it does not run away, but the cost falls on the "within N" window: two of its five spent carrying one change means a device holding an unpushed change is discarded that much sooner. The test asserts only that no data is lost through the collision -- both changes survive -- since the seqno arithmetic here is the defect and pinning it would make a fix look like a regression. The reasoning is recorded alongside so the fix does not have to rediscover it.
Ported from the dev-side change (Audric Ackermann, f0b9ab2) that better aligns the proof with the protobuf. Rather than the proof carrying a version selector, its format is bound entirely by the signing domain prefix (the `_v0` in BUILD_PROOF_DOMAIN is part of the signed bytes), and a future format arrives as its own protobuf field/message -- not a version bump on this one. - Drop the `version` field from the protobuf ProProof (regenerated pb.cc/pb.h), renumbering so field 1 stays reserved. - Remove the ProProofVersion enum; rename ProProof_v0 back to ProProof and drop the alias (a new format is a new type, not a vN of this one). - Replace ProStatus::UnsupportedVersion with the general ProStatus::Invalid: a proof we cannot read (sender attached none, or it is in a format we don't know, so to us it simply isn't present) is unusable, with no reason-specific status for callers to learn. Such a message is still delivered as non-pro rather than dropped. - Detect that via `!pro_msg.has_proof()` instead of a version check. - The json scoped-enum helper support stays (now caller-less), noted as such. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first config that does anything. A merged UserProfile now reaches the tables that answer queries: our own name and picture land on our accounts row, and note to self gets its priority and disappearing timer. It runs the opposite way round to the ones that follow. UserProfile is authoritative and the accounts row is a projection of it, because the config holds structure the row does not model: a second picture slot carrying the same image at a fresh URL, so a linked device that already has those bytes can skip fetching them again. Re-deriving the config from the row would flatten the two and destroy another device's reupload. What decides whether a conversation exists is the config, not whether there are messages in it. Emptying a conversation does not end it, and an account reimported onto a new client has nothing but its configs to rebuild the conversation list from -- so anything that existed only where messages had been would quietly disappear. For a contact that statement is the presence of an entry in Contacts, with priority saying separately whether it is shown. Note to self has no entry to be present or absent, because UserProfile exists from the moment the account does: it holds the account's own name. So priority carries both there, and a negative value is the only way that config can say "there is no note-to-self conversation". Hence a new account writes one -- an unset value reads back as 0, which would otherwise be indistinguishable from someone deliberately choosing to show it, and would make the conversation appear on every device on the account. And hence writing a note moves it back, since that is how "there is one now" gets recorded. Restoring an account is left alone: its real values are about to arrive from the swarm, and a local one would compete with them. Revealing it reconciles the whole config rather than just the priority. A timer set on another device has been waiting with nowhere to attach, and the moment the conversation exists is when it can land; nothing else would deliver it, because a local change is not a merge and reports nothing back to us. Creating the row is guarded by find_conversation rather than calling ensure_conversation outright: that helper bumps last_activity on a row that already exists, so unguarded it would shove note to self back to the top of the list on every config merge. nts_expiry carries a duration and no mode, because only one mode means anything when the reader is also the writer; it is stored as after-send. The test helpers build the other device from *our* dump rather than from nothing, since a device invented alongside us lands on the same seqno as the account defaults and has to be merged with them, which is a different scenario.
A change notification only arrives for state that changes after someone is listening, which leaves three ways for the tables to fall behind with nothing to announce it: a config merged by a version that did not yet know how to reconcile it, a crash between merging and reconciling, and a dump restored from an older state. In each the database is behind by an amount nothing recorded, and no further notification is owed -- a contact list that has stopped changing would stay unreconciled indefinitely. Core already polls and merges all four config namespaces, so this is not hypothetical: everything arriving for the three that have no reconciler yet is held in the configs and dumped, and would sit there unread even after a reconciler was written for it. Running the walk at startup makes all three self-healing rather than merely self-correcting, and it is safe to run when nothing has changed because reconciliation compares rather than replays: it costs a pass and writes nothing. Guarded on have_account(), since a Core opened with defer_account has no key to decrypt the configs with -- and nothing is missed, because an account that does not exist has no configs to have fallen behind, and whatever arrives once it does comes through a merge, which reports itself.
A safety catch for working against a real account: with push_enabled false, nothing this device does can reach the account's other devices or overwrite what they hold. Checked at the point of sending rather than at scheduling, so everything short of the wire still happens -- changes are held, dumped, and reported by needs_push(). It is deliberately not a pretence that the push succeeded: the state reads as unpublished rather than as settled, and switching it back on lets the next push carry everything accumulated since.
Groups have had `delete_before` and `delete_attach_before` since they existed: an instruction that messages (or just attachments) older than a timestamp are to be deleted, and arriving ones older than it dropped. One-to-one conversations never got them, which is why clearing a 1:1 is local-only while clearing a group as admin propagates, and why hiding one has to be faked from config *timestamps* instead. That fake is worth spelling out, because it is what this replaces. With nothing recording when a conversation was cleared, a client compares an incoming message's timestamp against the newest config message it has processed -- minus two minutes of slack for clock skew -- and drops the message if the config looks newer. So a conversation cleared at noon and some unrelated setting changed at five o'clock is treated as having been cleared at five: every message in between is discarded, permanently, because the swarm cursor moves past them. The imprecision is in the format rather than in any client -- no config records when a field changed -- so recording the instruction is the only fix. `d` and `D` in the contact dict, deliberately the same letters the group info config uses, and seconds to match it: these two will be read side by side forever by code asking "what is the delete-before for this conversation", and one unit across both is worth more than the second of precision milliseconds would buy. Note to self gets its own pair in UserProfile rather than a contacts entry. It has no contacts entry to carry them: UserProfile exists from the moment the account does, so there is nothing there to be present or absent. Keying one under our own pubkey was considered and rejected -- every client already carries a "self is not a contact" carve-out, and android has five contact iterations with no such filter, one of which would put us in the contact picker for group creation. `delete_attach_before` is only stored while it says something `delete_before` does not. Deleting a message takes its attachments with it, so setting either of the pair drops the attachment value when the message value already covers it, rather than leaving an instruction that means nothing.
`path` records where a file went or came from -- closer to a symlink than a handle. The file at the far end is the user's in both directions: theirs to attach, or theirs to save wherever they chose. So nothing here may unlink it, and the comment now says so rather than only hinting at it by mentioning that the file might vanish. Worth stating now because a delete-attachments-before instruction is coming, and the obvious reading of it -- delete the files -- would be deleting the user's own originals on the instruction of a config merge. What it will actually be entitled to remove is a cache of our own: attachments kept encrypted so they outlive the file server. That is a third thing at a path we picked, and it wants its own column; sharing this one would leave the delete path unable to tell our copy from the user's, and the safe implementation would then be the useless one. Comment-only: the schema fingerprint strips comments, so nothing to migrate.
Both directions at once, because they are not separable: the destructive rule that a contact absent from the merged config was deleted elsewhere is only safe while a locally-created contact is written into the config as it is created. Ship the reading half alone and the first merge eats every conversation we made ourselves. Each entry lands on three tables, because it carries three kinds of fact. Who someone is goes on accounts, which anyone we have merely *seen* also has. The relationship goes on contacts, where the row existing is what being a contact means. How the conversation behaves goes on conversations, which is also what brings that conversation into being: a conversation is not defined by having messages in it, or emptying one would lose it and reimporting an account would lose every empty one. Whose name wins is decided by profile_updated rather than by who spoke last -- the config's name is applied only when its stamp is at least as new as what we hold -- so a profile carried on a message that arrived out of order cannot displace a newer one. Out of order is the normal case, since a profile is observed from group messages as readily as from a DM. The reverse direction re-derives an entry from the rows rather than being applied alongside each change. The mapping then lives in one place and cannot drift from the tables it describes; a caller has to remember to call it, but cannot remember to call it wrongly. Presence follows the row too: no contacts row means no entry, so removing a contact is "delete the row and sync" rather than a second place that must remember to erase. Which is worth nothing unless the two directions agree exactly, so that is asserted: apply a config to the tables, derive one back, and the config must still be *clean*. Anything lost, rounded or defaulted in transit shows up as a dirty config -- and a mapping that dirties on every pass would push a pointless update after every merge, forever. Verified by perturbing one field by one second, which fails it. A nickname now wins over the name a contact chose for themselves, which is what the accounts/contacts split was for and had no effect until something populated it. Not here: deletion. A contact absent from the config is not yet acted on, so this pass only ever adds and updates. That lands next, on top of the round-trip property rather than before it.
A contact we hold that the merged config does not mention was removed elsewhere, and removing a contact takes the conversation and its history with it -- deliberately: someone who deletes a conversation means it deleted, not hidden on one device. Merely hiding is a negative priority and arrives as an ordinary settings change, so an absent entry can only mean the stronger thing. The account row stays. We may have seen that person in a group or a community and need their profile to render it; what is deleted is the relationship, not the person. Messages, raw content and attachments go with the conversation by cascade, and the files those attachments name are the user's -- theirs to attach or theirs to have saved -- so nothing is unlinked. _reconcile_all now derives outward before reconciling inward, and the order is load-bearing. Creating a contact commits the row and updates the config in memory, but the dump is written later, so a crash in between leaves a row the config has never heard of -- indistinguishable, inward, from one deleted elsewhere, and destroyed with its history. Publishing what we hold first tells those two apart. It is safe to run because assigning a config field its existing value does not dirty it, so the outward pass writes only where the dump was actually stale; a contact deleted elsewhere is still in our config at that point, so it is not resurrected and the deletion arrives and wins on its own merits. Both are tested, and the removal test had to be fixed before it tested anything: built from a fresh config, the second device is a rival at the same seqno and merges to the *union* of the two, so nothing is ever absent. It now descends from what we published, which is what a device that has seen our config actually is. Recorded but not built: once attachments are cached, freeing them cannot be done by deleting files alongside the rows that name them, because cascade takes those rows with no code of ours running. It wants the same treatment as everything else here -- list the paths still referenced, list the directory, unlink the difference -- which survives a cascade and also collects what a crash mid-download left behind.
A client offers four of these, and they differ by what they leave behind rather than by how they delete: blocking, clearing a conversation's messages, deleting the conversation, and deleting the contact. They are composed here rather than left for a UI to assemble, because which fields a deletion resets and what it tells the other devices is part of what the operation *is* -- three clients each deriving that separately is three chances to diverge. The schema does some of the composing for us. "Delete contact" resets the nickname, both approvals and the block before removing the entry in the other clients; here those are columns of the `contacts` row being deleted, because they exist only for as long as the relationship does. Deleting a blocked contact therefore unblocks them, which is what the others arrive at the long way round. Deletion is synced as an *instruction* and not as a local act. Clearing records the moment it happened in `delete_before`, so a device that was offline through the whole thing deletes the same history when it catches up, and one that never had a message cannot keep it. Applied retroactively -- what the instruction is about is the history that was there when someone chose to destroy it -- and never walked backwards, since a smaller value arriving later is not entitled to un-delete anything. Deleting a contact needs no such instruction: the entry leaving the config already removes the conversation everywhere. The value lives only in the config and gets no column. It is an instruction about history rather than a property of the conversation, and the only thing that reads it is the sweep that carries it out, so a column would be a second copy to keep in step for nothing. That does mean _sync_contact has to build on the entry that is there instead of replacing it, or it would quietly revoke what it cannot re-derive. Local changes now reach the config at all, which they did not before: setting a priority publishes it, and opening a conversation with someone makes them an approved contact -- choosing to write to them is what approving them is, and their entry is the only place a one-to-one conversation's settings can live. An incoming message from a stranger still does not, which is the message-request distinction, still to come.
A stranger's first message is a request rather than a conversation, and which one it is comes down to a single fact: whether we have ever written to them. Nobody sets that. Sending to someone approves them and receiving from someone means they approved us -- you cannot write to an account you have not accepted -- so both flags are a side effect of messages flowing, and answering a request is what accepts it. There is deliberately no accept operation to go with the four destructive ones; the reply is the accept, and declining is delete_contact. Approval only ever goes up, in either direction, because neither has a reverse: no config can make a message not have been sent. Copying the merged value verbatim would let an un-approval reach us -- other clients clear both flags on their way to deleting a contact, and a device that merged the clearing but not the deletion would file the conversation back under message requests, where no message anyone could send would get it out again. The gate is classification, not suppression: the message still needs somewhere to live, and a request opens as an ordinary conversation, so it gets a row and a config entry like any other. The entry matters -- a request answered on one device must not still be waiting on another. What differs is which list it is in, so `conversations()` and `message_requests()` are disjoint, `Conversation::request` says which, and the added/updated/removed handlers are shared between them. Both replacements are emitted together and never one alone: approval moves a conversation between the lists, hiding takes it out of whichever it was in, and a caller that had to work out which of those it just did would eventually get it wrong. Blocking now does something. A blocked account's messages are refused on arrival rather than filtered when drawing, so nothing they send becomes history, an unread count, or a reason to wake someone up. Only what they sent: our own copy of a conversation we later blocked is still ours.
The deletion pass in _reconcile_contacts asked, for every contact we store, whether the merged config still mentioned it -- by scanning the whole list of what it mentioned. That is quadratic in the size of the contact list, which is precisely the thing that is allowed to be large, and it ran on every merge that changed anything rather than only when something was actually deleted. Measured on a build with no optimisation, so the absolute numbers mean nothing, but the shape does: reconciling a merge of 100 contacts took 5 ms, 500 took 71 ms, and 2000 took 960 ms -- a fivefold and a fourfold increase in size costing fourteen and thirteen times the work. With the lookup it is 2.5 ms, 12 ms and 47 ms: four times the contacts, four times the time, at every size. Nothing else in the pass was superlinear. A reconcile that finds nothing to change still writes nothing, which is what matters on a device whose storage is slow rather than whose CPU is.
`request` says they wrote to us and we never wrote back. The mirror -- we wrote to them and they have never written back -- was stored as `approved_me` and surfaced nowhere, so a conversation we started looked exactly like one already under way. It is not: we are in *their* message requests, and until they answer there is nothing to say we have been accepted. Read the other way round from the same evidence, and it is deliberately not a list of its own: an outgoing request is an ordinary conversation of ours, in `conversations()` like any other, because we chose to start it. What it needs is a way to be drawn as waiting. Nothing clears it but a message from them. Nobody sends anything to announce an acceptance -- the explicit control message is optional and every client infers approval from a plain reply -- so a message arriving is the whole signal.
Reading a conversation on one device now stops it being bold on the others, which is the most-noticed thing the missing config was costing. It also brings `marked_unread` to life -- the deliberate "come back to this", which survives having read everything and is cleared by reading -- and fills in `pro_expiry` and `pro_revocation_tag`, which had no writer: Session Pro proof metadata is carried per-conversation here rather than with the rest of what we know about an account, so this is where it arrives from. The watermark only ever moves forwards, in both directions, and that is ours to enforce rather than something the config does. `set_base` lets a value be written backwards on purpose -- so a client can reset one -- and a conflict between two devices at the same seqno resolves by a tie-break that knows nothing about which value is newer. So a stale device can otherwise make read messages unread everywhere, and can also publish its own staleness back out. Both directions are guarded, matching what iOS does for the same reason. The flag is an ordinary setting with no such rule: marking a conversation unread elsewhere is an instruction, not a race. There is no deletion pass, and there must not be one. This config is pruned by age -- thirty days unread, forty-five on push -- rather than by anyone noticing a removal, so an entry missing from it means only that nothing has been read there lately. Reading absence the way Contacts does would destroy conversations for having been quiet. The same asymmetry runs the other way: an entry outlives the conversation it describes, so read state about one we do not have is skipped rather than used to create one, which would resurrect what another device deleted. Reconciling runs after whatever brings conversations into being, and again whenever that might have. Entry membership follows the other configs rather than being gated on having read anything, which is what desktop and iOS both do; pruning is what keeps the config small. Note to self is an ordinary one-to-one entry here, following desktop -- iOS excludes it from the unread flag on the grounds that UserProfile holds it, but UserProfile has no such field, so marking note to self unread there appears not to sync at all.
The protected: at the top of the members section was left behind by a reorganisation: the members it was written for sit behind a second one forty lines later. Its comment carried the only thing neither block said -- why those members are protected rather than private -- so that moves to where they actually are.
A cached file is named by a keyed hash of its url, and that hash was how
everything found it: "is this file here" hashed the url and looked the
result up. Two things follow from that, and both are worth being rid
of.
The hash becomes load-bearing for every file already downloaded rather
than only for the ones being written, so changing it cannot be done
without invalidating the lot -- which is exactly what 005 had to do.
And it does not run backwards, so eviction, which holds a file name and
needs the messages drawing that file, could not get to them at all: it
deleted the row and the file and said nothing, leaving conversations
showing pictures whose files had gone. `AttachmentAvailability` already
promises those changes are reported.
Give `attachment_cache` a surrogate key and have an attachment row
reference it, `ON DELETE SET NULL`. Nothing is duplicated -- the row
stores an integer, not a second copy of the name or the url -- and the
reference runs the way the questions do:
- availability comes back with the row, so reading a page no longer
hashes a url or looks anything up per attachment;
- finding an existing copy goes through the reference and uses the
name the entry was stored with, so the hash applies only to a file
being created and can change freely;
- eviction reads the messages off an index before the delete, and the
foreign key clears their reference on the way out.
The reconcile sweep drops its stale rows through the same path, so a
file that vanished behind our back is reported like one we deleted on
purpose. `_in_flight` moves to being keyed by url at the same time: the
hashed name is about what a directory listing reveals, which is not a
question a map in this process has.
Caching a file said nothing. On the download path that was hidden: the fetch's completion emitted right after storing, so the messages showing the file were told. An upload has no such completion, so keeping a copy of a file we sent -- which makes it drawable for every message quoting that url, and an attachment url is a hash of the encrypted body, so somebody having sent us the same file is the ordinary case -- told nobody but the message being sent. Move the emit to `_cache_attachment`, which is where a file becomes cached and is reached both ways. `store` then says whether it got that far, and the fetch reports the transfer ending only when it did not: a failure, a file that could not be written, and no cache at all are the same event to a reader, and all three leave nothing behind. The entry also has to leave `_in_flight` before any of that runs, or the report of a file arriving says it is still arriving. The test drives one file, shown by two messages, through every transition -- absent to fetching, back to absent for a transient failure, to cached, out again by eviction, to unavailable on a 404 and clear again on a resend -- and asserts which messages were told and what they were told the state was, rather than counting events. A second covers the upload case across conversations.
One cause routinely changes several messages, and reporting them one call at a time made the application reassemble what libsession had just taken apart. A file is the clearest case: it is one file, so evicting it, fetching it, or finding it unfetchable changes every message showing it at once -- and reporting a message also reports its replies, since a reply draws a preview of its target. So `message_added` and `message_updated` become `messages_added` and `messages_updated`, taking a vector. Two things are now promised that could not be before: - **ordered oldest first**, by the timestamp history is ordered on and then by id, so a handler applying a batch in order lands where a reload would put it. That is the reverse of `messages()`, which pages backwards from the newest; - **a batch may span conversations**, which is what the fan-out cases need. Nothing is lost by allowing it: `Message` already carries its own `conversation`, which is why the callbacks no longer take one. Eviction and the reconcile sweep now collect across the whole pass, so a message showing two of the files being dropped is told once rather than twice, and the ids are resolved to Messages only at the end -- one build per message, in its settled state rather than in whatever state it was in partway through the pass.
`FileTransferRequest::cancelled` was consulted on every upload path and on none of the streaming download ones, so cancelling a download only stopped us from using what arrived: the rest of the file came down the stream anyway and was dropped a chunk at a time. That is the common case rather than an unusual one. The stream scheme authenticates each chunk as it arrives, so a file that has been tampered with, or one that runs past the length its sender claimed, is known to be unusable from the bad chunk onwards -- and a large attachment can be most of the way through when that happens. Shutdown was the same: the routers cancel their active downloads and then close, with the transfers still running. Check the flag as each chunk arrives, which is the only moment this end of a transfer is given -- a download is driven by the server -- and abort the stream when it is set. A stalled stream still notices nothing, and the caller's timeouts remain what end those. The layer above needed one correction to suit it: a download we abandoned now completes with the cancellation we asked for, so the reason we abandoned it has to be preferred over the status when there is one. Otherwise a decryption failure would report itself as "download failed with status -10200", and the code that says a retry is pointless would be lost.
There was a getter for the cache limit and none for what is in it, so a client could show the ceiling but not how close to it the cache was. Summed from the cache index, in bytes on disk, which is the measure the limit is in and the one eviction compares against.
Eviction and `attachment_cache_size` were the same query written twice, which is also two copies of the answer to "why is this an optional" -- SQL sums an empty set to NULL, so an empty cache came back as no answer rather than as no bytes. Answer it once, in SQL, where the NULL is produced: `coalesce` makes the query incapable of returning one, so the C++ type stops advertising a state that cannot occur. That is how the rest of this file defaults a NULL already. The connection-taking overload is for eviction, which holds one and is about to write through it.
The schema grew over four migrations as this branch was built, one per commit that needed it, which is right while the commits have to stand on their own and wrong once they are merged: a migration is permanent, numbered and ordered, and these four describe intermediate states that have never existed anywhere but here. One migration per merged change instead. 005 now does what all four did, and the commits still show how it was arrived at. Its predecessors are exactly the databases that cannot take it -- they have some of the four recorded already -- so the branch goes on the schema-history skiplist: every commit of it but this one, since no state of it before this is something anything can have started from.
Nothing consumes a caption. It arrived on an incoming protobuf pointer, went into a column, came back out as a struct field, and went back out on the wire unchanged: no client renders it, nothing searches it, no index or constraint refers to it, and the conversation-preview query already skips it deliberately. A field that only round-trips is one more column to keep positionally correct in four statements for no behaviour. Field 11 is reserved rather than deleted: clients on the network keep sending it, and the number must never be handed to anything else. The checked-in protobuf output and the generated debug printer are regenerated to match, with protoc 3.21.12 - the version that produced the existing files.
One migration per merged change: dropping the caption column is part of what this branch does to message_attachments, so it belongs in 005 with the rest rather than in a 006 of its own.
`unavailable` was recorded across every row naming the url, on the reasoning that a url is a hash of the encrypted body and so names one file read one way. That holds for an honest sender and no one else: nothing ties a pointer's key, digest or size to its url, so anyone can send a pointer to a file they have seen with a key of their own making. Fetching it fails to authenticate, and that failure was then written to every copy of the file in every conversation -- permanently, and with the message that asking for a resend would not help. A 404 genuinely is the server's answer about the url, and stays url-wide. An authentication or size failure is about the claim, so it now marks only rows making the same claim; and a resend, which is evidence the server holds the file again, clears only the former, since the resent bytes would reproduce the latter. The same poisoning was reachable without the database, through joining: transfers were keyed by url, so a display asking for a file while a bad pointer's transfer ran would take that transfer's answer. They are now keyed by the whole claim, which honest copies share, so those still share one download; whether anything is fetching a file is still asked of the url. The struct carrying url, key, digest and size is renamed RemoteFile while it is being threaded through more places: it was called a pointer after the protobuf message it comes from, which is not what the word means in C++.
A fetch that landed marked every row naming the url as cached and left `unavailable` as it was, so a row could say both at once -- the file is here, and the file cannot be had. Reachable whenever a not_found was the server's passing trouble and a later attempt worked, or when one row's key failed and another's did not. A file in the cache is served to every row naming it, whatever key that row carries, so none of them can honestly keep a verdict that it is unobtainable. Clear it in the same statement that marks them cached.
`_download_decrypted` throws before sending anything when there is no network, the url is not a download url, or the key or digest is the wrong length. `_fetch_cached` had already registered the transfer and reported it as `fetching`, and the completion that would have removed it never runs -- so the file said `fetching` for the life of the process, and every later request for it joined a transfer that had never been sent and waited for ever. Register, try to start, and report `fetching` only once it has; on a throw, remove the entry and tell its waiters. Told here rather than rethrown, because not every caller still holds its handler by then: `profile_picture` moves it in. Two more of the same shape. An auto-download that throws -- a pointer with no url is enough -- escaped the arrival path before the message was announced, so it was stored and never reported; it is now caught per attachment. And a save that cannot start left its temporary file beside the destination.
Both callers already held one and spelled out all four of its fields to make the call; the function now takes it whole.
`Attachment` said two things about its file: `availability` -- cached, fetching or absent -- and `unavailable`, what the last attempt found. A display has one thing to decide, what to draw, and had to rank the two itself with nothing in the header saying how. `absent` in particular meant "go and look at the other field": fetchable if that was unset, not worth offering if it was. There is a ranking, and it is total: a cached file is what to draw whatever an earlier attempt found, a transfer running is about to settle it, and only with neither does the verdict matter. So say it once, as `not_found` and `unreadable` alongside the other three, and `absent` goes back to meaning only that the decision is the user's. The stored codes keep their own private type. The numbers are what is on disk, and the public enum's values are not meant to be anything in particular; tying the two together would let renumbering one silently change the meaning of the other. A verdict is also a fact about the last attempt rather than about the disk, so it is reported with no cache configured, where the other states all read as absent.
Its preamble described `name` as the hashed base url that a row could be matched against a message_attachments.url by, which is what keying the hash and adding the `cached` reference abolished: the hash does not run backwards, and nothing finds an entry by recomputing it. It also said the sweep adopts files that have no row, where it deletes them. message_raw_content's comment is put back above its own table; folding 002 into this file had left the cache's block between them, with a stray marker where the join was.
Every other migration here is bare SQL, with the explanation living in full_schema.sql beside the thing it explains; 005 had grown a copy of most of that prose, plus an account of how the branch arrived at it. Cut it to the statements and the one thing only a migration needs to say, that it throws the cache away.
A few this branch added describe what something used to be, or argue against an arrangement nobody would now propose, instead of saying why it is the way it is.
Keeping a copy of a file we sent reads it back and encrypts it whole, on the loop -- the same work caching a download does. A download is refused past MAX_REGULAR_SIZE, so that work is bounded there; an upload is not, and the only other check was the auto-download size limit, which is optional. With none set, sending a large file stalled the loop for as long as reading and encrypting it took. Apply the download's own ceiling. The cost is that a file past it cannot be drawn from the sender's own transcript -- but no recipient using this library can fetch it either, so that is where such a file already stands everywhere else.
An image attachment is a grey rectangle until its bytes arrive, and on a slow link that is most of the time the user spends looking at it. A ThumbHash rides on the pointer instead: 25 bytes at most, decodable into a blurred approximation of the picture, so the recipient draws something that resembles the image from the moment the message is listed. Carried opaquely. libsession stores the bytes, hands them back and puts them on the wire without decoding them or checking them against the file -- a client that wants a preview decodes it, and one that does not pays nothing. Like width and height beside it, encoding one needs the pixels, so it is the sender's to supply and an unset thumbhash stays unset. The 25-byte bound is the format's own ceiling: a 7x7 luminance DCT, 3x3 for each of P and Q, and 5x5 alpha. Over-long values are treated asymmetrically, as the surrounding code already treats everything else: thrown on the way out, where the value is our caller's and the mistake is theirs to see, and dropped on the way in, where it came from a remote peer and losing a placeholder must not cost the attachment it describes. Field 12, since 11 is reserved. The checked-in protobuf output and the generated debug printer are regenerated with protoc 3.21.12, the version that produced the existing files.
Brings liboxenquic 1.9.0, whose JobQueue owns the timers it runs, and the router's per-path stats. Loop::call_every is deprecated there, so every timer moves to JobQueue::add_timer: Core's poll, renew and probe timers onto Core's queue, and QuicFileClient's idle and upload progress timers onto a queue of its own, along with the jobs that reach the client. A queue cancels its timers when it stops, so Core no longer relies on declaring its poll timer last, and dropping a subscription no longer has to defer freeing a ticker it may be running inside. The progress timer is still removed on every exit from the upload, since its callback borrows the request. Stream::get_stats() gains a retained count, and a session's path is now reported with its stats rather than as a bare list of hops.
A subdirectory whose cmake_minimum_required predates 3.9 has CMP0069 OLD, so CMake ignores CMAKE_INTERPROCEDURAL_OPTIMIZATION there. protobuf's is 3.5, which made protobuf-lite the one library in an LTO build compiled without -flto, and a link that pulls its archive members in late -- testLogging's -- then fails on the inline std:: functions they reference. Default the policy to NEW beside the other settings that keep USE_LTO the single switch.
A failed download told its requester download_failed whatever went wrong, told its progress listener an integer status, and told every message quoting the file a separate verdict -- three vocabularies for one classification, and the one the requester got said nothing. Every failure now carries an Error code named after the object that failed: message.not_found and attachment.not_found for a request naming nothing, file.not_found and file.unreadable for the two verdicts AttachmentAvailability also shows, file.download_failed for an attempt that says nothing about the file, file.upload_failed, attachment.file_missing and file.save_failed for the rest, and network.unavailable when there is no network. AttachmentProgress and the other progress callbacks end with the same Expected<void> the requester gets, in place of a status integer, which retires the public ATTACHMENT_* constants; the verdict's on-disk values are Unavailable's own. Codes nothing emitted are gone. Inside, _download_decrypted's completion takes a DownloadResult and a reason, from which both the public Error and the recorded verdict are derived. An exception from the consumer of the bytes is failed rather than unreadable: it is our side failing, and recording it against the file would tell every message quoting it not to try again. Failures that used to escape as internal.exception now say what they are. A claim no download could satisfy -- no url, a url that is not one, a key or digest of the wrong length -- is unreadable, and one that reaches the download is recorded as a verdict without anything being sent; an incoming attachment with no url shows as unreadable. A request for a file that already has a verdict answers from it without downloading, since only a resend changes it.
Both want a transfer that fans each chunk out to its consumers and a cache writer that encrypts as it goes, which is a change of its own.
Attachment availability: say what we hold of a file, and tell everyone showing it
Two test cases swept shapes up to 100x100 and encoded every one of them. The component-ratio case ran 10000 encodes over 25.5M source pixels to discover a set of 13 ratios; the validity case swept to 100 in steps of 7 to discover 6 lengths. Both quantities depend only on the stored component counts, which are round(7 * w / max(w, h)) -- a function of the shape alone -- so every large pair lands in a bucket a small one already reached. A dense 20x20 sweep reaches all 13 ratios and all 6 lengths, tests more shapes than the sparse stride it replaces (400 rather than 225 per alpha), and cuts the encode work by 578x and 12.8x respectively. This was showing up badly on Debian stable armhf, where the two cases took 161.8s and 7.7s -- thumbhash alone was 170.5s against 55.9s for the other 1023 cases in the suite, and the slowest non-thumbhash test was 3.95s. That target is ARMv7 + VFPv3, which has no FMA instruction, so every std::fma in the DCT becomes a call into libm's software implementation: measured here, a software fma is ~6.8x a plain multiply-add and the call itself ~2.9x, on a machine already 4-10x slower than amd64. arm64 is the other way around -- it mandates FMA, and thumbhash runs faster there than on amd64. That disproportion is a real property of encoding for reproducibility rather than for speed, and it is left alone here; this only stops the tests from multiplying it by 25 million pixels of pointless work.
The multipart expiry test waited out 200ms/600ms timers with real sleeps, so anything slow enough to take longer than those windows between checks -- an unoptimized build under qemu, say -- failed it. Both timers read AdjustedClock, so the test now uses timers in minutes and moves the clock past them, which makes it independent of how long the test takes and drops ~0.9s of sleeping. The user profile timestamp test's 2s sleep was only there to make clock_now_s() tick over, which an offset does instantly.
s390x is here as a big-endian target to catch byte order bugs: built with crossbuild-essential-s390x against the :s390x -dev packages, with results run under qemu-user.
A big-endian build, to catch byte order bugs: cross-compiled on debian-forky-s390x-cross with the new toolchain file against multiarch :s390x packages, and tested under qemu-s390x. forky rather than testing so the job doesn't shift when forky releases. debian_build/debian_pipeline/apt_setup grow a foreign_arch (for `dpkg --add-architecture` ahead of the apt-get update) and debian_build a test_runner prefix for the test binaries; every other pipeline renders identically.
Add big-endian (s390x) CI job
thumbhash: stop sweeping 100x100 to learn 13 values
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This draft PR tracks the full client support being developed in libsession, to serve as the logic core of future versions of Session. This builds on top of PFS+PQ support (PR #103), and adds a ton of new features and capabilities needed to build a full Session client. The most notable starting point here is the
Clientclass which is the entry point for an active programmable Session client.This branch is not intended for review, but rather merely tracks the progress of the ongoing
clientbranch (which will eventually become thedevbranch).