Stop asserting exact location-version-id equality after replication

[delthas]

One failure in the recent CTST flakiness census, in Bucket Replication location stripping (4 MiB row). Replication itself succeeded, in 59 s — the failure was the assertion that follows it.

Two oplog populators were alive during a rolling restart and each allocated a Kafka-Connect connector for the same bucket, so the object was replicated twice (three entries for one source version). The replication status processor then discarded the newer stamp, logging "entry replication is already COMPLETED for this location, skipping metadata update", which left a stale <location>-version-id in source metadata while the destination retained the version written by the later replication. Both objects were present and identical in content; only the recorded version id disagreed.

Replication is at-least-once by design, so that stamp is not guaranteed to name the version that ends up current at the destination. Asserting strict equality asserts an invariant the product does not offer, which makes the test fail on correct behaviour.

So the assertion now covers content rather than identity: ETags are compared (normalising the surrounding quotes the same way get-object-attributes.ts:68 already does, since the CLI and the SDK disagree on whether they are kept), and the version-id stamp only has to be present. ContentLength was already compared. The neighbouring scal-version-id, scal-replication-status and <location>-replication-status assertions are untouched — scal-version-id records the source version, which both copies share, so it is unaffected by the double replication.

Checked before adding the ETag comparison: every destination this feature configures is awsbackend, awsbackendmismatch or a CRR loopback location, and the objects are 0–4 MiB single-part uploads, so the ETags are comparable MD5s. No Azure or GCP destination is involved, where that would not hold.

Two oplog populators concurrently owning the same bucket during rollover is a product wart worth fencing separately; it has no ticket yet and is out of scope here.

Issue: ZENKO-5339

[bert-e]

Hello delthas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request TBA
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers