Change validation resilience and parallelisation to RRI migration script#5347
Draft
backspace wants to merge 2 commits into
Draft
Change validation resilience and parallelisation to RRI migration script#5347backspace wants to merge 2 commits into
backspace wants to merge 2 commits into
Conversation
…ct files The post-run JSON check flagged any changed file that didn't parse strictly, but a prefix substitution can't turn valid JSON invalid (it never touches structural characters) — so every failure was a file that was already non-strict (trailing commas, unescaped embedded source) and which the realm server's lenient parser tolerates. Record which files parsed cleanly before editing and only fail on a valid -> invalid regression; report already-non-strict files as a note. A genuine break still exits non-zero. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Host Test Results 1 files ±0 1 suites ±0 2h 8m 20s ⏱️ +43s Results for commit 3f9f599. ± Comparison against earlier commit ef128dc. Realm Server Test Results 1 files ±0 1 suites ±0 8m 51s ⏱️ -31s Results for commit 3f9f599. ± Comparison against earlier commit ef128dc. |
Edit files concurrently via `xargs -P` to hide per-file I/O latency on networked filesystems (EFS), where the sequential per-file loop was the bottleneck on large realm trees. - Add -j/--jobs <n> flag (default 16). - Each worker writes its own patch fragment (concurrent appends to one shared patch file would interleave and corrupt it); fragments are concatenated into the rollback patch after the run. Worker results (changed-JSON list, processed count, errors) go to shared list files aggregated afterward, since arrays can't cross the xargs boundary. - JSON verification reads its path lists from files rather than argv, which also makes it ARG_MAX-safe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
backspace
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When running it in staging, I got a warning about some files not being valid JSON after migration. But the files already were invalid JSON!
This also adds a
--jobsflag to migrate more than one file at once, since it took quite a while to run.