Skip to content

refactor(Scripts): extract Xcode report policy - #284

Open
kyleve wants to merge 7 commits into
codex/harden-script-tooling-foundationfrom
codex/extract-xcode-script-python
Open

refactor(Scripts): extract Xcode report policy#284
kyleve wants to merge 7 commits into
codex/harden-script-tooling-foundationfrom
codex/extract-xcode-script-python

Conversation

@kyleve

@kyleve kyleve commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • move embedded Python out of test, profile, and flaky into importable modules under Tools/
  • share schema-tolerant xcresult traversal and snapshot reporting while keeping each command's policy separate
  • retain public Bash argument parsing, simulator/process orchestration, streaming output, and legacy exit behavior
  • add fake-Xcode public-command contracts for argv/environment, stream/status propagation, malformed results, interruption, and closed pipes

Why

The high-risk untested surface was structured parsing and reporting policy, not the public shell bootstrap. Extracting those pieces provides direct tests and reuse without recreating the discarded Swift command framework.

Design decisions

  • xcode_results.py owns xcresult traversal only; it does not invoke processes.
  • snapshot_reports.py is shared by test and profile; thresholds and failure semantics remain command-specific.
  • test_runner.py, profile_results.py, and flaky_results.py stay separate so affected-scope, profiling, and flake policies cannot blur together.
  • ./test keeps TTY redraws, captured-CI heartbeats, cached denominators, image progress, and zero-test failure.

Compatibility

Public paths, flags, scope-selector precedence, output streams, defaults, and exit policies match main.

Adversarial coverage

  • Xcode failure through pipelines; progress/tee failure after Xcode success
  • zero-test success refusal, invalid bytes, high-volume output, and unlaunchable executables
  • schema-shifted/truncated xcresult JSON, unknown nodes, parameterized names, and missing fields
  • profile parser failure warns after a valid run; build/test failure remains fatal
  • flaky test failures become observations; infrastructure/build failures remain fatal
  • Ctrl-C reaches the active Xcode child; a closed output pipe does not leave it running
  • mutations for zero-test success, lost child status, ignored progress failure, and trusted malformed roots are all killed

Testing

  • 62 Python tests
  • 15 retained Ruby tests / 212 assertions at this layer
  • ./test StuffCoreTests (real Xcode smoke, passed)
  • ./profile --tests-only --no-snapshots (2,119 tests parsed, passed at stack tip)
  • ./flaky --suite-runs 1 --iterations 2 --no-update (2,120 tests observed, passed at stack tip)
  • full retained-tool, ShellCheck, formatting, attribution, and shard-plan gates

Stack

Second PR; based on #283.

@kyleve kyleve changed the title codex/extract xcode script python refactor(Scripts): extract Xcode report policy Aug 17, 2026
Comment thread Tools/test_runner.py

def _parse_problems(bundles: Mapping[str, Mapping[str, object]]) -> list[str]:
problems = []
if len(bundles) < 15:

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review focus: this deliberately preserves the shell implementation’s fail-closed bundle floor instead of introducing a new target catalog in this extraction. The direct tests cover both rejection and dependency closure; a later manifest-format change must keep under-selection observable.

@kyleve
kyleve force-pushed the codex/extract-xcode-script-python branch from a573993 to 92ea609 Compare August 19, 2026 02:42
@kyleve
kyleve force-pushed the codex/extract-xcode-script-python branch from 402ac57 to 95d4d3a Compare August 19, 2026 03:22
@kyleve
kyleve force-pushed the codex/extract-xcode-script-python branch from d856f46 to 3739353 Compare August 19, 2026 04:07
@kyleve
kyleve force-pushed the codex/extract-xcode-script-python branch from 3739353 to 56c6dde Compare August 19, 2026 04:09
@kyleve
kyleve force-pushed the codex/extract-xcode-script-python branch from c0b358b to e39b463 Compare August 19, 2026 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant