Set exec bit on Linux/macOS binaries after download#741
Conversation
urlretrieve creates the file at 0644; the C++ backend then fails with Permission denied / exit 126 when the executor invokes the binary. The issue has been latent because users typically hit the _is_binary_present cache from a prior install where pip/wheel extraction set the bit, or never invoked the binary directly. Surfaced while validating v1.4.0 binaries on a clean Colab install. Closes #740 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #741 +/- ##
==========================================
+ Coverage 75.48% 75.63% +0.14%
==========================================
Files 57 57
Lines 8168 8180 +12
Branches 1595 1597 +2
==========================================
+ Hits 6166 6187 +21
+ Misses 1382 1373 -9
Partials 620 620
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Addresses Greptile P1 on PR #741: the original chmod-on-download fix left existing broken installations unhealed because _is_binary_present returns True for a cached binary with valid hash and URL — the new chmod in download_binaries never runs for upgraders whose pre-fix download left the file at 0644. Extract chmod logic into _ensure_executable() and call it from both download_binaries (fresh download) and _is_binary_present (cache hit). Idempotent: only chmods when the desired bits aren't already set. Tests: - test_download_sets_executable_bit: covers the fresh-download path with urlretrieve mocked to drop a 0644 file - test_existing_non_executable_binary_is_healed: seeds a cached binary at 0644 with valid metadata, asserts _is_binary_present returns True AND the exec bit is now set Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Addressed Greptile review (Confidence 3/5 → expecting 5/5 next pass). Greptile P1 — "existing broken installations won't be healed": Valid. Fix (commit f54800b): extracted the chmod logic into Greptile minor — Regression tests added (tests/test__init__.py):
Both tests are skipped on Windows where the exec bit is meaningless. They pass locally. |
test__init reloads kwave with a patched platform.system returning
"Unknown". The reload raises NotImplementedError as expected, but only
after setting PLATFORM = "unknown" on line 22 of kwave/__init__.py.
The module is left in a partial state with kwave.PLATFORM = "unknown"
and URL_DICT (from the initial successful import) still keyed by
{linux,darwin,windows} — so subsequent tests reading URL_DICT[PLATFORM]
hit KeyError: 'unknown'.
CI exposed this because it ran the new chmod-bit tests after test__init.
Locally the test order or import caching happened to mask it.
Fix: reload kwave once more in a finally block (without the patch) to
restore valid PLATFORM and URL_DICT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If the binary lives on a read-only filesystem or is owned by another user, os.chmod raises PermissionError. Without a guard, this propagates out of _is_binary_present at module import time, which turns "chmod failed" into "import kwave aborts" — a strictly worse failure mode than the original Permission-denied-on-execute bug we were fixing. Catch PermissionError and emit a logging.warning instead. Users get a clear hint about chmod +x or reinstall, and the rest of kwave (notably backend='python') keeps working. Addresses Greptile review feedback on #741. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Addressed latest Greptile feedback (Confidence 3/5, two findings): 1. PermissionError unwrapped on os.chmod — legitimate. If the binary is on a read-only filesystem or owned by another user, the chmod fails and the exception propagated out of 2. Stale carry-over P1 ("existing broken installations won't be healed") — already fixed in commit f54800b; Greptile is still anchored on the round-1 SHA 3. @greptileai review |
| try: | ||
| os.chmod(binary_filepath, desired_mode) | ||
| except PermissionError: |
There was a problem hiding this comment.
except PermissionError is too narrow to cover all filesystem-level chmod failures. On a truly read-only filesystem (e.g., a Docker image layer or a read-only bind-mount), os.chmod raises OSError with errno.EROFS (30), which is not a PermissionError — it's only a plain OSError. That unhandled exception would still propagate through _is_binary_present → binaries_present() → module-level if not binaries_present() and abort import kwave entirely, exactly the failure mode the try/except was meant to prevent. The fix is to widen the catch to OSError, which is the base class for all OS-level I/O errors including PermissionError.
| try: | |
| os.chmod(binary_filepath, desired_mode) | |
| except PermissionError: | |
| try: | |
| os.chmod(binary_filepath, desired_mode) | |
| except OSError: |
Greptile 4/5 follow-up: the os.stat call was outside the previous PermissionError guard, so a broken symlink or other OS-level failure could still abort import kwave. Single try/except OSError covers both stat and chmod, matching the function's intent: an unfixable exec-bit is degraded to a warning, never fatal.
The guard is cheap defensive code. Testing edge cases that can't realistically happen given the call site (binary path controlled by kwave itself, not a user-supplied symlink) is not worth ~25 lines of test surface. Three functional tests remain (download path, cache-heal path, basic init).
|
@greptileai review Two changes since your last review: |
Summary
After
urlretrievedownloads the C++ binary, set the executable bit on Linux/macOS so the C++ backend can actually invoke it.Why this was latent
Most users have hit
_is_binary_presentfrom a prior install where pip/wheel extraction set the bit, so the broken `urlretrieve` path wasn't exercised. Surfaced on a clean Colab install when validating v1.4.0:```
$ kspaceFirstOrder-CUDA --version
bash: kspaceFirstOrder-CUDA: Permission denied
exit=126
```
`ls -l` showed `-rw-r--r--`.
Change
In `kwave/init.py` `download_binaries`, after `urlretrieve`, OR-in the user/group/other exec bits on non-Windows platforms.
Test plan
Closes #740
🤖 Generated with Claude Code
Greptile Summary
This PR fixes a latent bug where
urlretrievedownloads binaries with 0644 permissions, causingPermission denied(exit 126) when the C++ backend tries to invoke them on Linux/macOS. A new_ensure_executablehelper ORs in the user/group/other execute bits and is called both after fresh downloads and during cache-hit validation in_is_binary_present, so existing installations are self-healed at import time._ensure_executableusesos.stat+os.chmodguarded by a broadexcept OSErrorthat degrades to a warning rather than abortingimport kwave._is_binary_presentnow calls_ensure_executablebefore returningTrue, healing cached non-executable binaries on the nextimport kwavewithout requiring a re-download.test_download_sets_executable_bit,test_existing_non_executable_binary_is_healed) using anext(urls[0] for … if urls)pattern that safely skips the emptydarwin["cuda"]list.Confidence Score: 5/5
Safe to merge — the change is narrowly scoped, correctly heals both fresh-download and cached-binary paths, and degrades gracefully on read-only filesystems.
The
_ensure_executablehelper is called in both the download and cache-hit paths, ensuring all users are covered at the nextimport kwave. Theexcept OSErrorcatch is broad enough to handle read-only mounts and wrong-ownership scenarios without aborting the import. Tests cover both code paths with proper platform guards and theif urlsfilter that prevents anIndexErroron macOS where the CUDA URL list is empty.No files require special attention.
Important Files Changed
_ensure_executable(OSError-safe chmod) and calls it in bothdownload_binariesand_is_binary_present, correctly healing both fresh downloads and cached binaries at import time.if urlsguard that safely handles the emptydarwin["cuda"]list; also fixes existingtest__initto restore module state after a reload-based exception test.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[import kwave] --> B{binaries_present?} B -- No --> C[install_binaries] C --> D[download_binaries] D --> E[urlretrieve] E --> F[_ensure_executable\nafter download] F --> G[_record_binary_metadata] B -- Yes --> H[_is_binary_present\nfor each binary] H --> I{file + hash\n+ URL valid?} I -- No --> J[return False\ntriggers re-download] I -- Yes --> K[_ensure_executable\nheals cached binary] K --> L[return True] F --> M{PLATFORM == windows?} K --> M M -- Yes --> N[return no-op] M -- No --> O[os.stat → compute\ndesired mode] O --> P{bits already set?} P -- Yes --> N P -- No --> Q[os.chmod] Q --> R{OSError?} R -- Yes --> S[log warning\nnever fatal] R -- No --> T[done ✓]Comments Outside Diff (1)
kwave/__init__.py, line 81-112 (link)_is_binary_presentreturnsTruefor a binary that is already on disk with a valid hash and matching URL — it never checks the executable bit. A user whose binary was downloaded before this fix (or whose pip install produced the binary without the exec bit) will havebinaries_present()returnTrue, soinstall_binariesis skipped and thechmodadded in this PR never runs for them. TheirPermission deniederror persists across upgrades.Reviews (9): Last reviewed commit: "Mark OSError defensive branch as no-cove..." | Re-trigger Greptile