Skip to content

Add age-based cleanup for simplecache - #2118

Open
be-student wants to merge 2 commits into
fsspec:masterfrom
be-student:codex/2105-simplecache-expiry
Open

Add age-based cleanup for simplecache#2118
be-student wants to merge 2 commits into
fsspec:masterfrom
be-student:codex/2105-simplecache-expiry

Conversation

@be-student

Copy link
Copy Markdown
Contributor

Fixes #2105.

SimpleCacheFileSystem does not persist cache metadata, so its inherited clear_expired_cache() had no records to inspect and silently retained every cached file. This override uses each cached file's local modification time, requires an explicit non-negative maximum age, and only modifies the writable cache location.

The regression tests cover files retained after writes, selective age-based removal, and invalid expiry arguments. The feature documentation and changelog now describe manual age-based cleanup.

Validation:

  • pytest -q fsspec/implementations/tests/test_cached.py (79 passed, 31 skipped)
  • all pre-commit hooks on changed files (including Ruff and codespell)
  • full suite: 1562 passed, 163 skipped, 2 xfailed; the 52 failures are the existing macOS failures in test_posix_tests_bash_stat, reproduced unchanged on a clean upstream/master worktree because BSD stat does not support -c

@martindurant

Copy link
Copy Markdown
Member

Runs into the following issue on windows (seen in the test):

>       os.utime(old_cached, (1, 1))
E       TypeError: utime: path should be string, bytes or os.PathLike, not NoneType

Test-only path normalization follows the cache lookup contract on Windows. Production behavior and public documentation are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

simplecache retains written files indefinitely

2 participants