Task Summary
fs (PyFilesystem2) is effectively EOL: 2.4.16 (May 2022) is the last release ever published, upstream master still imports pkg_resources at import time (PyFilesystem/pyfilesystem2#577 and #597 have been open for years), and there is no maintained successor package. Since setuptools 82 removed pkg_resources entirely, import fs hard-crashes there — which is the only reason amber pins setuptools==80.10.2 (#4199, #6412). PR #6880 silences the resulting deprecation warning in CI, so this issue keeps the underlying debt visible (as suggested in this review comment).
Today: ExecutorManager -> fs.open_fs("temp://") -> pkg_resources -> setuptools held < 82
After: ExecutorManager -> tempfile.mkdtemp + shutil.rmtree -> fs gone -> setuptools unpinned
ExecutorManager is the only consumer, and fs's TempFS is internally exactly tempfile.mkdtemp + shutil.rmtree, so the swap is behavior-preserving. Full footprint:
| File |
Change |
amber/src/main/python/core/architecture/managers/executor_manager.py |
replace fs.open_fs("temp://") with tempfile.mkdtemp() + shutil.rmtree, preserving lazy creation, sys.path handling, the never-materialized close guard, and the tolerated leak-on-force-kill semantics |
amber/src/test/python/core/architecture/managers/test_executor_manager.py |
two lines read manager.fs.getsyspath(...); no test mocks fs |
amber/requirements.txt |
drop fs==2.4.16 and the setuptools==80.10.2 pin plus its comment (fs is the only distribution in the closure requiring setuptools; pytz's pkg_resources use is a guarded fallback) |
amber/LICENSE-binary-python |
drop fs==2.4.16 and appdirs==1.4.4 (an fs-only transitive); keep six (python-dateutil needs it) |
The migration PR should also revert the two warning-filter hunks from #6880 (worker entry point + pytest filterwarnings), which become dead weight once fs is gone.
Implementation notes:
- Use
tempfile.mkdtemp, not tempfile.TemporaryDirectory — its GC/exit finalizer would silently clean up abandoned directories, changing the documented leak-on-force-kill behavior (see the TODO in executor_manager.py).
- Both license bullets must land in the same PR: stale
LICENSE-binary-python entries hard-fail check_binary_deps.py even in PR mode.
- torch (operator-requirements) still declares
setuptools<82 transitively; harmless, and invisible to the license check.
Task Type
Task Summary
fs(PyFilesystem2) is effectively EOL: 2.4.16 (May 2022) is the last release ever published, upstream master still importspkg_resourcesat import time (PyFilesystem/pyfilesystem2#577 and #597 have been open for years), and there is no maintained successor package. Since setuptools 82 removedpkg_resourcesentirely,import fshard-crashes there — which is the only reason amber pinssetuptools==80.10.2(#4199, #6412). PR #6880 silences the resulting deprecation warning in CI, so this issue keeps the underlying debt visible (as suggested in this review comment).ExecutorManageris the only consumer, and fs'sTempFSis internally exactlytempfile.mkdtemp+shutil.rmtree, so the swap is behavior-preserving. Full footprint:amber/src/main/python/core/architecture/managers/executor_manager.pyfs.open_fs("temp://")withtempfile.mkdtemp()+shutil.rmtree, preserving lazy creation,sys.pathhandling, the never-materialized close guard, and the tolerated leak-on-force-kill semanticsamber/src/test/python/core/architecture/managers/test_executor_manager.pymanager.fs.getsyspath(...); no test mocksfsamber/requirements.txtfs==2.4.16and thesetuptools==80.10.2pin plus its comment (fs is the only distribution in the closure requiring setuptools; pytz'spkg_resourcesuse is a guarded fallback)amber/LICENSE-binary-pythonfs==2.4.16andappdirs==1.4.4(an fs-only transitive); keepsix(python-dateutil needs it)The migration PR should also revert the two warning-filter hunks from #6880 (worker entry point + pytest
filterwarnings), which become dead weight oncefsis gone.Implementation notes:
tempfile.mkdtemp, nottempfile.TemporaryDirectory— its GC/exit finalizer would silently clean up abandoned directories, changing the documented leak-on-force-kill behavior (see the TODO inexecutor_manager.py).LICENSE-binary-pythonentries hard-failcheck_binary_deps.pyeven in PR mode.setuptools<82transitively; harmless, and invisible to the license check.Task Type