Only rebuild the Python dependency graph when files changed (#2599)

[mikkelke]

Fixes #2599.

The problem

PythonDeps.refresh_dep_graph reads and ast.parses every Python file under the apps directory. Dependencies.update calls it unconditionally, and check_app_updates calls update on every utility-loop iteration — so every pass re-parses the entire app tree even when nothing was touched.

Measured on a live install (79 app files, 2.76 MB, AppDaemon 4.5.13, Python 3.12), replaying the exact code path in-container:

py files considered : 79        total source bytes : 2,757,075
walk (iterdir+stat) :    17.5 ms
FileCheck stat pass :     0.9 ms
get_dependency_graph:  2235.3 ms  (failed=0)
  - pure read       :    13.5 ms
  - pure ast.parse  :  2191.8 ms

98% of the cost is ast.parse. Reading the files is 13 ms — this is CPU, not I/O.

It compounds: once the pass exceeds max_utility_skew, utility_loop skips its own sleep, so the loop goes hot. Measured from inside the container (482 sequential requests over 15 s):

slow (>200ms): 6, sum 12.91 s   max 2.240 s   median 0.0017 s
blocked fraction: 0.86

86% of the event loop blocked in ~2.2 s chunks every ~2.4 s, container pinned at 102% CPU. Because app-side sync API calls go through run_coroutine_threadsafe onto that same loop, app callbacks arrive up to ~2.4 s late — visible in app logs as stalls mid-callback.

The fix

FileCheck.update has already computed the new/modified/deleted sets by the time refresh_dep_graph is called, so it already knows whether anything needs re-parsing. This gates the refresh on there_were_changes via a needs_refresh hook, overridden only in PythonDeps.

AppDeps deliberately keeps refreshing unconditionally: its refresh_dep_graph walks the already-parsed app config rather than the filesystem, so it is cheap, and it is also called explicitly elsewhere when app configs change.

Files that previously failed to parse are still retried — a fixed file has a new mtime and therefore lands in FileCheck.modified. There is a test for exactly that.

Tests

Adds tests/unit/test_dependency_manager.py (7 tests): graph built on init; unchanged files do not rebuild; new/modified/deleted each do rebuild; a fixed bad file is retried; AppDeps still refreshes unconditionally. Three of them fail on dev and pass with the change. Full unit suite: 209 passed.

Note

This is a targeted fix, not the watchdog/inotify rework mentioned in #1897 — it just stops the redundant work. Raising utility_delay is a workaround but costs reload latency; this keeps detection at the normal interval.