Slow / apparently-hung first import from a fresh pip install

[micahpw]

Reported: a user installed nlr-gat from PyPI into a brand-new venv and ran from gat.scenariohandlers import PlexosScenario in the REPL. It appeared to hang — no output for a long time — and they Ctrl-C'd it. (The retry then hit pandas's known circular-import breakage after an interrupted import, which is a separate, expected consequence of the Ctrl-C, not a GAT bug.) Given time, the import does complete.

Reproduction and measurement

Fresh uv venv, pip install nlr-gat (built from current main), then:

$ time python -c "from gat.scenariohandlers import PlexosScenario"
...
2.20s user 0.58s system 5% cpu **51.216 total**

~51s wall clock but only ~2.8s of CPU time — the process is blocked, not computing. Re-running the identical command immediately after:

$ python -X importtime -c "from gat.scenariohandlers import PlexosScenario"
...
gat.scenariohandlers  cumulative: 1,365,155 µs (~1.37s)

So there are two distinct, additive problems:

1. One-time macOS Gatekeeper/codesign tax on first load (environmental, not a GAT bug per se)

The 51s-vs-1.37s gap matches the well-known macOS behavior where Gatekeeper/XProtect scans every newly-extracted compiled binary (.so/.dylib) the first time it's dlopen'd, before trusting it. GAT's dependency closure includes several large native-code packages — geopandas' GDAL/pyproj/shapely bindings, DuckDB, PyArrow, and (on macOS, via gat.quickplots's backend-selection logic) PySide6/Qt, which alone ships hundreds of dylibs. That's a lot of first-touch scanning on a fresh install.

We can't eliminate the OS-level scan, but we should stop making it worse than it needs to be (see #2) and set expectations: document in the README/troubleshooting that the very first import after installing can take noticeably longer on macOS, and it is not a hang.

2. Real, avoidable eager-import overhead (~1s, the addressable part)

Even on a warm cache, importing PlexosScenario alone pulls in far more than PLEXOS-loading needs, because gat/scenariohandlers/__init__.py eagerly imports every handler:

from .sienna import SiennaScenario
from .plexos import PlexosScenario
from .egret import EGRETScenario
from .reeds import ReEDsScenario
from .multi import MultiScenario
from .base import BaseScenario
from .file_scenario import FileScenario

This is inconsistent with the lazy __getattr__ pattern already used in the top-level gat/__init__.py (see its load/load_scenario_only/etc. lazy imports) — scenariohandlers never got the same treatment.

Consequences, from the -X importtime breakdown (cumulative µs):

Module	Cumulative	Why it's pulled in for a PLEXOS-only import
gat.quickplots / .utils	337,949 / 337,966	plexos.py imports random_color from gat.quickplots.utils — drags in matplotlib, matplotlib.pyplot, and macOS Qt-backend probing (matplotlib.backends.qt_compat, 81,706 µs) merely for one color-picking helper
gat.datahelpers (→ .sienna, geopandas)	258,236 / 139,204 / 133,207	scenariohandlers/base.py does from gat.datahelpers.parsers import *, which runs gat/datahelpers/__init__.py's from .sienna import * — pulling in geopandas (GDAL/pyproj/shapely) for a handler that never touches Sienna or geo data
gat.simulations (→ duckdb, polars)	274,855 / 101,838 / 163,059	pulled in via the eager SiennaScenario import even when only PlexosScenario was requested
gat.datahelpers.h5Parsers / h5py	114,691 / 107,424	same eager-datahelpers-package cause

Baseline pandas (necessary regardless) is 377,657 µs — so roughly ~1s of the ~1.37s total is avoidable overhead from importing code paths the caller didn't ask for.

Suggested fixes

1. Make gat/scenariohandlers/__init__.py lazy, mirroring gat/__init__.py's __getattr__ pattern, so from gat.scenariohandlers import PlexosScenario only imports plexos.py and its actual transitive dependencies.
2. Stop plexos.py (and any other handler) from importing gat.quickplots at module load just for random_color() — either inline a tiny color helper with no matplotlib dependency, or import it lazily inside the function that uses it. This also means fewer PLEXOS-only users pay the Qt-backend-detection cost at all.
3. Don't blanket-import * geopandas-dependent submodules from gat/datahelpers/__init__.py — sienna.py's geopandas usage should be lazy/scoped so importing gat.datahelpers.parsers (which base.py needs) doesn't transitively import geopandas for handlers that never touch it.
4. Consider the same lazy treatment for gat.simulations' duckdb/polars-backed submodules.

3. Related: unsolicited DEBUG log spam on plain import

Because @plot_function-decorated functions register themselves at module import time (in gat/quickplots/{dispatch,transmission,multi_system}.py), and GAT never calls logger.remove() outside the CLI's setup_cli_logging(), a plain import gat / from gat.scenariohandlers import ... in a fresh interpreter prints loguru's default DEBUG-level messages straight to stderr, e.g.:

2026-08-01 ... | DEBUG | gat.registry:decorator:68 - Registered plot function: plot_generation_capacities for MultiScenario

For a Python-API user (as opposed to gat CLI users, who get setup_cli_logging), this looks like noise or an error at best. As a library, GAT shouldn't leave a DEBUG-level default sink active for API consumers — either don't rely on loguru's bundled default handler (call logger.remove() at package import, matching common library practice of staying silent until the app configures logging) or defer plot-function registration out of import time entirely (fits naturally with fix #1/#2 above, since import-time registration is exactly what an __getattr__-based lazy scenariohandlers would also need to stop doing implicitly).

---

None of this is fatal — the import does complete — but combined, a first-time PyPI user's very first import can look exactly like a hang, and the underlying eager-import chain is worth trimming regardless of the OS-level tax.