Skip to content

compiler: Faster compilation time#2928

Open
FabioLuporini wants to merge 46 commits into
mainfrom
faster-python-rebased
Open

compiler: Faster compilation time#2928
FabioLuporini wants to merge 46 commits into
mainfrom
faster-python-rebased

Conversation

@FabioLuporini
Copy link
Copy Markdown
Contributor

the key ingredients are:

  • A lot of "avoided, useless reconstructions" -- across all layers (equations, Clusters, IET)
  • More caching (which needed the above), again at all levels
  • More memoization
  • New loop fusion heuristics to avoid useless and expensive analysis

Bonus: a lot of, IMHO, good refactoring. And several new tests.

Compilation time improvements vary between 1.2x and 5x depending on the complexity of the Operator

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

❌ Patch coverage is 93.63057% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.92%. Comparing base (91d8596) to head (392fc8a).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
devito/passes/clusters/fusion.py 88.82% 16 Missing and 4 partials ⚠️
devito/ir/support/guards.py 27.27% 8 Missing ⚠️
devito/symbolics/manipulation.py 80.95% 4 Missing and 4 partials ⚠️
devito/ir/clusters/cluster.py 94.31% 5 Missing ⚠️
devito/passes/iet/engine.py 87.80% 3 Missing and 2 partials ⚠️
devito/tools/memoization.py 93.58% 2 Missing and 3 partials ⚠️
devito/ir/iet/visitors.py 93.10% 4 Missing ⚠️
devito/tools/utils.py 71.42% 3 Missing and 1 partial ⚠️
devito/ir/support/basic.py 96.80% 1 Missing and 2 partials ⚠️
devito/ir/iet/nodes.py 94.44% 1 Missing and 1 partial ⚠️
... and 4 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2928      +/-   ##
==========================================
- Coverage   83.35%   82.92%   -0.43%     
==========================================
  Files         248      249       +1     
  Lines       51734    52402     +668     
  Branches     4463     4532      +69     
==========================================
+ Hits        43122    43456     +334     
- Misses       7859     8162     +303     
- Partials      753      784      +31     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 69.00% <88.11%> (+0.29%) ⬆️
pytest-gpu-gcc- 78.27% <91.81%> (+0.23%) ⬆️
pytest-gpu-icx- 78.18% <91.71%> (+0.22%) ⬆️
pytest-gpu-nvc-nvidiaX ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread devito/arch/archinfo.py
def get_visible_devices():
device_vars = (
'CUDA_VISIBLE_DEVICES',
'NVIDIA_VISIBLE_DEVICES',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is NVIDIA_VISIBLE_DEVICES visible inside the container? I believe it can be combined with CUDA_VISIBLE_DEVICES as well - do we think this is a case we need to cover? Even if not, we should note it in a comment


@cached_property
def _writes(self):
from devito.symbolics.queries import q_routine
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implies that maybe the search file is getting a bit tangly and needs to be split up and moved


@cached_property
def reads_explicit(self):
terminals = set(retrieve_accesses(self.rhs, deep=True))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should already return a set right? I could see potentially wanting an OrderedSet however...


@cached_property
def _reads(self):
return tuple(set(self.reads_explicit) | set(self.reads_conditional))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OrderedSet perhaps?

return tuple(set(self.reads_explicit) | set(self.reads_conditional))

@property
def reads(self):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this separate to _reads?

if all(a is b for a, b in zip(expr.args, args, strict=False)):
args = tuple(args)

if type(expr) is tuple:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The blocks for tuple and list can be consolidated here I think - it would improve readability

Like retrieve_terminals, but ensure that if a ComponentAccess is found,
the ComponentAccess itself is returned, while the wrapped Indexed is discarded.
"""
from devito.symbolics.manipulation import uxreplace
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implies this file is getting super tangly - there has been other search import oddness in this PR. Maybe something to address sooner rather than later

else:
return self._default

def get(self, key, default=None):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default=self._default?

Comment thread tests/test_iet.py
body = iet.body._rebuild(body=iet.body.body + (DummyExpr(x, x),))
return iet._rebuild(body=body), {}

monkeypatch.setattr(iet_engine, '_update_args',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a Claude test? It seems to love using monkeypatch for tests...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover? Also the filename is typoed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants