Disabling entry points from dependencies

[ascillitoe]

Python Version

3.8.11

pytest Version

7.1.3

Package Version

3.12.0

Description

For our package alibi detect, we've defined a pytest-randomly entry point in our setup.cfg:

[options.entry_points]
pytest_randomly.random_seeder =
    alibi_detect = alibi_detect.utils._random:set_seed

One of our dependencies (thinc), also defines an entry point:

[options.entry_points]
pytest_randomly.random_seeder =
    thinc = thinc.api:fix_random_seed

We don't need thinc's random seeder on our tests, however, it appears to be pulled in automatically and causes our tests to fail. My understanding was that pytest would only pay attention to whichever entry point was defined first, however I have tried to change the order of our pip installs to no avail. Is there a strategy to handle such a situation?

[adamchainz]

Hmm, this is tricky. I did not consider this when using entry points.

I think it would be more ideal if the API were to register with a pytest fixture in your conftest.py. Perhaps smething like:

@pytest.fixture(scope="module", autouse=True)
def reseed_alibi(randomly_register):
	randomly_register(set_seed)
	yield

Or maybe it can be done without fixtures:

import pytest_randomly

pytest_randomly.register(set_seed)

Either way, custom reseed functions wouldn't be declared in setup.cfg and visible wherever the package is installed. Instead, they would be visible only during the test suite run.

This would also work for non-installable projects, like most Django projects.

WDYT?

[ascillitoe]

I wonder if the first option might be nicer than the second, in that it would give a little more control over managing the scope of the custom random_seeder, rather than having to do pytest_randomly.register in each test file you want to use it?

However, I can also imagine that sometimes a user might actually want a custom seeder defined in a 3rd party dependency to be available (it is also quite nice and convenient to define via an entry point IMO). To me the problem is that currently these are pulled in and run by default, with no option to turn off. One idea would be what I hacked together in #497 (please feel free to delete!). Essentially adding a new pytest_randomly option similar to the pytest -p flag, so that the user can choose which seeders to run?

[ascillitoe]

@adamchainz any thoughts on the above?