Allow the user to specify `limited-api` from the command line

[arcondello]

I think this is a more specific version of #639.

Currently, so far as I know, the only way to tell meson-python to tag a wheel as ABI compatible is with a project setting via the pyproject.toml

[tool.meson-python]
limited-api = true

However, for my project I would like to be able to build abi3 wheels for Python 3.12+, and not for lower versions. So it would be nice to be able to instead specify it via a build config setting. Something like

python -m build --config-setting=limited-api=true

I'd be happy to take a crack at a PR if this seems reasonable?

[eli-schwartz]

However, for my project I would like to be able to build abi3 wheels for Python 3.12+, and not for lower versions. So it would be nice to be able to instead specify it via a build config setting. Something like

This is already possible, because meson-python's 'limited-api" setting can be overridden by meson's option --config-setting=setup-args=-Dpython.allow_limited_api=false.

What you can't do is override it via meson.build code, e.g.

if py.language_version().version_compare('>=3.12')
    kw = {'limited_api': '3.12'}
else
    kw = {}
endif

py.extension_module(
    'foo'
    'foo.c',
    install: true,
    kwargs: kw,
)

It will work from meson -- extensions for 3.11 will never pass Py_LIMITED_API, and won't use abi3.so -- but as far as meson-python knows, the wheel tag should still be abi3.

[arcondello]

Right, that's more-or-less the approach I went with.

This works when building wheels when I have control over the build system. But it does mean that if a user tries to build with Python <3.12 and fails to override the limited API they will get an error.

So, for instance, this means that users can't do an sdist install from the repo with Python<3.12 without knowing to override.

It would be nice if I could make the default behavior to not try to make an abi3 compatibile wheel. So instead of the user needing to explicitly opt-out of abi via the command line, they rather need to explicitly opt-in.

[eli-schwartz]

It would be nice if I could make the default behavior to not try to make an abi3 compatibile wheel. So instead of the user needing to explicitly opt-out of abi via the command line, they rather need to explicitly opt-in.

From meson, you can do:

project('myproject', 'c',
    default_options: ['python.allow_limited_api=false'],
)

This will prevent limited API use by default even on python 3.13, but could be overridden via --config-settings when using cibuildwheel to publish to PyPI.

[arcondello]

Yes that works! Great! Thank you

[oscarbenjamin]

The way that this is configured is a bit confusing. For the benefit of anyone else who gets here:

If you just want to try out building with the limited API / stable ABI then add the argument for all extension modules:

limited_api_version = '3.12' # or make a project option

py.extension_module(
   'foo',
   ...
   limited_api: limited_api_version,
)

That is what gives extension modules like foo.abi3.so.

These can then all be disabled globally by default with

project('myproject', 'c',
    default_options: ['python.allow_limited_api=false'],
)

Now a limited API build is off by default but can be enabled with

meson setup build -Dpython.allow_limited_api=true

However now meson-python would give a wheel containing abi3 extension modules but without the abi3 tag in the filename:

# gives a cp312-cp312 wheel even though the extension modules are abi3:
pip wheel . -Csetup-args=-Dpython.allow_limited_api=true

You need to tell meson-python to set the abi3 tag in the wheel filename:

[tool.meson-python]
limited-api = true

Now it might seem like this means you are always going to use the limited API or that it will be used by default but it does not mean either of those things. The default is controlled by the default_options in meson.build which you can set to true or false and the non-default case can still be configured with e.g.

pip wheel . -Csetup-args=-Dpython.allow_limited_api=true

So you should always set limited-api = true in pyproject.toml. What this setting means is: "if we use the limited API then meson-python should set the correct abi3 tag in the wheel filename" but that is always what you want so there is no point in ever setting limited-api = false.

Unless I am missing something the limited-api configuration is not very useful and should just be always true.

[eli-schwartz]

I think that it's necessary because meson-python only checks the meson options, it doesn't check whether the project actually builds any *.abi3.so files.

So if your project simply doesn't support the limited API in the first place, the default meson setting is "allow", but the wheel won't be an abi3 wheel.

Hence the pyproject.toml setting could be better thought of as tool.meson-python.can-support-limited-api = true and you should always set it if your project is capable of building abi3 wheels.

Setting it when your project doesn't configure py.extension_module()s appropriately would result in a wheel filename tag saying "abi3" but containing cp313 extension modules, unless I miss my guess.

[oscarbenjamin]

meson-python only checks the meson options, it doesn't check whether the project actually builds any *.abi3.so files.

I just tested having limited-api = true in pyproject.toml but not setting limited_api for the extension modules and I got:

$ pip wheel . -Csetup-args=-Dpython.allow_limited_api=true
...
meson-python: error: The package declares compatibility with Python
limited API but extension module 'flint/pyflint.cpython-313-darwin.so'
is tagged for a specific Python version.

Without limited-api = true in pyproject.toml it builds a wheel and uses cp313-cp313 for the wheel tag which is the correct ABI tag for the extension modules but unlikely to be the intended outcome of python.allow_limited_api=true.

I think a better configuration here would be a command line flag to meson-python directly that says you want to build for the stable ABI and allows specifying the version:

pip wheel . -Csetup-args=--stable-abi=cp312-abi3

It does not really make sense to configure this as a static option in pyproject.toml because it is a build time choice and I think should generally be off by default (e.g. for pip install --no-binary foo). If the project does not support building against the stable ABI then it would be better to error from meson.build rather than output a wheel with the wrong tag so can-support-limited-api would fit better in meson.build.

[rgommers]

Reopening this, since the discussion is quite relevant and we probably have some improvements to make.

[oscarbenjamin]

Another point I realise now is that even if you set limited_api to 3.10 when building all the extension modules if you build with CPython 3.13 then meson-python will tag the wheel as cp313-abi3 rather than cp310-abi3. The version of CPython that is used for the build should be irrelevant so there needs to be somewhere to tell meson-python about the version of the limited API that is in use.