Avoid using PyDictObject when compiling for RustPython

[bschoenmaeckers]

fixes #6047

Tested using:

~/repo/pyo3 ❱ RUSTFLAGS="--cfg RustPython" cargo clippy --features abi3-py314
   Compiling pyo3-ffi v0.28.3 (/Users/basschoenmaeckers/repo/pyo3/pyo3-ffi)
   Compiling pyo3 v0.28.3 (/Users/basschoenmaeckers/repo/pyo3)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.84s

[davidhewitt]

Sorry for the slow review; I'm not sure this is enough:

• See e.g. dict_len function which assumes the layout of PyDictObject; as long as the cast suceeds then this function is potentially wrong. Perhaps we should make PyDictObject opaque (or not exposed at all) in pyo3-ffi under RustPython?
• I haven't checked extensively, but I have similar concerns for other types. e.g. PyTupleMethods::as_slice.

[bschoenmaeckers]

Sorry for the slow review; I'm not sure this is enough:

• See e.g. dict_len function which assumes the layout of PyDictObject; as long as the cast suceeds then this function is potentially wrong. Perhaps we should make PyDictObject opaque (or not exposed at all) in pyo3-ffi under RustPython?
• I haven't checked extensively, but I have similar concerns for other types. e.g. PyTupleMethods::as_slice.

RustPython will never support compiling without the abi3 feature enabled. So PyDictObject will never be available. I can look into enforcing the Py_LIMITED_API cfg when compiling for RustPython.

[bschoenmaeckers]

In 55c2ea9, I force enable the Py_LIMITED_API flag so RustPython will never get compiled with incompatible api's.

[davidhewitt]

Thanks, this looks good. cc @ngoldbaum for interaction with #5786, I guess we won't need the is_free_threaded exclusion after that? Which PR should we merge first?

[davidhewitt]

Are the changes here still needed given the change to pyo3-build-config?

[bschoenmaeckers]

Not really but maybe a nice sanity check

[ngoldbaum]

Let's merge this and I'll go ahead and update #5786. It's already huge and this is pretty small. That said, I think #5786 is pretty much ready.