Py_LIMITED_API / PEP 384 (1357)

[GalacticEmperor1]

Issue №1357 opened by illume at 2019-10-01 13:18:27

pep-0384 defines a Stable API for python 3.

The benefit to pygame So it should be possible to use a single binary for all versions of python.

• research if this is possible (maybe not because we may use not-stable APIs, or because of Cython)
• define Py_LIMITED_API and
• py_limited_api=True for setuptools
  https://setuptools.readthedocs.io/en/latest/history.html# v25-4-0

Add Extension(py_limited_api=True). When set to a truthy value, that extension gets a filename appropriate for code using Py_LIMITED_API. When used correctly this allows a single compiled extension to work on all future versions of CPython 3. The py_limited_api argument only controls the filename. To be compatible with multiple versions of Python 3, the C extension will also need to set -DPy_LIMITED_API=… and be modified to use only the functions in the limited API.

Note that Cython does not support this at the time of writing. See issue here: cython/cython#2542 If we continue to use Cython this may be a problem preventing this.

---

Comments

# # nthykier commented at 2020-04-11 19:14:56

I think this is a noble goal, but I also think we should be realistic about this.

1. It is not feasible at the moment - both due to Cython but also the use of the Py Buffer API is "not yet" supported (AFAIR at least).
2. It would be a non-trivial undertaking and require special code paths for PY2, PY3 and PY3 limited API. While some of it can be hidden away by macros, it would still inflate the test matrices to ensure we over all code paths.

My recommendation would be to shelf this until Cython starts supporting it (or we are committed to replacing Cython) and are ready to drop support for all python versions before 3.4 (including 2.7), so we can avoid too many variations of the C code. (After 3.4, it is generally possible to use the same API regardless of whether or not you are compiling with -DPy_LIMITED_API=... and thus we can reduce the testing to always use -DPy_LIMITED_API=...).

---

# # pmp-p commented at 2022-03-12 09:38:12

It is not feasible at the moment - both due to Cython but also the use of the Py Buffer API is "not yet" supported (AFAIR at least).

good news everyone, from 3.11 Py_buffer is available in limited api.

or we are committed to replacing Cython

what about HPY https://github.com/hpyproject/hpy ?

we can reduce the testing to always use -DPy_LIMITED_API=...

indeed !

It would be best to drop anything <3.6 at once (3.6 is EOL after all) with a switch to limited API / SDL2 and do a major cleanup since after that maintenance will not be a burden (one abi3 module). I think there's more to fix in SDL2 than in pygame .... i also think it would be easier to backport toward a lower version of limited api after that ( can go down to 3.2 )

While testing the ability for pygame to support web assembly i tried to use Limited API it where i could. so i could start to upstream that.

btw i'm also for fixing function pointer calls asap ( more about that here https://blog.pyodide.org/posts/function-pointer-cast-handling/ ) i've done some for wasm port, but manually : it would be better to have a python parser check automatically for METH_NOARGS and matching function definitions.