Expand float and complex strict mode to allow ints and ints/float (for PEP 484 compatibility).

[InvincibleRMC]

Description

Breaking change to allow Python ints into float when under strict typing (noconvert), to better match PEP 484 numeric tower rules.

This change was also done to complex to allow floats and ints. Now that Python ints can be passed into floats it changes behavior for overload resolution. A method that takes float that is registered before a method that takes an int will now get executed when a Python int is passed in. This overload resolution also affects methods with std::complex.

Resolves #5878
Provides a work around for #5767

NOTE: This PR was reviewed extensively and is approved. However, because of the breaking change, it will be merged only after the next patch release.

Suggested changelog entry:

Breaking change for PEP 484 compatibility: Expand float and complex strict mode to allow ints and ints/float.

---

📚 Documentation preview 📚: https://pybind11--5879.org.readthedocs.build/

[gentlegiantJGC]

I have started working on this too.
Your code change is more complex than required.
I will finish up the documentation changes and commit what I have and you can take it from there.

[gentlegiantJGC]

Here is my implementation and documentation changes.
I have modified the behaviour of complex as well.
https://github.com/gentlegiantJGC/pybind11/tree/fix-pep-484
master...gentlegiantJGC:pybind11:fix-pep-484

It needs tests but I am going to stop for today.
You are welcome to use my commits.

[gentlegiantJGC]

You will need to remove the constexpr check. I didn't realise it was a newer C++ feature

[InvincibleRMC]

TODO: figure out why it doesn't work in C++11.

[gentlegiantJGC]

I have just found that a python float passed to an int argument will raise a TypeError.
It will happily except anything that implements __int__ (eg a numpy float) but explicity not a python float.

This is probably something we should resolve here because it effects these tests.

m.def("func", [](int){});

>>> func.__doc__
  func(arg: typing.SupportsInt) -> None
>>> func(5.5)
  TypeError
>>> func(numpy.float32(5.5))  # This works fine

[gentlegiantJGC]

Here is the commit that implemented that.
3f200fa

@rwgk Can I get your input on this?
The int type caster supports anything that implements __int__ with the explicit exception of the python float.

This conflicts with the typing.SupportsInt type hint.

[gentlegiantJGC]

Here is my proposed change. You will need to update the tests.
gentlegiantJGC@d42c8e8
https://github.com/gentlegiantJGC/pybind11/tree/pr-5879-jgc

Here are my unittests
test_cast.py
cast.cpp

[InvincibleRMC]

I noticed something else to have address. According to the documentation all of int,float,complex support falling back on __index__ when converting so updated the type hint from typing.SupportsInt -> typing.SupportsInt | typing.SupportsIndex. complex also supports a __float__ fall back so its type is now typing.SupportsComplex | typing.SupportsFloat | typing.SupportsIndex

[gentlegiantJGC]

I can't see any obvious issues

[gentlegiantJGC]

I will open a new issue about the custom float -> int conversion

[rwgk]

The extra tests (commit f1d8158) are:

• To be certain about the exact behavior in pathological cases, so we're sure we don't have silent float to int conversions.
• To find out if the PyPy code path in cast.h behaves identically.

This is a bit on the paranoid side, but I wanted to reassure myself that we're not putting the world through a behavior change without having looked at every dark corner.

[rwgk]

BTW, having looked at this so intensely ...

        bool py_err = py_value == (py_type) -1 && PyErr_Occurred();

this will be OK if Python is healthy, but if something goes wrong (e.g. UB, or bug in Python beta testing) and py_value is not -1 even though PyErr_Occurred() is true, the error will not be cleared and could get reported in the wrong context.

So that condition basically errs on the unsafe side, it prioritizes the return value check over error detection.

I think it'd be better to write:

        bool py_err = PyErr_Occurred();
        if (py_err) assert(py_value == (py_type) -1);

This prioritizes error detection over return value validation.

WDYT?

[rwgk]

I can't see any obvious issues

Same. I looked through everything several times now. Thanks!

[rwgk]

FYI — This PR is included in PR #5988

[rwgk]

Testing under #5988 was successful: Merging 1st of 4 PRs in quick succession.

Cherry-picking PYBIND11_INTERNALS_VERSION bump from #5988 into this PR.