The `datetime` C API is not thread-safe

[ZeroIntensity]

Bug report

Bug description:

datetime has a C API that we expose through a global variable. This does not work well under multiple threads. For example, if PyDateTime_IMPORT fails in one thread while another thread is invoking the datetime C API, then that will set PyDateTimeAPI to NULL, leading to NULL pointer dereferences in the original thread.

This global variable also forces us to do some nasty hacks with subinterpreters because it exposes static types. I helped fix this a few months ago by making _datetime static (see this thread), but there are probably other issues that I haven't found yet.

I think a bandaid fix would be to make PyDateTimeAPI a thread-local variable, but I suspect that would break a lot of things, nor would that fix this for subinterpreters. More generally, we could deprecate PyDateTime_IMPORT and PyDateTimeAPI in favor of letting users manage the pointer on their own, but I don't know if we can do this without too many code changes (see #141543 and #83785 for the docs side).

cc @pganssle

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

[sharktide]

Has anyone found any examples of this on GitHub? I imagine that this might happen because it seems like it would be common to use dateline to measure the time of something executing but at the same time why use datetime itself? There has to be better libs or inbuilt functions for most use cases

[ZeroIntensity]

Yes.