`PyDict_Next` should not lock the dict

[colesbury]

Bug report

PyDict_Next currently wraps _PyDict_Next in a critical section. We shouldn't do this -- the locking needs to be external to the call.

1. It's not sufficient to lock the dict just for each _PyDict_Next call because we return borrowed references and because pos becomes meaningless if the dictionary gets resized or rehashed.
2. It interferes with externally locking the dict because the inner critical sections can suspend the outer ones. In other words, if the caller use a critical section to lock the dict for multiple iterations, this will break that.

cpython/Objects/dictobject.c

Lines 2883 to 2890 in 8f17d69

	PyDict_Next(PyObject *op, Py_ssize_t *ppos, PyObject **pkey, PyObject **pvalue)
	{
	int res;
	Py_BEGIN_CRITICAL_SECTION(op);
	res = _PyDict_Next(op, ppos, pkey, pvalue, NULL);
	Py_END_CRITICAL_SECTION();
	return res;
	}

cc @DinoV

Linked PRs

• gh-120858: PyDict_Next should not lock the dict #120859
• [3.13] gh-120858: PyDict_Next should not lock the dict (GH-120859) #120964