Skip to content

[3.11] gh-94526: getpath_dirname() no longer encodes the path (GH-97645) #97677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 30, 2022
Merged

[3.11] gh-94526: getpath_dirname() no longer encodes the path (GH-97645) #97677

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
gh-94526: getpath_dirname() no longer encodes the path (GH-97645) 
Fix the Python path configuration used to initialized sys.path at
Python startup. Paths are no longer encoded to UTF-8/strict to avoid
encoding errors if it contains surrogate characters (bytes paths are
decoded with the surrogateescape error handler).

getpath_basename() and getpath_dirname() functions no longer encode
the path to UTF-8/strict, but work directly on Unicode strings. These
functions now use PyUnicode_FindChar() and PyUnicode_Substring() on
the Unicode path, rather than strrchr() on the encoded bytes string.
(cherry picked from commit 9f2f1dd)

Co-authored-by: Victor Stinner <vstinner@python.org>
  • Loading branch information
@vstinner @miss-islington
vstinner authored and miss-islington committed Sep 30, 2022
commit 0aa7e74dd4528320b747ea4c7cc11bc3782c19fa
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2022-09-29-15-19-29.gh-issue-94526.wq5m6T.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Fix the Python path configuration used to initialized :data:`sys.path` at
Python startup. Paths are no longer encoded to UTF-8/strict to avoid encoding
errors if it contains surrogate characters (bytes paths are decoded with the
surrogateescape error handler). Patch by Victor Stinner.
23 changes: 14 additions & 9 deletions Modules/getpath.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,27 +82,32 @@ getpath_abspath(PyObject *Py_UNUSED(self), PyObject *args)
static PyObject *
getpath_basename(PyObject *Py_UNUSED(self), PyObject *args)
{
const char *path;
if (!PyArg_ParseTuple(args, "s", &path)) {
PyObject *path;
if (!PyArg_ParseTuple(args, "U", &path)) {
return NULL;
}
const char *name = strrchr(path, SEP);
return PyUnicode_FromString(name ? name + 1 : path);
Py_ssize_t end = PyUnicode_GET_LENGTH(path);
Py_ssize_t pos = PyUnicode_FindChar(path, SEP, 0, end, -1);
if (pos < 0) {
return Py_NewRef(path);
}
return PyUnicode_Substring(path, pos + 1, end);
}


static PyObject *
getpath_dirname(PyObject *Py_UNUSED(self), PyObject *args)
{
const char *path;
if (!PyArg_ParseTuple(args, "s", &path)) {
PyObject *path;
if (!PyArg_ParseTuple(args, "U", &path)) {
return NULL;
}
const char *name = strrchr(path, SEP);
if (!name) {
Py_ssize_t end = PyUnicode_GET_LENGTH(path);
Py_ssize_t pos = PyUnicode_FindChar(path, SEP, 0, end, -1);
if (pos < 0) {
return PyUnicode_FromStringAndSize(NULL, 0);
}
return PyUnicode_FromStringAndSize(path, (name - path));
return PyUnicode_Substring(path, 0, pos);
}


Expand Down