gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter #136877

matthieucan · 2025-07-20T15:59:25Z

With those changes, the MIME parameter parser discards parameters with an invalid section number that uses a digit not convertible to integer such as super-script "²" or "𐩃" (Kharosthi number).

For backwards compatibility, keep accepting non-ASCII digits that can be converted to integers, such as NKO digits.

Before:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    reg('Content-Disposition', 'inline; foo*²=bar')
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 604, in __call__
    return self[name](name, value)
           ~~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 192, in __new__
    cls.parse(value, kwds)
    ~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/headerregistry.py", line 448, in parse
    kwds['parse_tree'] = parse_tree = cls.value_parser(value)
                                      ~~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2736, in parse_content_disposition_header
    disp_header.append(parse_mime_parameters(value[1:]))
                       ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2601, in parse_mime_parameters
    token, value = get_parameter(value)
                   ~~~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2464, in get_parameter
    token, value = get_section(value)
                   ~~~~~~~~~~~^^^^^^^
  File "/Users/matthieu/git/cpython/Lib/email/_header_value_parser.py", line 2417, in get_section
    section.number = int(digits)
                     ~~~^^^^^^^^
ValueError: invalid literal for int() with base 10: '²'

After:

>>> import email.headerregistry
>>> reg = email.headerregistry.HeaderRegistry()
>>> reg('Content-Disposition', 'inline; foo*0=bar')
'inline; foo="bar"'
>>> reg('Content-Disposition', 'inline; foo*X=bar')
'inline;'
>>> reg('Content-Disposition', 'inline; foo*0=bar; foo*߁=baz')
'inline; foo="barbaz"'
>>> reg('Content-Disposition', 'inline; foo*²=bar')
'inline;'

Issue: email: ValueError in get_section when parsing header with non-ASCII digit #87112

Lib/email/_header_value_parser.py

Lib/test/test_email/test__header_value_parser.py

Lib/email/_header_value_parser.py

Lib/test/test_email/test__header_value_parser.py

Lib/email/_header_value_parser.py

picnixz · 2025-07-20T17:32:09Z

Lib/email/_header_value_parser.py

        raise errors.HeaderParseError("Expected section number but "
                                      "found {}".format(value))
    digits = ''
-    while value and value[0].isdigit():
+    while value and '0' <= value[0] <= '9':


Suggested change

while value and '0' <= value[0] <= '9':

while value and ('0' <= value[0] <= '9'):

It will a bit clearer. Or you can still use a separate function to make it even cleareer. The bottleneck won't be the function call IMO.

I did that, but not the separate function. It was my understanding that @StanFromIreland was leaning towards not having an inner function

This is fine, I was against the function to check if it is in a dictionary.

Thank you! Just moved it to a separate function for extra-clarity

encukou · 2025-07-22T12:42:54Z

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts?
Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.

matthieucan · 2025-07-22T21:15:24Z

Could you also test with, for example, ߅ (NKO DIGIT FIVE), which int accepts? Given that some values worked before, IMO the HeaderParseError should only be raised if int() fails; for other non-ASCII numbers this should append an InvalidHeaderDefect.

Thank you for looking into this.

In my understanding, those are the possible scenarios:

The section number is made of ASCII digits ('0' <= d <= '9')
- Valid, remains unchanged with this PR
The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE)
- This was accepted before, without a defect
- This PR, in its current state, rejects those (they are ignored and trigger a defect)
- ➡️ ❓ @encukou Do I understand correctly that they should be accepted, and not raise a defect (even if they are not RFC-valid)? Or should they be accepted and raise a defect?
The section number is made of digits (isdigit(d)) which cannot be converted to integers (not int(d)), e.g. ²
- This used to fail with a raised exception
- This PR, in its current state, rejects those (they are ignored and trigger a defect)
The section number is not made of digits (not isdigit(d))
- Invalid, remains unchanged with this PR

encukou · 2025-07-23T08:53:36Z

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.

matthieucan · 2025-07-27T12:19:27Z

The section number is made of digits (isdigit(d)) which can be converted to integers (int(d)), e.g.߅ (NKO DIGIT FIVE). This was accepted before, without a defect

IMO, they should be accepted and raise a defect.

Thank you, I agree. This is now implemented.

matthieucan added 2 commits July 20, 2025 17:43

Only allow 0-9 digits in MIME parameter section numbers

ff4ca5c

Use gh bug nubmer

656b6fb

matthieucan requested a review from a team as a code owner July 20, 2025 15:59

bedevere-app bot added the awaiting review label Jul 20, 2025

bedevere-app bot mentioned this pull request Jul 20, 2025

email: ValueError in get_section when parsing header with non-ASCII digit #87112

Open

Add news entry

75636a5

StanFromIreland reviewed Jul 20, 2025

View reviewed changes

Lib/email/_header_value_parser.py Outdated Show resolved Hide resolved

Lib/email/_header_value_parser.py Outdated Show resolved Hide resolved

Lib/email/_header_value_parser.py Outdated Show resolved Hide resolved

picnixz changed the title ~~gh-87112: Do not fail when non 0-9 digit is used as section number in MIME header parameter~~ gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter Jul 20, 2025

picnixz reviewed Jul 20, 2025

View reviewed changes

Lib/test/test_email/test__header_value_parser.py Show resolved Hide resolved

matthieucan added 2 commits July 20, 2025 19:17

Comments from PR review, remove inner function

b15c404

Rephrase News entry

7cba42f

StanFromIreland reviewed Jul 20, 2025

View reviewed changes

Lib/email/_header_value_parser.py Outdated Show resolved Hide resolved

Lib/test/test_email/test__header_value_parser.py Outdated Show resolved Hide resolved

picnixz reviewed Jul 20, 2025

View reviewed changes

matthieucan added 2 commits July 20, 2025 19:35

Comments from PR review, improve phrasing

d2d59ee

Use separate function

54bbbb7

AA-Turner added the sprint label Jul 20, 2025

matthieucan added 2 commits July 27, 2025 14:16

Accept digits that are convertible to int for backwards compatibility

f12e424

Update news message

b364339

matthieucan changed the title ~~gh-87112: Ensure that only ASCII digits are accepted as section number in MIME header parameter~~ gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter Jul 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter #136877

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter #136877

matthieucan commented Jul 20, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz Jul 20, 2025

Uh oh!

matthieucan Jul 20, 2025

Uh oh!

StanFromIreland Jul 20, 2025

Uh oh!

matthieucan Jul 20, 2025

Uh oh!

encukou commented Jul 22, 2025

Uh oh!

matthieucan commented Jul 22, 2025

Uh oh!

encukou commented Jul 23, 2025

Uh oh!

matthieucan commented Jul 27, 2025

Uh oh!

Uh oh!

	while value and '0' <= value[0] <= '9':
	while value and ('0' <= value[0] <= '9'):

Uh oh!

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter #136877

Are you sure you want to change the base?

gh-87112: Ensure that only digits convertible to integers are accepted as section number in MIME header parameter #136877

Conversation

matthieucan commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

matthieucan Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

StanFromIreland Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

matthieucan Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

encukou commented Jul 22, 2025

Uh oh!

matthieucan commented Jul 22, 2025

Uh oh!

encukou commented Jul 23, 2025

Uh oh!

matthieucan commented Jul 27, 2025

Uh oh!

Uh oh!

matthieucan commented Jul 20, 2025 •

edited

Loading