email.generator.Generator does not correctly identify a boundary phrase when using CRLF line endings.

[henryivesjones]

Bug report

Bug description:

The boundary detection regexp cre = cls._compile_re('^--' + re.escape(b) + '(--)?\r?$', re.MULTILINE) does not correctly identify the boundary phrase when using linesep='\r\n'.

This is because the re.MULTILINE flag does not identify CRLF endings as a line ending.

CRLF line endings are extremely common and are even in the SMTP policy as defined in email.policy.SMTP.

Minimal Example:

from email.generator import Generator
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from io import StringIO
import random
import sys

# Generate a boundary token in the same way as _make_boundary
token = random.randrange(sys.maxsize)

def _patch_random_randrange(*args, **kwargs):
    return token

random.randrange = _patch_random_randrange


boundary = Generator._make_boundary(text=None)
boundary_in_part = (
    "this goes before the boundary\n--"
    + boundary
    + "\nthis goes after\n"
)
msg = MIMEMultipart()
msg.attach(MIMEText(boundary_in_part))
Generator(StringIO()).flatten(msg, linesep="\r\n")

# .0 is appended if the boundary was found.
assert msg.get_boundary() == boundary + ".0"

Test Case:

# test_email.test_generator.TestGeneratorBase
    def _test_boundary_detection(self, linesep):
        # Generate a boundary token in the same way as _make_boundary
        token = random.randrange(sys.maxsize)

        def _patch_random_randrange(*args, **kwargs):
            return token

        with test.support.swap_attr(
            random, "randrange", _patch_random_randrange
        ):
            boundary = self.genclass._make_boundary(text=None)
            boundary_in_part = (
                "this goes before the boundary\n--"
                + boundary
                + "\nthis goes after\n"
            )
            msg = MIMEMultipart()
            msg.attach(MIMEText(boundary_in_part))
            self.genclass(self.ioclass()).flatten(msg, linesep=linesep)
            # .0 is appended if the boundary was found.
            self.assertEqual(msg.get_boundary(), boundary + ".0")

    def test_lf_boundary_detection(self):
        self._test_boundary_detection("\n")

    def test_crlf_boundary_detection(self):
        self._test_boundary_detection("\r\n")

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-148192: Fix Generator._make_boundary behavior with CRLF line endings. #148193