Skip to content

Implement RFC 6855 (IMAP Support for UTF-8) in imaplib. #65999

Closed

Description

@zvyn
mannequin
BPO 21800
Nosy @warsaw, @pitrou, @ericvsmith, @bitdancer, @soltysh, @zvyn
Files
  • imaplib_utf8_no_doc.patch: Proposed solution.
  • imaplib_utf8_issue21800.patch
  • issue21800.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2015-05-16.18:06:45.846>
    created_at = <Date 2014-06-17.23:28:22.752>
    labels = ['type-feature', 'expert-email']
    title = 'Implement RFC 6855 (IMAP Support for UTF-8) in imaplib.'
    updated_at = <Date 2015-05-16.18:06:45.845>
    user = 'https://github.com/zvyn'

    bugs.python.org fields:

    activity = <Date 2015-05-16.18:06:45.845>
    actor = 'r.david.murray'
    assignee = 'none'
    closed = True
    closed_date = <Date 2015-05-16.18:06:45.846>
    closer = 'r.david.murray'
    components = ['email']
    creation = <Date 2014-06-17.23:28:22.752>
    creator = 'zvyn'
    dependencies = []
    files = ['35808', '39284', '39324']
    hgrepos = []
    issue_num = 21800
    keywords = ['patch']
    message_count = 10.0
    messages = ['222000', '236713', '242538', '242620', '242622', '242648', '242662', '242778', '242873', '242876']
    nosy_count = 10.0
    nosy_names = ['barry', 'pitrou', 'eric.smith', 'r.david.murray', 'jesstess', 'BreamoreBoy', 'python-dev', 'maciej.szulik', 'zvyn', 'H\xc3\xa5kan L\xc3\xb6vdahl']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue21800'
    versions = ['Python 3.5']

    Activity

    1. zvyn commented on Jul 1, 2014

      zvynmannequin
      MannequinAuthor

      I made a patch implementing the following changes to the IMAP4 class:

      • add a method 'enable_UTF8_accept()' sending "ENABLE UTF8=ACCEPT" to the server and setting internal encoding to UTF-8
      • use the UTF8 extencion in the 'append()' method if the internal encoding is UTF-8
      • add a keyword argument 'enable_UTF8=False' to the init method to trigger 'enable_UTF8_accept()' as soon as the authentication is done
      • always use UTF-8 for encoding credentials in authentication (before encoding it to base64)

      Does this look like a good idea to you? (I'll make a patch including docs when we agree on the API.)

    2. BreamoreBoy commented on Feb 26, 2015

      BreamoreBoymannequin
      Mannequin

      The patch contains changes to code and tests have been added so can we have a formal review please.

    3. bitdancer commented on May 4, 2015

      @bitdancer
      Member

      Here is an updated patch based on Milan's work, including docs. I've tweaked the API slightly: no dedicated method for doing the enable (instead it is inlined in authenticate), I added 'enable' to the exposed API (with a doc caveat about not using it for UTF8=ACCEPT), added None as a valid value for enable_UTF8 with the meaning "enable if possible" (True is now "must enable"), and exposed a utf8_enabled attribute so the program can easily tell what mode to use when specifying enable_UTF8=None. Someday, None should become the default. "None" is not the best choice for value, especially when it is not the default, so perhaps someone could suggest better values for that keyword.

      It would be great if Milan or Maciej could give me a review (or anyone else who feels like it). I want to get this in before the beta deadline.

    4. soltysh commented on May 5, 2015

      @soltysh

      David I did the review and there's one thing that worries me the most, actually two:

      1. changing the usual meaning of None in the IMAP's __init__ method, where None has the same meaning as True, where I think it should be the opposite.
      2. I'm not sure we want to have UTF8 enabled based on the init's flag. I've seen our IMAP library as a wrapper around protocol itself. Whereas the user must be aware of required steps needed to proceed. In this case enabling UTF8 support is just the next command the client can, but doesn't have to sent directly, but only in AUTH state.
    5. bitdancer commented on May 5, 2015

      @bitdancer
      Member

      Well, the problem with that is that we then have to parse the capability to see if it is utf8 that is being enabled. I don't like that as an API, it feels fragile. Since capabilities cannot later be disabled, there's no functional reason to keep it separate. However, it would solve the problem of values to use in the init flag, and would remove the caveat that you shouldn't use UTF8=ENABLE in an explicit enable call, so perhaps it is best after all.

      Do you have any interest in updating the patch? I won't be able to get back to it until this weekend.

    6. soltysh commented on May 6, 2015

      @soltysh

      Yes, I can update that (that IMAP testing bug - http://bugs.python.org/issue22137, is taking me longer than I expected it ;)).
      I just want to make sure if I understand you correctly what's needs to be done is removing the utf8_enable code from init, we will enable ascii by default and only explicit call to enable method will enable it, am I missing something?

    7. bitdancer commented on May 6, 2015

      @bitdancer
      Member

      An explicit call to enable with an argument string that contains 'UTF8=ACCEPT'. I did not go far enough through the BNF to determine if enable can be passed more than one capability at a time, but I suspect it can. In which case we should factor out the capability parsing such that we can reuse it for parsing the enable argument.

    8. soltysh commented on May 8, 2015

      @soltysh

      David, I've changed according to your suggestion, appreciate review.

    9. python-dev commented on May 10, 2015

      python-devmannequin
      Mannequin

      New changeset 195343b5e64f by R David Murray in branch 'default':
      bpo-21800: Add RFC 6855 support to imaplib.
      https://hg.python.org/cpython/rev/195343b5e64f

    10. bitdancer commented on May 11, 2015

      @bitdancer
      Member

      Thanks, Maciek (and Milan).

      I tweaked your patch slightly (mostly doc changes...I moved the discussion of the utf8 RFC into the enable method only, and added back the docs for utf8_enabled). I made some review comments about the changes other than that doc reorg that I made before commit, just FYI.

    11. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions