Skip to content

More correct string truncating in PyUnicode_FromFormat() #70278

Open

Description

@serhiy-storchaka
BPO 26090
Nosy @gvanrossum, @vstinner, @ezio-melotti, @serhiy-storchaka

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-01-12.09:54:14.845>
labels = ['interpreter-core', 'type-feature']
title = 'More correct string truncating in PyUnicode_FromFormat()'
updated_at = <Date 2016-06-21.07:22:59.885>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-06-21.07:22:59.885>
actor = 'Drekin'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2016-01-12.09:54:14.845>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 26090
keywords = []
message_count = 5.0
messages = ['258092', '258095', '258108', '258111', '258118']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'vstinner', 'ezio.melotti', 'serhiy.storchaka', 'Drekin']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue26090'
versions = ['Python 3.6']

Linked PRs

Activity

  1. serhiy-storchaka commented on Jan 12, 2016

    @serhiy-storchaka
    MemberAuthor

    The C code often uses %.<number><format> in PyUnicode_FromFormat(). %.200s protects from unlimited output when broken pointer points on random non-null-terminated data. %.200R is used to limit the size of human-readable messages.

    In all these case formatted string can look well-formed with short data, but mis-formed (not closed quote, truncated backslash escaping or � decoded from truncated UTF-8 sequence) with long data.

    I propose to make truncating in PyUnicode_FromFormat() more smart.

    1. Truncated %R should keep at least one end character (the quote or ">").
    2. Truncated output should include "..." or "[...]" as truncating sign.
    3. \c, \OOO, \xXX, \uXXXX, and \UXXXXXXXX should not be truncated. It is better to omit these sequences at all (cut the string before them) that output them truncated.
    4. Doesn't truncate UTF-8 sequence inside a character for %s.
  2. vstinner commented on Jan 12, 2016

    @vstinner
    Member

    See my old issue bpo-10833 which proposed to *remove* the arbitrary limit
    on strings. It was rejected.

  3. gvanrossum commented on Jan 12, 2016

    @gvanrossum
    Member

    Could we make this feature available at the Python level too? It sounds
    really useful.

    --Guido (mobile)
    On Jan 12, 2016 2:01 AM, "STINNER Victor" <report@bugs.python.org> wrote:

    STINNER Victor added the comment:

    See my old issue bpo-10833 which proposed to *remove* the arbitrary limit
    on strings. It was rejected.

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue26090\>


  4. serhiy-storchaka commented on Jan 12, 2016

    @serhiy-storchaka
    MemberAuthor

    I think we can make this feature available with classic formatting '%.100r', but with new formatting '{0:.100!r}' (especially with f-strings) this can be not so easy.

  5. gvanrossum commented on Jan 12, 2016

    @gvanrossum
    Member

    Well it seems a little odd to spend effort on a corner case of the C-level
    error messages if we can't even replicate it in pure Python.

  6. transferred this issue fromon Apr 10, 2022
  7. added a commit that references this issue on Jun 11, 2024
  8. serhiy-storchaka commented on Jun 11, 2024

    @serhiy-storchaka
    MemberAuthor

    #120365 solves item 4. It is the simplest of the mentioned problems. It only exists at the C level.

  9. vstinner commented on Jun 12, 2024

    @vstinner
    Member

    Truncated output should include "..." or "[...]" as truncating sign.

    I suggest adding (...) to indicate that the string is truncated.

  10. added a commit that references this issue on Jun 24, 2024
  11. added a commit that references this issue on Jun 30, 2024
  12. added a commit that references this issue on Jul 11, 2024
  13. added a commit that references this issue on Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions