Repository navigation
Improve speed of stdlib functions by replacing re uses #130167
Description
Activity
- changed the title
[-]Improve speed of functions that use `re` of various stdlib modules[/-][+]Improve speed of functions by replacing `re` uses[/+]on Feb 16, 2025 - changed the title
[-]Improve speed of functions by replacing `re` uses[/-][+]Improve speed of stdlib functions by replacing `re` uses[/+]on Feb 16, 2025 - RE patterns are compiled once and then cached. So less relevant the more times a pattern is used.
- Please list RE patterns, replacements, and RE timings that do and do not include compile time. This will indicate max and min time saved, depending on reuses. Should a table be added to Regular Expression HOWTO, Use String Methods. Are there string methods other than
replaceandtranslateto consider? - This is known, and there have been cases where we delay re imports until needed, if ever, to reduce startup time. But re import cannot be removed entirely unless all uses are removed.
- Core developers should know regular expressions, though the point applies to some other contributors.
Unless you find and list examples of useful replacements in an stdlib module, this is not properly an issue. If you have or do, make a PR. Or suggest a revision of the HOWTO section.
@terryjreedy thank you for your comment and I know about the counterarguments you brought up but it doesn't change the increase in speed and the fact that we can get rid of unnecessary imports (sometimes or often, I don't know the statistics yet :)
I've only submitted one PR so far but I'm open to finding more examples since we use
rein the stdlib all the timeI removed the paragraph for new contributions as this issue is not necessary something we want to address immediately. Before opening a PR, benchmarks must be given and be convincing. If we lose the readability at the cost of an improved import time, I'm not sure it's worth unless we're like 5 or 6 times faster and if it's an important module that is usually imported and that
reis not usually imported.If we lose the readability at the cost of an improved import time, I'm not sure it's worth unless we're like 5 or 6 times faster and if it's an important module that is usually imported and that
reis not usually imported.Usually, i think, the readability of code without regular expressions is better. And from a performance standpoint, I think the speed of the function is the main improvement, although import time is also important.
you can find
import revery often and only one useIf you can remove
rewith simpler alternatives, then I'm fine. What I don't want to see is essentially PRs that would move lots ofreimports to local ones if they hurt readability and don't improve start up time by much.Note that it's also important to possibly plan for future extensions of the regex usage. Like getting the matched substring could be useful later. Now, I'm not against improving the various
reusages whenever possible but benchmarks must always be given (with sufficient coverage).Reacted by Semyon MorozI've restored your guidelines but added the requirements of having benchmarks
Reacted by Semyon Moroz- added a commit that references this issue
on Mar 31, 2025 - added a commit that references this issue
on May 1, 2025 - added a commit that references this issue
on Dec 30, 2025 - added a commit that references this issue
on Dec 30, 2025
We can often find the module
rein the standard library modules but it can be replaced (if it is possible). I don't suggest removing it everywhere, there are places where its use is appropriate, but there are also places where it is an unnecessary solution and leads to unpleasant consequences (they can be found below)Cons of regular expressions and reasons to replace regular expressions with functions and methods:
import rewhich will affect import timeImportant
For those who want to work on the issue, please:
pyperf,hyperfine, andtunatogether with-X importtimeto compare import times and execution time.gh-130167: Improve speed of `module.function` by replacing `re`Linked PRs
difflib.IS_LINE_JUNKby replacingre#130170inspect.formatannotationby replacingre#130242ftplib.parse150by replacingre#130243textwrap.dedent()#131919textwrap.dedent()optimization #131925textwrap.{de,in}dent#131924_pydecimal._all_zerosand_pydecimal._exact_halfby replacingre#132065textwrap.dedent#132666textwrap.{de,in}dent(GH-131924) #143292