14
votes

On Python 3.7 (tested on Windows 64 bits), the replacement of a string using the RegEx .* gives the input string repeated twice!

On Python 3.7.2:

>>> import re
>>> re.sub(".*", "(replacement)", "sample text")
'(replacement)(replacement)'

On Python 3.6.4:

>>> import re
>>> re.sub(".*", "(replacement)", "sample text")
'(replacement)'

On Python 2.7.5 (32 bits):

>>> import re
>>> re.sub(".*", "(replacement)", "sample text")
'(replacement)'

What is wrong? How to fix that?

1
Clearly a bug. Not sure what kind of answer you're hoping to get. - Aran-Fey
Only happens with .* (or .*$), not with .+ or ^.*. And, well, you have an infinite number of zero-byte strings at the end of your match, so you might as well be glad that you get only one repetition. :) - Charles Duffy
@Aran-Fey, since .* is greedy, I expect to get '(replacement)' only once. Why two? - Laurent LAPORTE
Is it really a bug in Python though? Even the PCRE regex behaves exactly the same. Also note the Python variant behaves the same on regex101 as well, but they could be using 3.7.2 also... oddly enough if you had nothing in there, the replacement only happens once. I'm guessing beginning of string ^ and end of string $ counts as two empty space characters? - r.ook

1 Answers

17
votes

This is not a bug, but a bug fix in Python 3.7 from the commit fbb490fd2f38bd817d99c20c05121ad0168a38ee.

In regex, a non-zero-width match moves the pointer position to the end of the match, so that the next assertion, zero-width or not, can continue to match from the position following the match. So in your example, after .* greedily matches and consumes the entire string, the fact that the pointer is then moved to the end of the string still actually leaves "room" for a zero-width match at that position, as can be evident from the following code, which behaves the same in Python 2.7, 3.6 and 3.7:

>>> re.findall(".*", 'sample text')
['sample text', '']

So the bug fix, which is about replacement of a zero-width match right after a non-zero-width match, now correctly replaces both matches with the replacement text.