python - Remove repeating characters from words

9
votes

I was wondering what is the best way to convert something like "haaaaapppppyyy" to "haappyy".

Basically, when parsing slang, people sometimes repeat characters for added emphasis.

I was wondering what the best way to do this is? Using set() doesn't work because the order of the letters is obviously important.

Any ideas? I'm using Python + nltk.

4
pythonnlpnltk
Essentially, I'm only doing this if a letter is repeated > 2 times in a row in the same word, so realistically the word would acutally be "haappy" by that logic. However, I could use enchant to spellcheck. - user179169
oh sorry, forgot I repeated the "y", yes, it would be "haappyy" - user179169
Matching against a dictionary is also a nice problem: find the shortest real word you can get by dropping repetitions. It's not trivial since some words have two or more double letters (think "bookkeeper"). - alexis

4 Answers

32
votes

It can be done using regular expressions:

>>> import re
>>> re.sub(r'(.)\1+', r'\1\1', "haaaaapppppyyy")     
'haappyy'

(.)\1+ repleaces any character (.) followed by one or more of the same character (because of the backref \1 it must be the same) by twice the character.

8
votes

You can squash multiple occurrences of letters with itertools.groupby:

>>> ''.join(c for c, _ in groupby("haaaaapppppyyy"))
'hapy'

Similarly, you can get haappyy from groupby with

>>> ''.join(''.join(s)[:2] for _, s in groupby("haaaaapppppyyy"))
'haappyy'
1
votes

You should do it without reduce or regexps:

>>> s = 'hhaaaaapppppyyy'
>>> ''.join(['' if i>1 and e==s[i-2] else e for i,e in enumerate(s)])
'haappyy'

The number of repetitions are hardcoded to >1 and -2 above. The general case:

>>> reps = 1
>>> ''.join(['' if i>reps-1 and e==s[i-reps] else e for i,e in enumerate(s)])
'hapy'
0
votes

This is one way of doing it (limited to the obvious constraint that python doesn't speak english).

>>> s="haaaappppyy"
>>> reduce(lambda x,y: x+y if x[-2:]!=y*2 else x, s, "")
'haappyy'