796
votes

I want to remove all empty strings from a list of strings in python.

My idea looks like this:

while '' in str_list:
    str_list.remove('')

Is there any more pythonic way to do this?

13
@Ivo, neither of those statements are true. You should never modify a list that your iterating over using for x in list If you are using a while loop then it's fine. the loop demonstrated will remove empty strings until there are no more empty strings and then stop. I actually hadn't even looked at the question (just the title) but I answered with the exact same loop as a possibility! If you don't want to use comprehensions or filters for sake of memory, it's a very pythonic solution.aaronasterling
Still a very valid point to never change the list you're iterating over :)Eduard Luca
@EduardLuca if the point of iterating over a list is to change it, then that's the opposite of what you should do. You just have to be careful that you know that you do not cause an unexpected behavior by doing so.Jacqlyn
@EduardLuca, @JFA : The point is that he is NOT iterating over any list. He would if he had written something in the form for var in list:, but here, he has written while const in list:. which is not iterating over anything. it's just repeating the same code until a condition is false.Camion
You can use filter to remove the empty strings. The code should look something like this... data = list(filter(None, str_list))Jacob Ward

13 Answers

1326
votes

I would use filter:

str_list = filter(None, str_list)
str_list = filter(bool, str_list)
str_list = filter(len, str_list)
str_list = filter(lambda item: item, str_list)

Python 3 returns an iterator from filter, so should be wrapped in a call to list()

str_list = list(filter(None, str_list))
313
votes

Using a list comprehension is the most Pythonic way:

>>> strings = ["first", "", "second"]
>>> [x for x in strings if x]
['first', 'second']

If the list must be modified in-place, because there are other references which must see the updated data, then use a slice assignment:

strings[:] = [x for x in strings if x]
76
votes

filter actually has a special option for this:

filter(None, sequence)

It will filter out all elements that evaluate to False. No need to use an actual callable here such as bool, len and so on.

It's equally fast as map(bool, ...)

32
votes
>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']

>>> ' '.join(lstr).split()
['hello', 'world']

>>> filter(None, lstr)
['hello', ' ', 'world', ' ']

Compare time

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
4.226747989654541
>>> timeit('filter(None, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.0278358459472656

Notice that filter(None, lstr) does not remove empty strings with a space ' ', it only prunes away '' while ' '.join(lstr).split() removes both.

To use filter() with white space strings removed, it takes a lot more time:

>>> timeit('filter(None, [l.replace(" ", "") for l in lstr])', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
18.101892948150635
20
votes

Reply from @Ib33X is awesome. If you want to remove every empty string, after stripped. you need to use the strip method too. Otherwise, it will return the empty string too if it has white spaces. Like, " " will be valid too for that answer. So, can be achieved by.

strings = ["first", "", "second ", " "]
[x.strip() for x in strings if x.strip()]

The answer for this will be ["first", "second"].
If you want to use filter method instead, you can do like
list(filter(lambda item: item.strip(), strings)). This is give the same result.

14
votes

Instead of if x, I would use if X != '' in order to just eliminate empty strings. Like this:

str_list = [x for x in str_list if x != '']

This will preserve None data type within your list. Also, in case your list has integers and 0 is one among them, it will also be preserved.

For example,

str_list = [None, '', 0, "Hi", '', "Hello"]
[x for x in str_list if x != '']
[None, 0, "Hi", "Hello"]
8
votes

Depending on the size of your list, it may be most efficient if you use list.remove() rather than create a new list:

l = ["1", "", "3", ""]

while True:
  try:
    l.remove("")
  except ValueError:
    break

This has the advantage of not creating a new list, but the disadvantage of having to search from the beginning each time, although unlike using while '' in l as proposed above, it only requires searching once per occurrence of '' (there is certainly a way to keep the best of both methods, but it is more complicated).

6
votes

Use filter:

newlist=filter(lambda x: len(x)>0, oldlist) 

The drawbacks of using filter as pointed out is that it is slower than alternatives; also, lambda is usually costly.

Or you can go for the simplest and the most iterative of all:

# I am assuming listtext is the original list containing (possibly) empty items
for item in listtext:
    if item:
        newlist.append(str(item))
# You can remove str() based on the content of your original list

this is the most intuitive of the methods and does it in decent time.

6
votes

As reported by Aziz Alto filter(None, lstr) does not remove empty strings with a space ' ' but if you are sure lstr contains only string you can use filter(str.strip, lstr)

>>> lstr = ['hello', '', ' ', 'world', ' ']
>>> lstr
['hello', '', ' ', 'world', ' ']
>>> ' '.join(lstr).split()
['hello', 'world']
>>> filter(str.strip, lstr)
['hello', 'world']

Compare time on my pc

>>> from timeit import timeit
>>> timeit('" ".join(lstr).split()', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
3.356455087661743
>>> timeit('filter(str.strip, lstr)', "lstr=['hello', '', ' ', 'world', ' ']", number=10000000)
5.276503801345825

The fastest solution to remove '' and empty strings with a space ' ' remains ' '.join(lstr).split().

As reported in a comment the situation is different if your strings contain spaces.

>>> lstr = ['hello', '', ' ', 'world', '    ', 'see you']
>>> lstr
['hello', '', ' ', 'world', '    ', 'see you']
>>> ' '.join(lstr).split()
['hello', 'world', 'see', 'you']
>>> filter(str.strip, lstr)
['hello', 'world', 'see you']

You can see that filter(str.strip, lstr) preserve strings with spaces on it but ' '.join(lstr).split() will split this strings.

6
votes

Keep in mind that if you want to keep the white spaces within a string, you may remove them unintentionally using some approaches. If you have this list

['hello world', ' ', '', 'hello'] what you may want ['hello world','hello']

first trim the list to convert any type of white space to empty string:

space_to_empty = [x.strip() for x in _text_list]

then remove empty string from them list

space_clean_list = [x for x in space_to_empty if x]
3
votes

Sum up best answers:

1. Eliminate emtpties WITHOUT stripping:

That is, all-space strings are retained:

slist = list(filter(None, slist))

PROs:

  • simplest;
  • fastest (see benchmarks below).

2. To eliminate empties after stripping ...

2.a ... when strings do NOT contain spaces between words:

slist = ' '.join(slist).split()

PROs:

  • small code
  • fast (BUT not fastest with big datasets due to memory, contrary to what @paolo-melchiorre results)

2.b ... when strings contain spaces between words?

slist = list(filter(str.strip, slist))

PROs:

  • fastest;
  • understandability of the code.

Benchmarks on a 2018 machine:

## Build test-data
#
import random, string
nwords = 10000
maxlen = 30
null_ratio = 0.1
rnd = random.Random(0)                  # deterministic results
words = [' ' * rnd.randint(0, maxlen)
         if rnd.random() > (1 - null_ratio)
         else
         ''.join(random.choices(string.ascii_letters, k=rnd.randint(0, maxlen)))
         for _i in range(nwords)
        ]

## Test functions
#
def nostrip_filter(slist):
    return list(filter(None, slist))

def nostrip_comprehension(slist):
    return [s for s in slist if s]

def strip_filter(slist):
    return list(filter(str.strip, slist))

def strip_filter_map(slist): 
    return list(filter(None, map(str.strip, slist))) 

def strip_filter_comprehension(slist):  # waste memory
    return list(filter(None, [s.strip() for s in slist]))

def strip_filter_generator(slist):
    return list(filter(None, (s.strip() for s in slist)))

def strip_join_split(slist):  # words without(!) spaces
    return ' '.join(slist).split()

## Benchmarks
#
%timeit nostrip_filter(words)
142 µs ± 16.8 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

%timeit nostrip_comprehension(words)
263 µs ± 19.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter(words)
653 µs ± 37.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_map(words)
642 µs ± 36 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_comprehension(words)
693 µs ± 42.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_filter_generator(words)
750 µs ± 28.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

%timeit strip_join_split(words)
796 µs ± 103 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
0
votes

match using a regular expression and a filter

lstr = ['hello', '', ' ', 'world', ' ']
r=re.compile('^[A-Za-z0-9]+')
results=list(filter(r.match,lstr))
print(results)
-3
votes

For a list with a combination of spaces and empty values, use simple list comprehension -

>>> s = ['I', 'am', 'a', '', 'great', ' ', '', '  ', 'person', '!!', 'Do', 'you', 'think', 'its', 'a', '', 'a', '', 'joke', '', ' ', '', '?', '', '', '', '?']

So, you can see, this list has a combination of spaces and null elements. Using the snippet -

>>> d = [x for x in s if x.strip()]
>>> d
>>> d = ['I', 'am', 'a', 'great', 'person', '!!', 'Do', 'you', 'think', 'its', 'a', 'a', 'joke', '?', '?']