regex - Splitting out the output of ps using Python

Question

On Linux, the command ps aux outputs a list of processes with multiple columns for each stat. e.g.

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
...
postfix  22611  0.0  0.2  54136  2544 ?        S    15:26   0:00 pickup -l -t fifo -u
apache   22920  0.0  1.5 198340 16588 ?        S    09:58   0:05 /usr/sbin/httpd

I want to be able to read this in using Python and split out each row and then each column so they can be used as values.

For the most part, this is not a problem:

ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
processes = ps.split('\n')

I can now loop through processes to get each row and split it out by spaces, for example

sep = re.compile('[\s]+')
for row in processes:
    print sep.split(row)

However, the problem is that the last column, the command, sometimes has spaces in. In the example above this can be seen in command

pickup -l -t fifo -u

which would be split out as

['postfix', '22611', '0.0', '0.2', '54136', '2544', '?', 'S', '15:26', '0:00', 'pickup', '-l', '-t', 'fifo', '-u']

but I really want it as:

['postfix', '22611', '0.0', '0.2', '54136', '2544', '?', 'S', '15:26', '0:00', 'pickup -l -t fifo -u']

So my question is, how can I split out the columns but when it comes to the command column, keep the whole string as one list element rather than split out by spaces?

Answer 1

Use the second parameter to split which specifies the maximum number of fields to split the string into. I guess you can find the number by counting the number of fields in the first line, i.e. the column titles.

ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
processes = ps.split('\n')
# this specifies the number of splits, so the splitted lines
# will have (nfields+1) elements
nfields = len(processes[0].split()) - 1
for row in processes[1:]:
    print row.split(None, nfields)

Answer 2

Check out the python.psutils package.

psutil.process_iter returns a generator which you can use to iterate over all processes. p.cmdline is a list of each Process object's cmdline arguments, separated just the way you want.

You can create a dictionary of pids vs (pid,cmdline,path) with just one line and then use it anyway you want.

pid_dict = dict([(p.pid, dict([('pid',p.pid), ('cmdline',p.cmdline), ('path',p.path)]))
                 for p in psutil.process_iter()]))

Answer 3

Why don't you use PSI instead? PSI provides process information on Linux and other Unix variants.

import psi.process
for p in psi.process.ProcessTable().values(): …

Answer 4

The maxsplit optional argument to the split method might help you:

sep.split.(row, maxsplit=42)

Answer 5

Here's a nice routine and usage to get you going:

def getProcessData():
    ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
    processes = ps.split('\n')
    # this specifies the number of splits, so the splitted lines
    # will have (nfields+1) elements
    nfields = len(processes[0].split()) - 1
    retval = []
    for row in processes[1:]:
        retval.append(row.split(None, nfields))
    return retval

wantpid = int(contents[0])
pstats = getProcessData()
for ps in pstats:
    if (not len(ps) >= 1): continue
    if (int(ps[1]) == wantpid):
        print "process data:"
        print "USER              PID       %CPU        %MEM       VSZ        RSS        TTY       STAT      START TIME      COMMAND"
        print "%-10.10s %10.10s %10.10s %10.10s %10.10s %10.10s %10.10s %10.10s %10.10s  %s" % (ps[0], ps[1], ps[2], ps[3], ps[4], ps[5], ps[6], ps[7], ps[8], ps[9])