Subprocess module in python claims 'command not found' for 'module list'?

1
votes

I am currently using the subprocess module in python for scripting purposes, and have been unable to execute the command 'module list' despite this working when I run it in shell and despite any other kind of command working fine when using the subprocess module to execute commands.

Two variations I have tried:

p = subprocess.Popen('module list', shell=True)
print(p.communicate()[0])

and 

p = Popen(["module", "list"], stdout=subprocess.PIPE)
print(p.communicate()[0])

For commands such as 'echo hello world' and even longer commands with multiple arguments, either of these formats works just fine. Is the terminal I run commands from different from the shell used to execute commands using subprocess? The error I get is as follows:

/bin/bash: line 1: module: command not found

2
pythonscriptingsubprocessenvironment-modules
Welcome to SO! What does running which module in your shell give you? - Niayesh Isky
Also, have you tried print(subprocess.check_output('module list', shell=True)) or print(subprocess.run('module list', shell=True, check=True, stdout=PIPE).stdout) (depending on your Python version)? - Niayesh Isky
Hey, thanks for your help. Using the alias 'which module' described in place of 'module' got the command working. - Farhana
@NiayeshIsky actually so it doesn't seem to work as expected for 'module load'. When I run the command it outputs something like ''/tmp/modulescript_somenumber' - Farhana
For example, I installed environment-modules on my Ubuntu distro using this tutorial, and for me, type module tells me that module is a function, which means I can't call it from Python's subprocess directly. - Niayesh Isky

2 Answers

0
votes

Based on what you've said in the comments, I believe you're going about using environment modules in Python the wrong way: There is actually a method in Modules itself to import module functionality into Python, as explained here:

>>> execfile('/usr/local/Modules/default/init/python.py')
>>> module('list')
No Modulefiles Currently Loaded.
>>> module('load','foo')
>>> module('list')
Currently Loaded Modulefiles:
  1) foo/1.0

Of course, it's not very safe to use execfile(), so I slightly prefer the import method described here (slightly altered for Python 3 support):

import os

if 'PYTHONPATH' in os.environ:
    os.environ['PYTHONPATH'] +=':'+os.environ['MODULESHOME']+"/init"
else:
    os.environ['PYTHONPATH'] = os.environ['MODULESHOME']+"/init"

from python import module
0
votes

The documentation of the Environment Modules software provides a recommendation on how to initialize the module command in Python (that should work on either Python 2 or 3):

import os
exec(open('/usr/share/Modules/init/python.py').read())

Once initialized, the module function is available and could be used in the following way:

module('sub-command', 'arg1', 'arg2', ...)

For example: 

module('load', 'foo', 'bar')
module('list')
module('avail')