8
votes

I'm currently working on a project involving tests to be run at a remote host (bash). Unfortunately, the remote python interpreter does not respect the available site-packages (it's an embedded one: abaqus python (2.6)). Using the PYTHONPATH variable, however, works to specify local installations and makes additional packages available. Hence, on the remote machine I simply add a respective line to my .bashrc file.

Unfortunately, when distributing tests using xdist only a "bare" bash is invoked, without any profile specific rcs loaded. Thus, the tests fail with some import errors as argparse, which is required by pytest is not available.

Is there a way the setup a remote host before it starts executing any pytest code (which requires argparse)? In other words is there a way to add environment variables on the hosts before the pytest imports start?

I tried using fixtures with session scope and autouse=True which (of course) didn't work. Moreover I tried something like

# in conftest.py
import sys    
def pytest_configure_node():
    sys.path.insert(1, "/somepath/")
    print sys.path

This looks like it is executed on the remote host but sys.path remains the hosts one and the argparse module still cannot be imported.

I start the tests using

py.test --tx ssh=user@server//python="abaqus613 python" -vs --dist=each --rsyncdir foo

This starts the right python interpreter (Python 2.6.2 for Abaqus 6.13-2), but fails with

ImportError: No module named argparse
2
What command are you using to run xdist? - Reut Sharabani
@ReutSharabani please see my updated question. - Jakob
what line are yout using to set PYTHONPATH in .bashrc when testing manually? - Reut Sharabani
something like export PYTHONPATH=/path/to/argparse/library - Jakob
Would something like adding: --rsyncdir /local/path/to/argparse argparse work? It should sync argparse to the remote directory. I'm not sure about how you can do this remotely without breaking stuff. When you say bare bash, what do you mean? Did you add the PYTHONPATH in the logged user's env? Can you do it system-wide using /etc/environment or ~/.profile? I may try reproducing the error later. - Reut Sharabani

2 Answers

1
votes

I finally figured out a quite hackish but reasonably working way. It is possible to specify a series of commands in the python call, hence I source a script which setups the environment before calling abaqus python.

The setup script (setup.sh), located on the remote machine, looks like:

export PYTHONPATH=/path/to/libraries

and the complete call is now

py.test --tx ssh=user@server//python="source setup.sh;abaqus613 python" -vs --dist=each --rsyncdir foo

This way I get the necessary imports working in abaqus python.

0
votes

My plugin https://github.com/pytest-dev/pytest-cloud should help you

it automatically rsyncs python virtual environment to the slaves, one requirement is that your virtualenv folder should be inside of the folder where test folder is

after rsync it also activates the virtual environment so all your dependencies are there properly