Running Conda commands in NodeJS

2
votes

I can't run a Conda command using exec with my NodeJS app.

var conda_path = '~/miniconda3/bin/conda'

var cmd = conda_path + ' init bash & ' + conda_path + ' activate XYZ'
exec(command,
    function(error, stdout, stderr){

    }
);

I get the following error:

/bin/sh: /Users/username/Desktop/repos/project/XYZ: is a directory

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run

$ conda init <SHELL_NAME>

Currently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

I can find no related information online regarding running conda commands from NodeJS app.

How can I make this work?

2
node.jsshellconda
If you need to run a command or script in a Conda environment, consider using conda run as shown in this answer, which circumvents having to manually activate. If you are trying to manipulate Conda envs programmatically, then I suppose that's a different story. – merv
I might try it out. I figured out a working solution tho. – vaid
Now that I see your command, yes you should definitely switch to conda run. Also, conda init is really only meant to be run once and has a user-level side-effect of editing the shell initialization script - fortunately it's idempotent, but it's a waste here. You should actually look at the code it generates to see how to initialize a shell session. – merv

2 Answers

1
votes

conda run

Conda provides the conda run command for executing programs or running scripts inside an environment without having to manually activate it. In your case, you only need one command then, e.g.,

var conda_exec = "C:\\anaconda\\Scripts\\conda.exe"
var env_name = "XYZ"
var sub_cmd = "XYZ arg1 <some_file_input> arg2 arg3 arg4 <some_file_output>"

var command = [conda_exec, 'run', '-n', env_name, sub_cmd].join(" ")

...

Be aware that conda run currently does not return I/O results until the very end. This means you won't be able to measure progress of the executing command, and also that you shouldn't use this if you expect a large return (i.e., won't fit in memory).

Suggestion: Consider naming the env XYZ_env or even include versioning (e.g., XYZ_v0_1_env), so you distinguish between XYZ the env and XYZ the program.

1
votes

t seems to work using the & character to run multiple commands.

I also needed to specify the path to conda.sh which resides inside /users/username/miniconda??/etc/profile.d/.

This file can be copied to any location, e.g your NodeJS app's root folder.

Here's the working code:

Windows 10

var commands = [
        'C:\\anaconda\\Scripts\\activate.bat C:\\anaconda',
        'conda activate XYZ_v0_1_env',
        'XYZ arg1  <some_file_input> arg2 arg3 arg4 <some_file_output>',
    ]

exec(commands.join(' & '),
    function(error, stdout, stderr){
        console.log(error)
        console.log(stdout)
        console.log(stderr)
    }
);

OSX

const exec = require('child_process').exec;

var conda_path = __dirname + '/conda.sh'

var commands = [
        conda_path,
        conda_path + ' init',
        conda_path + ' activate XYZ_v0_1_env',
        'XYZ arg1  <some_file_input> arg2 arg3 arg4 <some_file_output>',
    ]

exec(commands.join(' & '),
    function(error, stdout, stderr){
        console.log(error)
        console.log(stdout)
        console.log(stderr)
    }
);

EDIT: Using spawn instead of exec to read the live output of the process, i.e to generate live progress:

var commands = [
        'C:\\anaconda\\Scripts\\activate.bat C:\\anaconda',
        'conda activate XYZ_v0_1_env',
        'XYZ arg1  <some_file_input> arg2 arg3 arg4 <some_file_output>',
    ]

var spawn_ = spawn(commands.join('&'), { shell: true });

spawn_.stdout.on('data', function (data) {
    //do something
});

spawn_.stderr.on('data', function (data) {
    //do something
});

spawn_.on('exit', function (code) {
    //do something
});