Subprocess run, check_output, Popen returns empty string when I run the script from a batch file and from Task Scheduler

Question

I have a batch file which is running a python script and in the python script, I have a subprocess function which is being ran.

I have tried subprocess.check_output, subprocess.run, subprocess.Popen, all of them returns me an empty string only when running it using a batch file.

If I run it manually or using an IDE, I get the response correctly. Below is the code for subprocess.run:

    response = subprocess.run(fileCommand, shell=True, cwd=pSetTableauExeDirectory, capture_output=True)
    self.writeInLog(' Command Response: \t' + str(response))

Response is in stdout=b''

When ran in batch file and from task scheduler:

Command Response: CompletedProcess(args='tableau refreshextract --config-file "Z:\XXX\tableau_config\SampleSuperStore.txt"', returncode=0, stdout=b'', stderr=b'')

When ran manually or in IDE:

Command Response: CompletedProcess(args='tableau refreshextract --config-file "Z:\XXX\tableau_config\SampleSuperStore.txt"', returncode=0, stdout=b'Data source refresh completed.\r\n0 rows uploaded.\r\n', stderr=b'')

Batch file which runs the python program. Parameters are parsed to the python application

SET config=SampleSuperStore.txt
CALL C:\XXX\AppData\Local\Continuum\anaconda3\Scripts\activate.bat
C:\XXX\AppData\Local\Continuum\anaconda3\python.exe Z:\XXX\pMainManual.py "%config%"

Why is that??

--Complete python code---

try:
    from pWrapper import wrapper
    import sys
except Exception as e:
    print(str(e))

class main:

    def __init__(self):
        self.tableauPath = 'C:\\Program Files\\Tableau\\Tableau 2018.3\\bin\\'
        self.tableauCommand = 'tableau refreshextract --config-file' 

    def runJob(self,argv): 
        self.manual_sProcess(argv[1])

    def manual_sProcess(self,tableauConfigFile):    
        new_wrapper = wrapper()
        new_wrapper.tableauSetup(self.tableauPath,self.tableauCommand)
        if new_wrapper.tableauConfigExists(tableauConfigFile):
            new_wrapper.tableauCommand(tableauConfigFile)           

if __name__ == "__main__":
    new_main = main()
    new_main.runJob(sys.argv)

Wrapper class:

def tableauCommand(self,tableauConfigFile):    
    command = self.setTableauExeDirectory + ' ' + self.refreshConfigCommand + ' "' + tableauConfigFile + '"'
    self.new_automateTableauExtract.runCommand(tableauConfigFile,command,self.refreshConfigCommand,self.tableauFilePath,self.setTableauExeDirectory)

Automate Class:

def runCommand(self,pConfig,pCommand,pRefreshConfigCommand,pFilePath,pSetTableauExeDirectory):
    try:
        fileCommand = pRefreshConfigCommand + ' "' + pFilePath + '"'
        response = subprocess.run(fileCommand, shell=True, cwd=pSetTableauExeDirectory, capture_output=True)
        self.writeInLog(' Command Response: \t' + str(response))
    except Exception as e:
        self.writeInLog('Exception in function runCommand: ' + str(e))

UPDATE: I initially thought that the bat file was causing this issue but it looks like it works when running manually a batch file but not when it is set on task scheduler

We need way more of your code to help. The batch and python files, to start. — wundermahn
Very likely culprit is the "shell=True" on Windows. IDE and batch are prob. using different shell, or using the quotes differently. Instead of using shell=True, first, construct the argv for subprocess so that you know exactly what your command arguments are. That includes your tableauCommand part too. One way to debug this as is, is to swap the command to "echo" and see the command it is generating. — Naoyuki Tai

Muhammad Usman Muhammad Usman · Accepted Answer · 2019-12-09T18:32:37

Updated

First of all, if there is a need to run anaconda-prompt by calling activate.bat file, you can simply do as follows:

import subprocess


def call_anaconda_venv():
    subprocess.call('python -m venv virtual.env')
    subprocess.call('cmd.exe /k /path/venv/Scripts/activate.bat')


if __name__ == "__main__":
    call_anaconda_venv()

The result of the above code would be a running instance of anaconda-prompt as required.

Now as Problem Seems Like:

I have a batch file which is running a python script and in the python script, I have a subprocess function which is being run.

I have implemented the same program as required; Suppose we have

Batch File ---> script.bat **** includes a command to run python script i.e test.py. ****
Python Script File ---> test.py **** includes a method to run commands using subprocess. ****
Batch File ---> sys_info.bat **** includes a command which would give the system information of my computer. ****

Now First, script.bat includes a command that will run the required python script as given below;

python \file_path\test.py 
pause

Here, pause command is used to prevent auto-closing console after execution. Now we have test.py, python script which includes subprocess method to run required commands and get their output.

from subprocess import check_output


class BatchCommands:

    @staticmethod
    def run_commands_using_subprocess(commands):
        print("Running commands from File: {}".format(commands))
        value = check_output(commands, shell=True).decode()

        return value

    @staticmethod
    def run():
        commands_from_file = "\file-path\sys_info.bat"

        print('##############################################################')
        print("Shell Commands using >>> subprocess-module  <<<")
        print('##############################################################')

        values = BatchCommands.run_commands_using_subprocess(commands_from_file)
        print(values)


if __name__ == '__main__':
    BatchCommands.run()

Now, in the end, I have a sys_info.bat file which includes commands to renew the IP-Adress of my computer. Commands in sys_info.bat file are as follows;

systeminfo

Place multiple commands in sys_info.bat file, then you can also run multiple commands at a time like:

ipconfig/all
ipconfig/release
ipconfig/reset
ipconfig/renew
ipconfig

Before to use the file, set all files directory paths, and run the batch file i.e script.py in command-prompt as follows;

Run command-prompt or terminal as an administrator.
```
 run \file_path\script.py
```

Here is the result after running the batch file in the terminal.

Subprocess run, check_output, Popen returns empty string when I run the script from a batch file and from Task Scheduler

2 Answers

Updated