Powershell calling external program with progress animation displays oddly

Question

I have a PowerShell script to convert a local vmware instance to a remote AWS instance. Two steps in this process are calls to external CLI programs that have their own progress bars. The problem is they're not being displayed properly.

When run by itself, vmware-vdiskmanager displays the following line, with the percentage number updating as each point is completed. When called in PowerShell, I get a new line every time the percentage point is updated, with the whole text repeated.

Convert: 0% done.

Another step is ec2-import-instance which has a slightly fancier progress bar. When run by itself, it fills the space and updates the percentage. From PowerShell, it never updates. It just displays as below static until the job is finished.

0% |--------------------------------------------------| 100%

Both commands in my PowerShell script have their output piped to Out-Host. I don't understand how PowerShell interacts with the output of programs. I never had problems like this in batch, but there are a few steps in this process where I need the power of a real programming language.

Any idea what's going on or how to make it display like they do when run by themselves?

Edit: As an example, here's the command for the first one.

vmware-vdiskmanager -r $filename -t 5 stream.vmdk | Out-Host

There's obviously logic earlier defining $filename. It does the conversion just fine, I just get 101 lines of output.

Creating disk 'stream.vmdk'

  Convert: 0% done.
  Convert: 1% done.
  Convert: 2% done.
  Convert: 3% done.
  Convert: 4% done.
  Convert: 5% done.
  Convert: 6% done.
  Convert: 7% done.
  Convert: 8% done.
  Convert: 9% done.
  Convert: 10% done.
  Convert: 11% done.
  Convert: 12% done.

The second example just doesn't update the output. It has more parameters and references some environment variables. This is all I see for output.

Uploading the manifest file
Uploading 19979618816 bytes across 1906 parts
0% |--------------------------------------------------| 100%
   |===================================================|
Done
Average speed was 1.378 MBps

My best guess was something like PowerShell doesn't support printing backspace characters or positioning output of external executables, that Out-Host is a unidirectional stream. The difference in two behaviors could be a matter of flushing output in the external program itself. I'm just guessing here.

I haven't boiled this down to a self-contained test case anyone can run. It's just a command with parameters piped to Out-Host. Is there reason to think this is some kind of syntactical nuance? I'll edit an example line in anyway. — icrf
What happens if you issue the VM command from PS prompt without the '| out-host'? What about a single line PS script with the VM command without '| out-host'? — andyb
As a workaround, you could try the CMD /C trick e.g. cmd/c "vmware-vdiskmanager -r $filename -t 5 stream.vmdk" — andyb

andyb andyb · Accepted Answer · 2014-08-05T00:51:24

The problem seems to appear when PowerShell is capturing the output of the external command. In this case, piping to 'out-host' causes PowerShell to capture output. This happens one line at a time and is triggered by the CR issued by the external program.

In the 'Convert: 0% done.' case, the app issues a CR after each line before writing the new line. Each line is sent PowerShell as a newline, hence multiple lines.

In the 'progress bar' case, the app prints the empty progress bar, then issues a CR to put cursor back to the beginning of the line (PowerShell sees this as line of output). The app then moves the cursor to the right (some control char - not sure which) and one character at a time updates the progress bar until complete. PowerShell only gets to see this line when the CR is issued after completion, hence only one additional line.

EDIT: Alternative to above theory is that app uses sequence of backspaces to move cursor to beginning of bar. This triggers a new line for PowerShell because the number of characters output (including the backspaces) exceeds console line length.

I don't have access to VMWare, but wrote a short console app exhibits the behaviour I've described above.

If you don't need to capture the output, just run without the 'out-host' and make sure the output is not included in any pipeline operation.

Powershell calling external program with progress animation displays oddly

1 Answers