Powershell - Why is Using Invoke-WebRequest Much Slower Than a Browser Download?

73
votes

I use Powershell's Invoke-WebRequest method to download a file from Amazon S3 to my Windows EC2 instance.

If I download the file using Chrome, I am able to download a 200 MB file in 5 seconds. The same download in PowerShell using Invoke-WebRequest takes up to 5 minutes.

Why is using Invoke-WebRequest slower and is there a way to download at full speed in a PowerShell script?

5
powershellamazon-ec2amazon-s3powershell-3.0
Is IE configured to use a proxy which Chrome is not on your system? – alroc
@alroc File is downloaded at the same speed in IE as in Chrome. It only slows down using Invoke-WebRequest – Lloyd Banks
Please show your code. What kind of memory utilization are you seeing while the download is happening in PowerShell? – alroc

5 Answers

125
votes

Without switching away from Invoke-WebRequest, turning off the progress bar did it for me. I found the answer from this thread: https://github.com/PowerShell/PowerShell/issues/2138 (jasongin commented on Oct 3, 2016)

$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest <params>

For my 5MB file on localhost, the download time went from 30s to 250ms.

Note that to get the progress bar back in the active shell, you need to call $ProgressPreference = 'Continue'.

47
votes

I was using

Invoke-WebRequest $video_url -OutFile $local_video_url

I changed the above to 

$wc = New-Object net.webclient
$wc.Downloadfile($video_url, $local_video_url)

This restored the download speed to what I was seeing in my browsers.

10
votes

$ProgressPreference = 'SilentlyContinue' I got this down from 52min down to 14sec, for a file of 450 M. Spectacular.

9
votes

One-liner to download a file to the temp directory:

(New-Object Net.WebClient).DownloadFile("https://www.google.com", "$env:temp\index.html")
1
votes

I just hit this issue today, if you change the ContentType argument to application/octet-stream it is much faster (as fast as using webclient). The reason is because the Invoke-Request command will not try and parse the response as JSON or XML.

Invoke-RestMethod -ContentType "application/octet-stream" -Uri $video_url  -OutFile $local_video_url