Downloading large file with cURL in PHP - Page hangs

Question

I have a PHP script that downloads videos from various locations. The video files can be any where from 20mb to 100mb+

I've got PHP currently saving the video file in a directory using CURLOPT_FILE. This is working fine with no problems.

Because of the large files that are being dowloaded I've set the cURL timeout period to 45 minutes to allow the file to download. I have also set set_time_limit(0) so that the PHP page should continue processing after the download has completed. I've also set ini_set("memory_limit","500M"); When the download completes it should echo "Downloaded" and then update a mysql record stating the file has been downloaded.

What is happening though, is the video file is being downloaded correctly by cURL but it is not displaying "Downloaded" in the browser BUT it is updating mysql.

Why is this? I've tried to come up with a solution myself, but I cannot work out what the issue here is...

Answer 1

If you're in a browser environment, the browser will timeout after a certain time, and so will stop listening for output from the script, even though the script will continue to run. It varies across browsers, but the number I've seen is 30 seconds.

To overcome this problem, you should send output (even if meaningless echo "<!--empty comment-->";) every so often.

I recently had a similar problem, and I dealt with it by not outputting any content from the script, and instead polling from the browser every so often using AJAX to see if it was done.

Or, don't use the browser environment (as it's not ideally suited for this problem), and instead use a command line prompt, as it does not have (to my knowledge) these timeouts.