Connection reset by peer after 500 seconds

Question

I have this strange problem when i run an import cronjob in made. It processes a 1.5GB XML file containing about 400.000 products. The script works fine and would take multiple hours to complete, but after about 500/600 seconds i get the following email from the cron-deamon.

PHP Warning: file_get_contents(http://test.nl/admin/cron_index.php?route=module/EZImport&cron): failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in /home/test.nl/public_html/admin/controller/tool/EZImport_cron.php on line 8

Warning: file_get_contents(http://test.nl/admin/cron_index.php?route=module/EZImport&cron): failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in /home/test.nl/public_html/admin/controller/tool/EZImport_cron.php on line 8 bool(false)

My apache error-logs say:

[Fri Nov 02 09:43:39 2012] [warn] [client 176.9..174] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server [Fri Nov 02 09:43:39 2012] [error] [client 176.9..174] Premature end of script headers: cron_index.php

This is the cron file called by the cronjob


    require_once('../../config.php');
    $opts = array('http' =>
        array('timeout' => 36000)
    );
    $context  = stream_context_create($opts);
    $url = HTTP_SERVER."cron_index.php?route=module/cronMod&cron";
    $result = file_get_contents($url, false, $context);
    var_dump($result);
    die();

I need to run this cron via file_get_contents

Environment:

DEBIAN, OpenCart

The max execution time in webmin (php config) is set to 36000 seconds.

Mod_fcgid (and its apache2 module equivalent) really isn't made for running such longlived tasks. Although you specify that you 'need' to run this cron using your file_get_contents method, you really should consider switching your cron job to use the CLI version of PHP which does not suffer from the time limits the other SAPI's impose. — Rem.co

symcbean symcbean · Accepted Answer · 2012-11-02T09:28:51

processes a 1.5GB XML file

Erk, this is a bit silly - you need a minimum of 2 passes to verify the document is well formed and there's lot's of scope for bad things to happen.

The max execution time in webmin (php config) is set to 36000 seconds

For which end?

You also need to configure the timeout for the webserver and every other component in the chain between client and server, however trying to transfer a 1.5Gb file over HTTP is just silly - you might get it to work - but it's not the right way to solve the problem. Break it up into more manageable chunks.

Connection reset by peer after 500 seconds

2 Answers