1
votes

I have a PHP script that processes data downloaded from multiple REST APIs into a standardized format and builds an array or table of this data. The script currently executes everything synchronously and therefore takes too long.

I have been trying to learn how to execute the function that fetches and processes the data, simultaneously or asynchronously so that the total time is the time of the slowest call. From my research it appears that ReactPHP or Amp are the correct tools.

However, I have been unsuccessful in creating test code that actually executes correctly. A simple example is attached, with mysquare() representing my more complex function. Due to a lack of examples on the net of exactly what I'm trying to achieve I have been forced to use a brute force method with 3 examples listed in my code.

Q1: Am I using the right tool for the job?

Q2: Can you fix my example code to execute asynchronously?

NB: I am a real beginner, so the simplest possible code example with a minimum of high level programming lingo would be appreciated.

<?php
require_once("../vendor/autoload.php");

for ($i = 0; $i <= 4; $i++) {

    // Experiment 1
    $deferred[$i] = new React\Promise\Deferred(function () use ($i) {
        echo $x."\n";
        usleep(rand(0, 3000000));  // Simulates long network call
        return array($x=> $x * $x);
    });

    // Experiment 2
    $promise[$i]=$deferred[$i]->promise(function () use ($i) {
        echo $x."\n";
        usleep(rand(0, 3000000));  // Simulates long network call
        return array($x=> $x * $x);
    });

    // Experiment 3
    $functioncall[$i] = function () use ($i) {
        echo $x."\n";
        usleep(rand(0, 3000000));  // Simulates long network call
        return array($x=> $x * $x);
    };
}

$promises = React\Promise\all($deferred); // Doesn't work
$promises = React\Promise\all($promise); // Doesn't work
$promises = React\Promise\all($functioncall); // Doesn't work

// print_r($promises);  // Doesn't return array of results but a complex object

//  This is what I would like to execute simulatenously with a variety of inputs
function mysquare($x)
{
    echo $x."\n";
    usleep(rand(0, 3000000));  // Simulates long network call
    return array($x=> $x * $x);
}
2
Additionally, the script is always run from the CLI - Viktorius

2 Answers

1
votes

Asynchronous doesn't mean multiple threads execute in parallel. 2 functions can only really run at the 'same time', if they (for example) do IO such as a HTTP request.

usleep() blocks, so you gain nothing. Both ReactPHP and Amp will have some kind of 'sleep' function themselves that's built right into the event loop.

For the same reason you will not be able to just use curl, because it will also block out of the box. You need to use the HTTP libraries that React and Amp provide and/recommend.

Since your end-goal is just doing HTTP requests, you could also not use any of these frameworks and just use the curl_multi functions. They're a bit hard to use though.

0
votes

I'm answering my own question in an attempt to help other users, however this solution was developed alone without the help of an experienced programmer and so I do not know if it is ultimately the best way to do this.

TL;DR

I switched from ReactPHP because I didn't understand it to using amphp/parallel-functions which offers a simplified end user interface... sample code using this interface attached.

<?php
require_once("../vendor/autoload.php");

use function Amp\ParallelFunctions\parallelMap;
use function Amp\Promise\wait;

$start = \microtime(true);

$mysquare = function ($x) {
    sleep($x);  // Simulates long network call
    //echo $x."\n";
    return $x * $x;
};

print_r(wait(parallelMap([5,4,3,2,1,6,7,8,9,10], $mysquare)));

print 'Took ' . (\microtime(true) - $start) . ' milliseconds.' . \PHP_EOL;

The example code executes in 10.2 seconds which is slightly longer than the longest running instance of $mysquare().

In my actual use case I was able to fetch data via HTTP from 90 separate sources in around 5 seconds.

Notes:

The amphp/parallel-functions library appears to be using threads under the hood. From my preliminary experience this appears to require a lot more memory than just a single threaded PHP script, but I haven't yet ascertained the full impact. This was highlighted when I was passing a large array to $mysquare via the "use ($myarray)" expression and array was 65Mb. This brought the code to a standstill and it increased execution time exponentially so much so that it took orders of magnitude longer than synchronous execution. Also the memory usage peaked at over 5G! at one point leading me to believe that amphp was duplicating $myarray for each instance. Reworking my code to avoid the "use ($myarray)" expression fixed that problem.