bulk index with elasticsearch and php curl

Question

I am trying to perform a php CURL request to ElasticSearch to index several entries at a time (with _bulk), but I always get a different error based on the request body

>         $posted = '';
>         for ($i = 0; $i < 10; $i++) {
>             $posted .= json_encode(array(
>                 'index' => new stdClass(),
>             )) .'\n';
> 
>             $posted .= json_encode(array(
>                 'id' => $i,
>                 'name' => 'XX' . $i,
>             ));
>             
>             if($i < 9){
>                 $posted .'\n';
>             }
>         }
> 
>         $fullURL = 'http://127.0.0.1:9200/myindex/mytype/_bulk';
>         $conn = curl_init($fullURL);
> 
>         curl_setopt($conn, CURLOPT_RETURNTRANSFER, true);
>         curl_setopt($conn, CURLOPT_SSL_VERIFYPEER, false);
>         curl_setopt($conn, CURLOPT_SSL_VERIFYHOST, false);
>         curl_setopt($conn, CURLOPT_FAILONERROR, false);
>         curl_setopt($conn, CURLOPT_CUSTOMREQUEST, 'POST');
>         curl_setopt($conn, CURLOPT_POSTFIELDS, $posted);
>         $res = curl_exec($conn);
>         echo $res;

with the above param string I get this error:

"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: no requests added;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: no requests added;"},"status":400}

I have tested it with single requests encoded in JSON and it works fine.
So, how can I perform _bulk indexing with php curl?

EDIT: --- Edited for more readability, see the answer below. I tried everything I found on the web for 2 days, hope it helps someone else.

user1626498 user1626498 · Accepted Answer · 2016-10-08T17:57:51

I finally Got it working!!!

>         $b = array();
>         $sets = array();
> 
>         $params = array(
>             '_id' => null,
>             '_index' => 'myindex',
>             '_type' => 'mytype'
>         );
> 
>         for ($i = 0; $i < 10; $i++) {
>             $doc = array(
>                 'id' => $i,
>                 'name' => 'name ' . $i,
>             );
>             $set = array(
>                 array('index' => $params),
>                 $doc
>             );
>             $sets[] = $set;
>         }
> 
>         foreach ($sets as $set) {
>             foreach ($set as $s) {
>                 $b[] = json_encode($s);
>             }
>         }
>         $body =  join("\n", $b) . "\n";
>         
>         $conn = curl_init();
>         $requestURL = 'http://127.0.0.1:9200/myindex/mytype/_bulk';
>         curl_setopt($conn, CURLOPT_URL, $requestURL);
>         curl_setopt($conn, CURLOPT_TIMEOUT, 5);
>         curl_setopt($conn, CURLOPT_SSL_VERIFYPEER, FALSE);
>         curl_setopt($conn, CURLOPT_SSL_VERIFYHOST, FALSE);
>         curl_setopt($conn, CURLOPT_RETURNTRANSFER, TRUE);
>         curl_setopt($conn, CURLOPT_FAILONERROR, FALSE);
>         curl_setopt($conn, CURLOPT_CUSTOMREQUEST, strtoupper('POST'));
>         curl_setopt($conn, CURLOPT_FORBID_REUSE, 0);
> 
>         if (is_array($body) && count($body) > 0) {
>             curl_setopt($conn, CURLOPT_POSTFIELDS, json_encode($body));
>         } else {
>             curl_setopt($conn, CURLOPT_POSTFIELDS, $body);
>         }
>         
>         $response = curl_exec($conn);
>         echo $response;

bulk index with elasticsearch and php curl

1 Answers