Jest api bulk update to ES

Question

I have huge number of documents that has to be indexed to Elasticsearch everyday. I tried with 50,000 records using JEST client. But the bulk operation fails after indexing 5000 or so records.

Is it possible to bulk update say 200K records in single operation using JEST bulk update API ?

        for(Log log : logs) {
            Index temp = new Index.Builder(log).index(index).type(type).build();
            list.add(temp);
        }

        // to bulk import
        Bulk bulkupdate = new Bulk.Builder().addAction(list).build();
        BulkResult finalresult = client.execute(bulkupdate);

error: Jest HttpClient socket read timeout error

Answer 1

Increase connection and read timeouts for the IR client being used.