Error with Elasticsearch Helpers Scan API when using query parameter

Question

I am using Elasticsearch Helpers Scan API to output all bulk records from my index. I wanted to use the query parameter to filter the certain records first and then use scan for which I referred to the standard doc for scan here . But as soon as I apply the query parameter it gives me a 400 error. Here is my line of code and error.

Script:

import elasticsearch
import elasticsearch.exceptions 
import elasticsearch.helpers as helpers
import time


es = elasticsearch.Elasticsearch('XXX.XX.XX.XX'],retry_on_timeout=True)  


scanResp = helpers.scan(client=es,scroll="5m",query={"match":{"channel_id": "34"}},index="my-index",timeout="10m",size=500)

resp={}
start_time = time.time()
for resp in scanResp:
    data = resp
    #print data.values()[3]


print("--- %s seconds ---" % (time.time() - start_time))

Output:

RequestError: TransportError(400, u'SearchPhaseExecutionException[Failed to execute phase [init_scan], all shards failed; shardFailures {[lEAncRHvQHKpPqlfw1NWEQ][dev-godam][0]: SearchParseException[[dev-godam][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"match": {"master_channel_id": "34"}}]]]; nested: SearchParseException[[dev-godam][0]: from[-1],size[-1]: Parse Failure [No parser for element [match]]]; }{[lEAncRHvQHKpPqlfw1NWEQ][dev-godam][1]: SearchParseException[[dev-godam][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"match": {"master_channel_id": "34"}}]]]; nested: SearchParseException[[dev-godam][1]: from[-1],size[-1]: Parse Failure [No parser for element [match]]]; }{[lEAncRHvQHKpPqlfw1NWEQ][dev-godam][2]: SearchParseException[[dev-godam][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"match": {"master_channel_id": "34"}}]]]; nested: SearchParseException[[dev-godam][2]: from[-1],size[-1]: Parse Failure [No parser for element [match]]]; }{[lEAncRHvQHKpPqlfw1NWEQ][dev-godam][3]: SearchParseException[[dev-godam][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"match": {"master_channel_id": "34"}}]]]; nested: SearchParseException[[dev-godam][3]: from[-1],size[-1]: Parse Failure [No parser for element [match]]]; }{[lEAncRHvQHKpPqlfw1NWEQ][dev-godam][4]: SearchParseException[[dev-godam][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"match": {"master_channel_id": "34"}}]]]; nested: SearchParseException[[dev-godam][4]: from[-1],size[-1]: Parse Failure [No parser for element [match]]]; }]')

Looked on the internet but not sure what I have done wrong here.

Any help is appreciated

Val Val · Accepted Answer · 2015-10-23T11:27:45

Your body parameter should be a dictionary instead of a string and use double quotes instead of single quotes, try this:

scanResp = helpers.scan(client=es,scroll="5m",query={"query":{"match": {"channel_id": "34"}}},index='stg-index',timeout="10m",size=500)

Error with Elasticsearch Helpers Scan API when using query parameter

1 Answers