3
votes

I am trying to create an Apache Nifi model that allows me to read in all my data from Elasticsearch and store it to a file. I have everything connected correctly but the issue I am having is the FetchElasticsearch process is requiring a document identifier (as it should) but I want to get every single item in the index it is searching not just a document with ID 1 for example. I know that Nifi and the process property supports expression language so I have tried simply using a regex expression that should match all characters which should just be

${'*'}

but I get warning when I did this because the process actually looks for the literal document id that is * which of course does not exist. Below are screenshots so hopefully it can help with understanding my issue.

Nifi Flow

FetchElasticsearch Process Property

I am searching localhost:9300/tweet_library/tweet/(regex expression) so I want all of the documents in tweet_library. Any help is appreciated, thanks.

1

1 Answers

3
votes

Answer from horton works community:

"The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents."