4
votes

I'm using Spring Data Elasticsearch, and am interested in utilizing Elasticsearch 6.2's Multi Search API in order to execute more than one search in a single API request.

https://www.elastic.co/guide/en/elasticsearch/reference/6.2/search-multi-search.html

Currently using Spring Data Elasticsearch's NativeSearchQueryBuilder to build search queries, and ElasticsearchTemplate to execute search requests. I haven't managed to find any methods exposed by either of these for constructing or submitting Multi Search requests after looking through the Spring Data code and browsing documentation.

Does Spring Data Elasticsearch support Multi Search API, via their ElasticsearchTemplate or some other client/mechanism I may be unaware of?

Elasticsearch offers this functionality in their Java API, so what I am after is something analogous to the following, but unfortunately I am bound to using Spring Data Elasticsearch.

https://www.elastic.co/guide/en/elasticsearch/client/java-api/6.2/java-search-msearch.html

2

2 Answers

3
votes

No, Spring Data Elasticsearch doesn't support multi search for the time being. I also met the same problem, and I am working a PR to add this feature. Now, I come up with the following workarounds:

@Override
public YYY findBy(XXX xxx) {
  Client client = template.getClient();
  MultiSearchRequest request = new MultiSearchRequest();
  // build searchQuery like normal
  for (NativeSearchQuery searchQuery : queries) {
    request.add(prepareSearch(client, searchQuery));
  }
  ActionFuture<MultiSearchResponse> future = client
      .multiSearch(request);
  MultiSearchResponse response = future.actionGet();
  Item[] items = response.getResponses();
  for (int i = 0; i < items.length; i++) {
    AggregatedPage<XXX> ts = resultMapper.mapResults(items[i].getResponse(), XXX.class, page);
    // do with page
  }
}

private SearchRequestBuilder prepareSearch(Client client, SearchQuery searchQuery) {
  Assert.notNull(searchQuery.getIndices(), "No index defined for Query");
  Assert.notNull(searchQuery.getTypes(), "No type defined for Query");

  int startRecord = 0;
  SearchRequestBuilder searchRequest = client.prepareSearch(toArray(searchQuery.getIndices()))
      .setSearchType(searchQuery.getSearchType()).setTypes(toArray(searchQuery.getTypes()));

  if (searchQuery.getSourceFilter() != null) {
    SourceFilter sourceFilter = searchQuery.getSourceFilter();
    searchRequest.setFetchSource(sourceFilter.getIncludes(), sourceFilter.getExcludes());
  }

  if (searchQuery.getPageable().isPaged()) {
    startRecord = searchQuery.getPageable().getPageNumber() * searchQuery.getPageable().getPageSize();
    searchRequest.setSize(searchQuery.getPageable().getPageSize());
  }
  searchRequest.setFrom(startRecord);

  if (!searchQuery.getFields().isEmpty()) {
    searchRequest.setFetchSource(toArray(searchQuery.getFields()), null);
  }

  if (searchQuery.getSort() != null) {
    for (Sort.Order order : searchQuery.getSort()) {
      searchRequest.addSort(order.getProperty(),
          order.getDirection() == Sort.Direction.DESC ? SortOrder.DESC : SortOrder.ASC);
    }
  }

  if (searchQuery.getMinScore() > 0) {
    searchRequest.setMinScore(searchQuery.getMinScore());
  }

  if (searchQuery.getFilter() != null) {
    searchRequest.setPostFilter(searchQuery.getFilter());
  }

  if (!isEmpty(searchQuery.getElasticsearchSorts())) {
    for (SortBuilder sort : searchQuery.getElasticsearchSorts()) {
      searchRequest.addSort(sort);
    }
  }

  if (!searchQuery.getScriptFields().isEmpty()) {
    //_source should be return all the time
    //searchRequest.addStoredField("_source");
    for (ScriptField scriptedField : searchQuery.getScriptFields()) {
      searchRequest.addScriptField(scriptedField.fieldName(), scriptedField.script());
    }
  }

  if (searchQuery.getHighlightFields() != null) {
    for (HighlightBuilder.Field highlightField : searchQuery.getHighlightFields()) {
      searchRequest.highlighter(new HighlightBuilder().field(highlightField));
    }
  }

  if (!isEmpty(searchQuery.getIndicesBoost())) {
    for (IndexBoost indexBoost : searchQuery.getIndicesBoost()) {
      searchRequest.addIndexBoost(indexBoost.getIndexName(), indexBoost.getBoost());
    }
  }

  if (!isEmpty(searchQuery.getAggregations())) {
    for (AbstractAggregationBuilder aggregationBuilder : searchQuery.getAggregations()) {
      searchRequest.addAggregation(aggregationBuilder);
    }
  }

  if (!isEmpty(searchQuery.getFacets())) {
    for (FacetRequest aggregatedFacet : searchQuery.getFacets()) {
      searchRequest.addAggregation(aggregatedFacet.getFacet());
    }
  }

  return searchRequest.setQuery(searchQuery.getQuery());
}
0
votes

May be it's too late, but when I faced the same problem I reached the question without any answer. I hope my solution helps to other searchers. Spring data elasticsearch has many limitations, however you can do anything using ElasticSearch's java client.

Here are maven dependencies:

<!--ELASTICSEARCH-->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>${elasticsearch.version}</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.plugin</groupId>
        <artifactId>transport-netty4-client</artifactId>
        <version>${elasticsearch.version}</version>
    </dependency>

Here is my solution:

@Service
public class UniversalSearchHelper {
    @Autowired
    protected Client client;

    public List<SearchResult> matchQuery(String term) {
        MultiSearchRequest request = new MultiSearchRequest();
        request.add(searchRequest(QueryBuilders.boolQuery()
                .should(QueryBuilders.matchQuery("name", term))
                .should(QueryBuilders.matchQuery("full_name", term)), "movie_index"));
        request.add(searchRequest(QueryBuilders.matchQuery("name", term), "taxonomy_index"));
        ActionFuture<MultiSearchResponse> r = client.multiSearch(request);
        return getTopTen(r.actionGet(2000));
    }

    public List<SearchResult> fuzzyQuery(String term) {
        MultiSearchRequest request = new MultiSearchRequest();
        request.add(searchRequest(QueryBuilders.boolQuery()
                .should(QueryBuilders.fuzzyQuery("name", term))
                .should(QueryBuilders.fuzzyQuery("full_name", term)), "movie_index"));
        request.add(searchRequest((QueryBuilders.fuzzyQuery("name", term), "taxonomy_index"));
        ActionFuture<MultiSearchResponse> r = client.multiSearch(request);
        return getTopTen(r.actionGet(2000));
    }

    private SearchRequest searchRequest(QueryBuilder queryBuilder, String... indices) {
        SearchSourceBuilder search = new SearchSourceBuilder();
        search.query(queryBuilder);
        SearchRequest request = new SearchRequest();
        request.indices(indices);
        request.source(search);
        return request;
    }

    private List<SearchResult> getTopTen(MultiSearchResponse response) {
        List<SearchResult> results = new ArrayList<>();
        response.iterator().forEachRemaining(item -> {
            if (!item.isFailure()) {
                item.getResponse().getHits().iterator().forEachRemaining(hit -> results.add(new SearchResult(hit)));
            }
        });

        results.sort((searchResult1, searchResult2) -> (int) (searchResult2.getScore() - searchResult1.getScore()));
        return results.stream().filter(sr -> sr.getTaxonomy() != null).limit(10).collect(Collectors.toList());
    }
}

and entity to get the results:

import org.elasticsearch.search.SearchHit;

public class SearchResult {
    private Object id;
    private float score;
    private String term;
    private String type;
    private String taxonomy;

    public Object getId() {
        return id;
    }

    public SearchResult setId(Object id) {
        this.id = id;
        return this;
    }

    public float getScore() {
        return score;
    }

    public SearchResult setScore(float score) {
        this.score = score;
        return this;
    }

    public String getTerm() {
        return term;
    }

    public SearchResult setTerm(String term) {
        this.term = term;
        return this;
    }

    public String getType() {
        return type;
    }

    public SearchResult setType(String type) {
        this.type = type;
        return this;
    }

    public String getTaxonomy() {
        return taxonomy;
    }

    public void setTaxonomy(String taxonomy) {
        this.taxonomy = taxonomy;
    }

    public SearchResult() {
    }

    public SearchResult(SearchHit hit) {
        this.id = hit.getSource().get("id");
        this.term = (String)hit.getSource().get("name");
        this.score = hit.getScore();
        this.type = hit.getType();
        this.taxonomy = (String) hit.getSource().get("taxonomy");
    }
}