Get all different values for a certain field in elasticsearch with spring data

Question

I am trying to get all the different values for a certain field (e.g. "name") in ElasticSearch with SpringData.

As a first approach, I have this JSON which does what I want:

{
    "aggs" : {
        "nameAgg" : {
            "terms" : { "field" : "name.key", "size":10000 }
        }
    },
    "size":0
}

It works fine if I perform a GET over the index, retrieving data like this:

"aggregations": {
        "nameAgg": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "Paul",
                    "doc_count": 12
                },
                {
                    "key": "John",
                    "doc_count": 7
                }
]
}
}

Since I only need all the distinct values for the field, this suits my need. Now, I am trying to achieve this in Java with spring-data-elasticsearch.

The closest try I did was this:

AbstractAggregationBuilder<TermsAggregationBuilder> agBuilder = AggregationBuilders.terms("name.key").field("name.key").size(10000);
    
Query query = new NativeSearchQueryBuilder().withQuery(QueryBuilders.matchAllQuery()).addAggregation(agBuilder).build();

But the output contains all the data for each indexed document and it does not match my expected output. How could I execute this type of request with spring-data-elasticsearch?

UPDATE: I have tried to specify a Pageable argument to the query, just like this:

AbstractAggregationBuilder<TermsAggregationBuilder> agBuilder = AggregationBuilders.terms("name.key").field("name.key");
    
    Query query = new NativeSearchQueryBuilder()   
        .withPageable(PageRequest.of(0, 0))
        .withQuery(QueryBuilders.matchAllQuery())
        .addAggregation(agBuilder).build();

But then I receive this exception:

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalArgumentException: Page size must not be less than one!
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:497)
Caused by: java.lang.IllegalArgumentException: Page size must not be less than one!
    at org.springframework.data.domain.AbstractPageRequest.<init>(AbstractPageRequest.java:50)
    at org.springframework.data.domain.PageRequest.<init>(PageRequest.java:43)
    at org.springframework.data.domain.PageRequest.of(PageRequest.java:70)
    at org.springframework.data.domain.PageRequest.of(PageRequest.java:58)

which version of Spring Data Elasticsearch do you use? and how do you execute the shown Query? — P.J.Meisch
I am using: Version Spring Data Elasticsearch: 4.0.3.RELEASE Version Elasticsearch Client: 7.6.2 — JLumos

JLumos JLumos · Accepted Answer · 2021-01-15T08:58:49

I have found the solution.

Since spring data throws an exception if you try to use a Page with size 0, I did a workaround creating a class implementing the Pageable interface:

public class CustomBlankPage implements Pageable {
  
  public static final CustomBlankPage PAGE = new CustomBlankPage();

  @Override
  public int getPageNumber() {
    return 0;
  }

  @Override
  public int getPageSize() {
    return 0;
  }

  @Override
  public long getOffset() {
    return 0;
  }

  @Override
  public Sort getSort() {
    return null;
  }

  @Override
  public Pageable next() {
    return null;
  }

  @Override
  public Pageable previousOrFirst() {
    return null;
  }

  @Override
  public Pageable first() {
    return null;
  }

  @Override
  public boolean hasPrevious() {
    return false;
  }
}

The impl of the search is as follows:

AbstractAggregationBuilder<TermsAggregationBuilder> agBuilder = AggregationBuilders.terms("name.key").field("name.key").size(10000);
    
Query query = new NativeSearchQueryBuilder()   
        .withPageable(CustomBlankPage.PAGE)
        .withQuery(QueryBuilders.matchAllQuery())
        .addAggregation(agBuilder).build();

Then I explore the results retrieved with this code:

SearchHits<Person> hits = operations.search(query, Person.class);
Aggregations aggs = hits.getAggregations();
ParsedStringTerms namesTerm = (ParsedStringTerms) aggs.get("name.key");
List<String> keys = namesTerm.getBuckets()
      .stream()
      .map(b -> b.getKeyAsString())
      .collect(Collectors.toList());

Get all different values for a certain field in elasticsearch with spring data

3 Answers