Filter Azure Search query result with a big list of IDs

Question

I have 2 indexes in Azure Search:

// index-1. record type
{
  "id": "123456"
}

// index-2. records group type
{
  "groupId": "1",
  "groupRecords": [ "123456" ]
}

I need to query all records from index-1 that belong to the group in index-2.

From ElasticSearch documentation I see it supports cross-index queries.

Is that possible to do a similar query with Azure Search? Or is that possible to filter out Search result with a dataset from Cosmos?

p.s. I may have hundreds of millions of records, groups of millions of records, thousands of groups.

Answer 1

Is that possible to do a similar query with Azure Search?

As far as I know, cross-index searches are not supported in Azure Search. Search is scoped/limited to a single index only.

I found this feedback from 2016 that asked for this feature and it has been under review for almost 2 years now.

https://feedback.azure.com/forums/263029-azure-search/suggestions/16700506-search-multiple-indexes-at-once

Answer 2

The common approach with search engines is to flatten the record while consuming the data. I.e. you would read the data from both sources and combine them into a new, flat record that contains all the data.

// index-x, flat records
{
  "id": "123456",
  "groupRecords": [ "123456" ]
}

I'm not sure I understand exactly what you are trying to do, but flattening the data is common practice when you need to join data.