Document size adjusting Search.Score - virtually reducing Scoring profile score

Question

We are using scoring profile for driving the relevance and adjusting scores i.e. boost the relevance for a attribute isActive is 1 by 50 using function in scoring profile, While searching for a specific fields on the Index by passing &searchFields=******

however Search.Score seems highly squeezed by size of the document , smaller the size high score probably due to TF-IDF…..

And this is defeating the purpose of using scoring profile , however in our case we don’t want score to be impact due to size of document since we are passing searchFields.

Cases where searchFields are not passed we want scores to be adjusted by size i.e. free form search in all searchable fields.

example search query -

agency temps&$count=true&$top=30&$skip=0&searchMode=All&$filter=(CompanyCode eq '13453' and VNumber eq '00023232312016') &scoringProfile=BusinessProfile1&searchFields=VCategory

Answer 1

I wonder if the new featuresMode preview capability would be helpful for you? Using this, you can get a lot more information back from the search query such as uniqueTokenMatches and termFrequency on a field by field basis. Using this, you could adjust the ordering as needed on the client side.

Also, you are correct that the default is a TF-IDF like scoring, however, you might also be interested in trying BM25 which although does not solve what you are asking for, could be more effective for helping to get scores you are looking for.

Answer 2

For now I adopted the approach to adjust the parameters for algorithm BM25 as advised by Liam, and added b as 0.0 in index creation json, so that document size is not used during TF-IDF while calculating score for the document,

   "similarity": {
    "@odata.type": "#Microsoft.Azure.Search.BM25Similarity",
    "b" : 0.0,
    "k1" : 1.3
}

however same time identified another field on the index having a correlation with size of the record on the index i.e. larger the size higher the value of that field and using that in scoring profile for the case where document size should be considered in scoring.