Marklogic 9 rebalancer not working with assigned query policy

Question

I am trying to tier Marklogic 9 via query assignment policy. I created 2 tiers (as partitions) in "Documents" database with partition names as:

Part1 , partition number is 1, Default partition (no assigned query)
Part2 , partition number is 2

In database documents are stored as JSON and i want to tier them according do "DYil" property of documents.

Sample JSON doc that exists in db:

{
"Yerlesim": "Izmir", 
"Ad": "AAA", 
"@timestamp": "2018-06-02T21:16:23.647Z", 
"SoyAd": "BBB", 
"@version": "1", 
"DYil": "2010-01-01", 
"host": "dhcppc6", 
"Yas": 8, 
"type": "testA", 
}

The configuration of the database is done according to Marklogic documentation, following these instructions:

Assignment Policy as query

Locking as strict

DYil index as date

The assigned query for partition Part2 is:

<partition-query-properties xmlns="http://marklogic.com/manage/partition-query/properties">
  <partition-number>2</partition-number>
  <query>
    <cts:json-property-range-query operator="&lt;" xmlns:cts="http://marklogic.com/cts">
      <cts:property>DYil</cts:property>
      <cts:value xsi:type="xs:date" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">1960-01-01</cts:value>
    </cts:json-property-range-query>
  </query>
</partition-query-properties>

I can see query is assigned correctly to partition via rest api by calling http://localhost:8002/manage/v2/databases/Documents/partition-queries

But when I load data to Marklogic, all data is loaded to default partition and stays there even if I force rebalancer to work.

Xquery console query is:

cts:search(fn:collection(), cts:json-property-range-query("DYil", "<", xs:date("1960-01-01")), (), (), ts:partition-forests(xdmp:database("Documents"),"Part1"))

I converted cts:query expression cts:json-property-range-query("DYil", "<", xs:date("1960-01-01")) to XML format with using <root>{cts:json-property-range-query("DYil", "<", xs:date("1960-01-01"))}</root>/node() code as suggested at Marklogic site.

There is no failed log record at Error logs so I suspect there is a problem about assigned query but not able to locate problem.

I am adding data via kafka, using a kafka connector for marklogic, github.com/sanjuthomas/kafka-connect-marklogic. Documents are already as JSON at kafka topics — cnon

asusu asusu · Accepted Answer · 2018-06-06T00:16:44

WRT query matching, here's some code I tried to check things out once. Run in the content db, and it will check for the partition queries and see how much matches each one.

xquery version "1.0-ml";
declare namespace ts = 'http://marklogic.com/xdmp/tieredstorage';

let $xpath := "
declare namespace ts = 'http://marklogic.com/xdmp/tieredstorage';
/ts:partition-query
"
for $pq in xdmp:eval ($xpath, (), <options xmlns='xdmp:eval'><database>{xdmp:schema-database()}</database></options>)
let $query := cts:query ($pq/ts:query/*)
let $number := $pq/ts:partition-number/fn:data ()
let $matched := xdmp:estimate (cts:search (/, $query))
return ('query is: ', $pq, 'cts q is: ', $query, 'p# is: ', $number, '# docs matched is:', $matched)

Marklogic 9 rebalancer not working with assigned query policy

1 Answers