Record created in Cassandra doesn't get indexed into Solr if it has a Map column which is not even indexed

Question

I have a DSE Cassandra table which has a column of type Frozen< Map< text,text>>. When I tried indexing the table in DSE Search Solr, I didnot index the above Map column. A record with NULL Frozen< Map< text,text> gets indexed in Solr while the one with a VALUE doesn't get indexed.

As per DSE Search documentation, Indexing frozen maps is not supported.

But, I have not indexed the frozen column at all (Refer schema.xml below). Its just there in the C* table.

Here are the steps to reproduce the issue,

Created Keyspace

cqlsh> CREATE KEYSPACE arun 
          WITH REPLICATION = { 
            'class' : 'SimpleStrategy', 
            'replication_factor' : 1 };

cqlsh> use arun

Created Cassandra Table

cqlsh:arun> create table employee ( 
             emp_id int, 
             emp_name string, 
             emp_dept string, 
             emp_dob string, 
             unique_identifier frozen<map<text, text>>, 
             primary key (emp_id, emp_name));

Created Solr Core

$./dsetool create_core arun.employee generateResources=true coreOptionsInline=include_columns:emp_id,emp_name,emp_dept

Auto generated: schema.xml

<schema name="autoSolrSchema" version="1.5">
  <types>
    <fieldType class="org.apache.solr.schema.TrieIntField" name="TrieIntField"/>
    <fieldType class="org.apache.solr.schema.TextField" name="TextField">
      <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>
    <fieldType class="org.apache.solr.schema.StrField" name="StrField"/>
  </types>
  <fields>
    <field docValues="true" indexed="true" multiValued="false" name="emp_id" stored="true" type="TrieIntField"/>
    <field indexed="true" multiValued="false" name="emp_dept" stored="true" type="TextField"/>
    <field indexed="true" multiValued="false" name="emp_name" stored="true" type="StrField"/>
  </fields>
  <uniqueKey>(emp_id,emp_name)</uniqueKey>
</schema>

Inserted Records to Cassandra Table

cqlsh> insert into arun.employee 
(emp_id, emp_name, emp_dept, emp_dob) values 
(1,'arun chan','et','19790101',
      {'emp_dept': 'et', 'emp_id': '1', 'emp_name': 'arun chan'});

cqlsh> insert into arun.employee 
(emp_id, emp_name, emp_dept, emp_dob) values 
(2,'mritula chan','ml','19840101');

cqlsh> insert into arun.employee 
(emp_id, emp_name, emp_dept, emp_dob, unique_identifier) values
(3,'jaya guru','ph','19870101', 
      {'emp_id' : '3', 'emp_name' : 'jaya guru', 'emp_dept' : 'ph'});

Cassandra Query: Returns 3 rows inserted

cqlsh> select * from arun.employee;

 emp_id | emp_name     | emp_dept | emp_dob  | solr_query | unique_identifier
--------+--------------+----------+----------+------------+------------------------------------------------------------
      1 |    arun chan |       et | 19790101 |       null | {'emp_dept': 'et', 'emp_id': '1', 'emp_name': 'arun chan'}
      2 | mritula chan |       ml | 19840101 |       null |                                                       null
      3 |    jaya guru |       ph | 19870101 |       null | {'emp_dept': 'ph', 'emp_id': '3', 'emp_name': 'jaya guru'}

(3 rows)

PROBLEM: Why does DSE Solr Query Returns Only One?

cqlsh> select json * from arun.employee where solr_query = '{"q":"*:*"}';

 [json]
-----------------------------------------------------------------------------------------------------------------------------------
 {"emp_id": 2, "emp_name": "mritula chan", "emp_dept": "ml", "emp_dob": "19840101", "solr_query": null, "unique_identifier": null}

(1 rows)

Arun Chandrasekaran Arun Chandrasekaran · Accepted Answer · 2017-07-25T12:52:43

Datastax support said that this is a known issue.

Here is their internal defect tracking number,

DSP-13997: Non indexed frozen maps silently erroring

Record created in Cassandra doesn't get indexed into Solr if it has a Map column which is not even indexed

1 Answers