Multivalued copy field support in DSE SOLR

0
votes

I have a SOLR schema as following:

<field name="category_id1" type="integer" indexed="false" stored="true" />
<field name="category_id2" type="integer" indexed="false" stored="true" />
<field name="category_id3" type="integer" indexed="false" stored="true" />
<field name="category_ids" type="integer" multiValued="true" indexed="true" stored="true"/>

and a copy section:

<copyField source="category_id1" dest="category_ids" />

but whenever I tried to inject the data into DSE/Cassandra, I got this error

InvalidRequestException(why:(Expected 4 or 0 byte int (14)) [diem][business][category_ids] failed validation) me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:(Expected 4 or 0 byte int (14)) [diem][business][category_ids] failed validation) Exception in thread "main" me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:(Expected 4 or 0 byte int (14)) [diem][business][category_ids] failed validation) at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:45) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:264) at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecuteOperation(ExecutingKeyspace.java:97) at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:243) at com.diem.db.crud.CassandraStorageManager.insertMultiColumns(CassandraStorageManager.java:197) at com.diem.db.dao.impl.AbstractDaoImpl.saveUUIDEntity(AbstractDaoImpl.java:47) at com.diem.db.dao.impl.BusinessDaoImpl.saveBusiness(BusinessDaoImpl.java:81) at com.diem.data.LoadBusinesses.execute(LoadBusinesses.java:187) at com.diem.data.LoadContent.run(LoadContent.java:121) at com.diem.data.LoadBusinesses.main(LoadBusinesses.java:45) Caused by: InvalidRequestException(why:(Expected 4 or 0 byte int (14)) [diem][business][category_ids] failed validation) at org.apache.cassandra.thrift.Cassandra$batch_mutate_result.read(Cassandra.java:20833) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_batch_mutate(Cassandra.java:964) at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:950) at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:246) at me.prettyprint.cassandra.model.MutatorImpl$3.execute(MutatorImpl.java:243) at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:258) ... 8 more

A copy into a multiValued solr.IntField (integer) isn't something special and we could do it before using DSE/SOLR. But I can't seem to get this work inside DSE/SOLR combination. Logically speaking, I can't see any reason why this fails, because DSE should not interfere with operation on category_ids field, which is used primarily for indexing. Does anyone see anything wrong with the situation? What could I do in this situation to prevent the validation error (note: I can't use a text/string type for category_ids)?

Thank you!

2
solrcassandradatastax-enterprisedatastax

2 Answers

0
votes

I could find out the problem, my CF has a default_validation_class=BytesType, so the multiValued field category_ids is validated using BytesType in DSE/Solr, which will cause the error. So unless I change my CF into CQL declaration using the type of LIST<int> and do not use Hector (at least for this CF), I won't be able to work with multiValued fields other than text/string fields in Solr.

0
votes

If I understand it correctly, you are using thrift tables, so you either declare the category_ids column as UTF8Type (the Solr field can be of any type), or you declare the category_ids Solr field as stored=false (in which case the copy field will not be stored, only indexed).

Let us know if any of the two works for you.