Hive error finding unmapped keyspaces

1
votes

I am trying to make an external table in Hive as shown on page 88 of the Datastax Enterprise 3.1. Documentation.

The statement is further below together with the error message.

What am I doing wrong?

Regards Hans-Peter

hive> create external table testext (m string, n string, o string, p string)
> STORED BY 'org.apache.hadoop.hive.cassandra.cql3.CqlStorageHandler'
> TBLPROPERTIES ( "cassandra.ks.name" = "cql3ks",
> "cassandra.cf.name" = "test",
> "cassandra.cql3.type" = "text, text, text, text");
FAILED: Error in metadata:       
com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: 
There was a problem   with the Cassandra Hive MetaStore: Problem finding unmapped
keyspaces
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask   

2013-10-15 12:47:36,657 WARN  conf.HiveConf (HiveConf.java:(63)) - DEPRECATED: Ignoring hive-default.xml found on the CLASSPATH at /etc/dse/hive/hive-default.xml
2013-10-15 12:48:41,003 WARN  config.DatabaseDescriptor (DatabaseDescriptor.java:loadYaml(253)) - Please rename 'authority' to 'authorizer' in cassandra.yaml
2013-10-15 12:48:42,988 ERROR exec.Task (SessionState.java:printError(400)) - FAILED: Error in metadata: com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: There was a problem with the Cassandra Hive MetaStore: Problem finding unmapped keyspaces
org.apache.hadoop.hive.ql.metadata.HiveException: com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: There was a problem with the Cassandra Hive MetaStore: Problem finding unmapped keyspaces
        at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:544)
        at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3305)
        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:242)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: There was a problem with the Cassandra Hive MetaStore: Problem finding unmapped keyspaces
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.createKeyspaceSchemasIfNeeded(SchemaManagerService.java:230)
        at com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStore.setConf(CassandraHiveMetaStore.java:112)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
        at org.apache.hadoop.hive.metastore.RetryingRawStore.(RetryingRawStore.java:62)
        at org.apache.hadoop.hive.metastore.RetryingRawStore.getProxy(RetryingRawStore.java:71)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:346)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:333)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:371)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:278)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.(HiveMetaStore.java:248)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:114)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2092)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2102)
        at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:538)
        ... 17 more
Caused by: com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: There was a problem with the Cassandra Hive MetaStore: There was a problem retrieving column families for keyspace demo
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.createUnmappedTables(SchemaManagerService.java:277)
        at com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStore.getDatabase(CassandraHiveMetaStore.java:148)
        at com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStore.getDatabase(CassandraHiveMetaStore.java:136)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.isKeyspaceMapped(SchemaManagerService.java:186)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.finUnmappedKeyspaces(SchemaManagerService.java:137)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.createKeyspaceSchemasIfNeeded(SchemaManagerService.java:224)
        ... 31 more
Caused by: com.datastax.bdp.hadoop.hive.metastore.CassandraHiveMetaStoreException: There was a problem with the Cassandra Hive MetaStore: Problem creating column mappingsorg.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.buildTable(SchemaManagerService.java:481)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.createUnmappedTables(SchemaManagerService.java:254)
        ... 36 more
Caused by: java.lang.IllegalArgumentException
        at java.nio.Buffer.limit(Buffer.java:247)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:51)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:60)
        at org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:226)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.addTypeToStorageDescriptor(SchemaManagerService.java:846)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.buildColumnMappings(SchemaManagerService.java:546)
        at com.datastax.bdp.hadoop.hive.metastore.SchemaManagerService.buildTable(SchemaManagerService.java:460)
        ... 37 more

2013-10-15 12:48:42,990 ERROR ql.Driver (SessionState.java:printError(400)) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
2
cassandrahive
Can you post a full stacktrace from the hive.log? Which version of DSE? I checked your steps and it works for me. - Piotr Kołaczkowski
In the hive.log there is the same message plus a java stack trace. I cannot past it here because it is too big. - user1393650
Can't you add it to the post (edit)? - Piotr Kołaczkowski
Hi Piotr I have added it to the post. - user1393650

2 Answers

0
votes

I am not sure what the actual problem, but I ran into this while creating a normal table in Hive.

I started Hive with sudo access, and can now run queries as expected.

$ sudo bin/dse hive
0
votes

So something that worked for me was to totally wipe out the HiveMetaStore keyspace in Cassandra and recreate just the keyspace with the NetworkTopologyStrategy replica strategy. I made sure to add the Analytics datacenter to the new keyspace as well, so it looked something like:

CREATE KEYSPACE HiveMetaStore WITH REPLICATION = {'class' : 'NetworkTopologyStrategy', 'Analytics' : 2};

I then restarted DSE on my analytics nodes and they correctly created the MetaStore table within the HiveMetaStore keyspace and everything started working again!