Hive installation issues: Hive metastore database is not initialized

44
votes

I tried to install hive on a raspberry pi 2. I installed Hive by uncompress zipped Hive package and configure $HADOOP_HOME and $HIVE_HOME manually under hduser user-group I created. When running hive, I got the following error message: hive

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)

So I ran the command suggested in the above error message: schematool -dbType derby -initSchema I got the error message:

Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000) org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !! * schemaTool failed *

It seems there aren't any helpful information when I try to google this error online. Any help or any explanation on how Hive works with Derby would be appreciated!

5
hadoopinstallationhivederby
Did you ever get around to fixing this issue ? – Newtopian

5 Answers

141
votes

After installing hive, if the first thing you did was run hive, hive attempted to create/initialize the metastore_db, but apparently might not get it right. On that initial run, maybe you saw your error:

Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)

Running hive, even though it fails, creates a metastore_db directory in the directory from which you ran hive:

ubuntu15-laptop: ~ $>ls -l |grep meta
drwxrwxr-x 5 testuser testuser 4096 Apr 14 12:44 metastore_db

So when you then tried running

ubuntu15-laptop: ~ $>schematool -initSchema -dbType derby

The metastore already existed, but not in complete form.

Soooooo the answer is:

  1. Before you run hive for the first time, run

    schematool -initSchema -dbType derby

  2. If you already ran hive and then tried to initSchema and it's failing:

    mv metastore_db metastore_db.tmp

  3. Re run

    schematool -initSchema -dbType derby

  4. Run hive again

**Also of note: if you change directories, the metastore_db created above won't be found! I'm sure there's a good reason for this that I don't know yet because I'm literally trying to use hive for the first time today. Ahhh here's information on this: metastore_db created wherever I run Hive

19
votes

I installed hive with HomeBrew(MacOS) at /usr/local/Cellar/hive and afer running schematool -dbType derby -initSchema I get the following error message:

Starting metastore schema initialization to 2.0.0 Initialization script hive-schema-2.0.0.derby.sql Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000) org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

However, I can't find either metastore_db or metastore_db.tmp folder under install path, so I tried:

  1. find /usr/ -name hive-schema-2.0.0.derby.sql
  2. vi /usr/local/Cellar/hive/2.0.1/libexec/scripts/metastore/upgrade/derby/hive-schema-2.0.0.derby.sql
  3. comment the 'NUCLEUS_ASCII' function and 'NUCLEUS_MATCHES' function
  4. rerun schematool -dbType derby -initSchema, then everything goes well!

hope this help someone.

15
votes

I encountered a similar issue, which I fixed by removing the derby database files rm -Rf $HIVE_HOME/metastore_db

Be aware, this will remove your schema completely!

After cleaning the old schema I could reinitialize with:

$HIVE_HOME/bin/schematool -initSchema -dbType derby

It might differ from where you call hive, try to go to your hive installation directory and run ./bin/hive

2
votes

Try this one: schematool -dbType mysql -initSchema

Read the documentation of Hive SchemTool: https://cwiki.apache.org/confluence/display/Hive/Hive+Schema+Tool

0
votes

If you use Derby, you can remove DB files and run the init process.

Or if problem persists, you can edit the SQL files Hive use. For example:

/usr/hdp/2.5.6.0-40/hive2/scripts/metastore/upgrade/mysql/ contains those for mysql.