Hive streaming not working

4
votes

I trying to enable hive streaming by following
https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest#StreamingDataIngest-StreamingRequirements

I had changed all configuration properties to enable hive streaming, but hive metastore service running with below error,

18/02/09 12:22:51 ERROR compactor.Initiator: Caught an exception in the main loop of compactor initiator, exiting MetaException(message:Unable to connect to transaction database org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does not exist

Note: I am using PostgreSQL for JDBC metastore and hive version 2.0.1

Help me to solve this error and start working with Hive Streaming.

1
postgresqlhadoophivehive-metastore
Me too face this issue. Based on the below task, Hive is trying to update some values in "compaction_queue" table. But it is not auto created by Hive when streaming related properties are enabled. issues.apache.org/jira/browse/HIVE-11444 – Dinesh Kumar P

1 Answers

1
votes

The definition of this table (and others related to ACID tables/streaming ingest) can be found in https://github.com/apache/hive/blob/branch-2.0/metastore/scripts/upgrade/postgres/hive-txn-schema-2.0.0.postgres.sql. All of these are required for streaming to function properly.