Hive argument using Variable Substitution (-d|--define) fails with a string argument

Question

When I run the a hive script with the command

hive -d arg_partition1="p1" -f test.hql

It returns the error

FAILED: SemanticException [Error 10004]: Line 3:36 Invalid table alias or column reference 'p1': (possible column names are: line, partition1)

Script with name test.hql

DROP TABLE IF EXISTS test;
CREATE EXTERNAL TABLE IF NOT EXISTS test (Line STRING)
    PARTITIONED BY (partition1 STRING);
ALTER TABLE test ADD PARTITION (partition1="p1") LOCATION '/user/test/hive_test_data';
SELECT * FROM test WHERE partition1=${arg_partition1};

If I modify the partition to be an integer then it works fine and returns the correct results.

How do I run a Hive script with a string argument?

jkovacs jkovacs · Accepted Answer · 2013-07-18T16:31:29

You'll have to escape your quotes when invoking hive, such as -d arg_partition1=\"p1\" for this to work.

However, I don't see why you'd have to add the quotes to the replacement string in any case. Presumably you know the data types of your fields when writing the query, so if partition1 is a string then include the quotes in the query, such as WHERE partition1="${arg_partition1}"; and if it's an integer just leave them out entirely.

Hive argument using Variable Substitution (-d|--define) fails with a string argument

1 Answers