Hive always create mapreduce job

0
votes

I'm been using hive at my work, when i run a select like this 

"Select * from TABLENAME"

hive executes a mapreduce job and when I run

"Select * from TABLENAME LIMIT X" independently of x.

hive doesn't execute mapreduce jobs.

I use hive 1.2.1, HDP 2.3.0, hue 2.6.1 and hadoop 2.7.1

Any ideas about this fact?

Thanks!

2
hadoopmapreducehive
Possible duplicate stackoverflow.com/questions/30908203/… – invoketheshell
Duplicate of stackoverflow.com/questions/30908203/… – Mangat Rai Modi
My case is different, because a simple query "select * from TABLE" executes mapreduce job instead of reading from HDFS. I know how hive and mapreduce work. – sonia

2 Answers

1
votes
Select * from table;

Requires no map nor reduce. There is no filter(where statement) or aggregation function here. This query simply reads from HDFS.

-1
votes

This is hive's essential task. It is just an abstraction to map-reduce jobs. The former facebook engineers had to write 100s of map-reduce jobs for ad-hoc analysis and map-reduce jobs are somewhat a pain in the ass. So they abstracted it by an sql-language that will be translated in map-reduce jobs.

This is the same with Pig (yahoo).

P.S Some queries are so easy that they aren't translated to map-reduce jobs but are executed locally on one node as far as i know