Mapreduce job not launching when running hive query with where clause

Question

I am using apache-hive-1.2.2 on Hadoop 2.6.0. When am running a hive query with where clause it is giving results immediately without launching any MapReduce job. I'm not sure what is happening. Table has over 100k records.

i created emp table with 11 columns and the table is having 20 rows .. — amit
when i fire select * from emp query its giving results immediately with out launching any mapreduce job which is expected. — amit
But when i fire select first_name,last_name from emp; which is also giving the results immediately with out running any mapreduce job.I tried loading 100 rows into the emp table but the same result with the second query. — amit
I tested the same table with same data on hive-0.11 version..for first query i.e select * from emp its giving results immeditely with out MR job...But launching the MR job for second query even with 20rows of data which is expected. — amit

Bhaskar Das Bhaskar Das · Accepted Answer · 2017-10-01T19:40:43

I am quoting this from Hive Documentation

hive.fetch.task.conversion

Some select queries can be converted to a single FETCH task, minimizing latency. Currently the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incur RS – ReduceSinkOperator, requiring a MapReduce task), lateral views and joins.

Any type of the sort of aggregation like max or min or count is going to require a MapReduce job. So it depends on your data-set you have.

select * from tablename;

It just reads raw data from files in HDFS, so it is much faster without MapReduce and it doesn't need MR.

Mapreduce job not launching when running hive query with where clause

2 Answers