Basic thing about Hadoop and Hive

Question

I have started working with Hadoop recently. There is table named Checkout that I access through Hive. And below is the path where the data goes to HDFS and other info. So what information I can get if I have to read the below three lines?

          Path                                      Size           Record Count      Date Loaded
/sys/edw/dw_checkout_trans/snapshot/2012/07/04/00   1.13 TB       9,294,245,800      2012-07-05 07:26
/sys/edw/dw_checkout_trans/snapshot/2012/07/03/00   1.13 TB       9,290,477,963      2012-07-04 09:37
/sys/edw/dw_checkout_trans/snapshot/2012/07/02/00   1.12 TB       9,286,199,847      2012-07-03 07:08

So my question is-

1) Firstly, We are loading the data to HDFS and then through Hive I am querying it to get the result back? Right?

2) Secondly, When you look into the above path and other things, the only thing that I am confuse is, when I will be querying using Hive then I will be getting data from all the three paths above? or the most recent one at the top?

As I am new to these stuff, so I am having lot of problem. Can anyone explain me hive gets the data from where? And we store all the data in HDFS and then we use Hive or Pig to get data back from HDFS? And it will be great if some one give high level knowledge of Hadoop and Hive.

David Gruzman David Gruzman · Accepted Answer · 2012-07-06T06:59:03

I think you need to get the difference between Hive's native table and Hive's external table.
Hive native table mean that you load data into hive, and it takes care how data is stored in the HDFS. We usually do not care what is directory structure in this case.
Hive External table mean that we put data in some directory (if we forget about partitioning for the moment) and tell to Hive - it is table's data. Please treat is as such. And hive enable us to query it, join with other external or regular table. And it is our responsibility to add data, delete it, etc

Basic thing about Hadoop and Hive

1 Answers