Load Default Dump file using Pig

Question

which is the PigStorage() need to use to get the load the file for the following query

INSERT OVERWRITE DIRECTORY 'doop'
select a.* from cdr.cell_tower_info

the output for the above query is like this

Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591 Haryana Ambala 404 20 80 30021 76.76746 30.373488 404-20-80-30021 Haryana Ambala 404 20 80 37591 76.76746 30.373488 404-20-80-37591

I am working with CDR analysis where first i need to retrieve some fields from a table using select and save it into a HDFS folder , this result is again need to load using pig for further analysis

Answer 1

Try this

CREATE EXTERNAL TABLE cell_tower_info
   FIELDS TERMINATED BY ','
   LOCATION 'doop'
   AS
SELECT .* from cdr.cell_tower_info

Answer 2

Hive default delimiter is ctrl-A (\001).

I think PigStorage('\001') should work to use the Hive output data in PIG.

Alternatively Hive table can be defined as fields terminated by '\t' , so that the result can be used directly in PIG.

Answer 3

Can you try this?
input.txt

Haryana Ambala  404 20  80  37591   76.76746    30.373488   404-20-80-37591
Haryana Ambala  404 20  80  30021   76.76746    30.373488   404-20-80-30021
Haryana Ambala  404 20  80  37591   76.76746    30.373488   404-20-80-37591

PigScript:

A = LOAD 'input.txt' AS line;
B = FOREACH A GENERATE FLATTEN(STRSPLIT(line,'\\s+')) AS (f1:chararray,f2:chararray,f3:int,f4:int,f5:int,f6:int,f7:double,f8:double,f9:chararray);
C = FOREACH B GENERATE f1,f2,f8,f9;
DUMP C;

Output:

(Haryana,Ambala,30.373488,404-20-80-37591)
(Haryana,Ambala,30.373488,404-20-80-30021)
(Haryana,Ambala,30.373488,404-20-80-37591)

Answer 4

if your data is stored in hive table then,

X = LOAD 'hdfs://localhost:9000/user/hive/warehouse/input.txt' using PigStorage('\t')  AS line;
Y = FOREACH A GENERATE FLATTEN(STRSPLIT(line,'\\s+')) AS (A1:chararray,A2:chararray,A3:int,A4:int,A5:int,A6:int,A7:double,a8:double,A9:chararray);
Z = FOREACH B GENERATE A1,A2,A8,A9;
DUMP Z;

in my case port is 9000. give according to your system.