Unable to parse date in Pig

1
votes

I'm unable to parse date in pig.
Date format is Mon, 10/11/10 01:02 PM

I load data using the following command:

data = load 'CampaignData.csv' using PigStorage(';');

Next I generate the date column as a chararray using the following command:-

date_data = foreach data generate (chararray) $272 as dates;

when I dump date_data I get the following in output:

Mon

How to get the complete date?

1
apache-pig
$272 is location of the date column – Siddharth Baijal
I think i got the answer to the above problem. Since the file is comma separated, I need to use using PigStorage(','). Then i need to merge columns $272 and $273, which would give me the complete date. Thanks for the help. but now i face another issue. how do I convert this date to datetime format foreach fdate generate ToDate($0, 'EEE MM/dd/yy hh:mm aaa'); where fdate = Mon 10/11/10 01:02 PM – Siddharth Baijal
ToDate($0, 'EEE MM/dd/yy hh:mm aaa'); doesn't work? – VK_217
No, it is not working. I keep getting error 1066 – Siddharth Baijal

1 Answers

0
votes

You don't need $272 to convert date provided to datetime object. You can simply follow this :

date_data = foreach data generate ToDate($273, ' MM/dd/yy hh:mm aaa');

Just make sure $273 is chararray and there is space before data format string specified in ToDate function above. Space is required only to make sure format string looks exactly as data that would be present after parsing row using comma delimiter.