PIG subtract two dates

4
votes

I'm trying to subtract two dates with PIG.

I have such data:

key_one, activation_date , deactivation_date (1456,2010-06-14 00:00:00,2011-01-01 00:00:00) (6524,2015-01-15 00:00:00,2015-02-07 00:00:00) (1541,2010-07-17 00:00:00,2012-03-07 00:00:00)

I want to make a date difference between deactivation_date and activation_date and in result receive 'key_one' and 'days between two dates'.

DATA_OUTPUT  = foreach MY_DATA generate key_one,
              DaysBetween(deactivation_date, activation_date) as days_between_two_dates;

Backend error : Exception while executing [POUserFunc (Name: POUserFunc(org.apache.pig.builtin.DaysBetween)[long] - scope-231 Operator Key: scope-231) children: null at []]: java.lang.NullPointerException

Any ideas, how to solve this problem?

1
apache-pigdatediff

1 Answers

7
votes

Are deactivation_date and activation_date of type DateTime?

If not, the following should help:

DATA_OUTPUT  = foreach MY_DATA generate key_one,
          DaysBetween(ToDate(deactivation_date, 'yyyy-MM-dd HH:mm:ss'), ToDate(activation_date, 'yyyy-MM-dd HH:mm:ss')) as days_between_two_dates;