How to use oozie to find data path in HDFS with uncertain date

0
votes

I have a Hive table partitioned by date(e.g. 20150730)

Furthermore, I created a hive query which consumes today's partition date and the most recent previous partition date which is not necessary to be yesterday. (e.g. 20150730 and 20150728)

I want to use oozie to execute this hive query daily, but how can I find the previous partition date? (again the previous proportional date is unpredictable)

1
hadoophivehdfsoozieoozie-coordinator
can you create hive query that can answer what is the previous partition date ? – Mzf

1 Answers

0
votes

The Same kind of ques have been answed in the link. Hive Data selecting latest value based on timestamp

please have a check, you might need to apply some modification in mention example.

As you mention Date is unpredictable, but believe it should be in format

also You can try someting like

SELECT DISTINCT(date) FROM partition_table ORDER BY usage_dt DESC limit 2;

these will give u previous Date even though if its not predictate...