Below is what am trying in order,
- create partitioned table in hive based on current hour.
- use spark hive context and perform msck repair table.
- delete the hdfs folders of one of the added partitions manually.
- use spark hive context again and perform a> msck repair this does not remove the partition added already with no hdfs folder. seems like known behavior with respect to "msck repair" b> select * from tablexxx where (existing partition); Fails with exception : Filenotfound exception pointing to hdfs folder which was deleted manually.
Any insights on this behavior would be of great help.