While removing a couple of nodes from our hbase cluster, we prematurely exited the hadoop safemode (before decommisioning finished). When we ran a hdfs fsck, we realized that there were a few missing blocks. We immediately added the nodes we removed back into the cluster. The fsck still reports that some blocks are missing.
In the mean time, HBase is unable to find these missing blocks from the regions. What's the best way to fix things? We know the location of the HDFS files which HBase is unable to find.
UPDATE:
Correction - the namenode only has the meta data about the missing block. The actual blocks are missing. How do we now fix HBase to remove these data blocks from the region servers?
