Try to confirm my understanding of HBase and MapReduce behavior

0
votes

I'm trying to do some process on my HBase dataset. But I'm pretty new to the HBase and Hadoop ecosystem.

I would like to get some feedback from this community, to see if my understanding of HBase and the MapReduce operation on it is correct.

Some backgrounds here:

  1. We have a HBase table that is about 1TB, and exceeds 100 million records.2. It has 3 region servers and each region server contains about 80 regions, making the total region 240.3. The records in the table should be pretty uniform distributed to each region, from what I know.

And what I'm trying to achieve is that I could filter out rows based on some column values, and export those rows to HDFS filesystem or something like that.

For example, we have a column named "type" and it might contain value 1 or 2 or 3. I would like to have 3 distinct HDFS files (or directories, as data on HDFS is partitioned) that have records of type 1, 2, 3 respectively.

From what I can tell, MapReduce seems like a good approach to attack these kinds of problems.

I've done some research and experiment, and could get the result I want. But I'm not sure if I understand the behavior of HBase TableMapper and Scan, yet it's crucial for our code's performance, as our dataset is really large.

To simplify the issue, I would take the official RowCounter implementation as an example, and I would like to confirm my knowledge is correct.

So my questions about HBase with MapReduce is that:

  1. In the simplest form of RowCounter (without any optional argument), it is actually a full table scan. HBase iterates over all records in the table, and emits the row to the map method in RowCounterMapper. Is this correct?

  2. The TableMapper will divide the task based on how many regions we have in a table. For example, if we have only 1 region in our HBase table, it will only have 1 map task, and it effectively equals to a single thread, and does not utilize any parallel processing of our hadoop cluster?

  3. If the above is correct, is it possible that we could configure HBase to spawn multiple tasks for a region? For example, when we do a RowCounter on a table that only has 1 region, it still has 10 or 20 tasks, and counting the row in parallel manner?

Since TableMapper also depends on Scan operation, I would also like to confirm my understanding about the Scan operation and performance.

  1. If I use setStartRow / setEndRow to limit the scope of my dataset, as rowkey is indexed, it does not impact our performance, because it does not emit full table scan.

  2. In our case, we might need to filter our data based on their modified time. In this case, we might use scan.setTimeRange() to limit the scope of our dataset. My question is that since HBase does not index the timestamp, will this scan become a full table scan, and does not have any advantage compared to we just filter it by our MapReduce job itself?

Finally, actually we have some discussion on how we should do this export. And we have the following two approaches, yet not sure which one is better.

  1. Using the MapReduce approach described above. But we are not sure if the parallelism will be bound by how many regions a table has. ie, the concurrency never exceeds the region counts, and we could not improve our performance unless we increase the region.

  2. We maintain a rowkey list in a separate place (might be on HDFS), and we use spark to read the file, then just get the record using a simple Get operation. All the concurrency occurs on the spark / hadoop side.

I would like to have some suggestions about which solution is better from this community, it will be really helpful. Thanks.

1
hadoopmapreducehbase

1 Answers

1
votes

Seems like you have a very small cluster. Scalability is dependent on number of region servers(RS) also. So, just by merely increasing number of regions in table without increasing number of region servers wont really help you speed up the job. I think 80 Regions/RS for that table itself is decent enough. I am assuming you are going to use TableInputFormat, it works by running 1 mapper/region and performs server side filter on basis of scan object. I agree that scanning using TableInputFormat is optimal approach to export large amount of data from hbase but scalability and performance not just proportional to number of regions. There are many many other factors like # of RS, RAM and Disk on each RS, uniform distribution of data are some of them.

In general, I would go with #1 since you just need to prepare a scan object and then hbase will take care of rest.

#2 is more cumbersome since you need to maintain the rowkey state outside hbase.