Region splitting in Hbase

3
votes

How many regions can be served by one region server? Basically i got a hadoop job which is doing bulk load in hbase but looking at hbase master UI , i could see that whole data traffic is moving to single region server.Because of which single region server is getting overloaded and others are ideal. I wanted to distributed these regions across region server to keep my cluster balanced.

I am not too sure but I am assuming that there should be some configuration for defining number of regions per regions server. Correct me if I am wrong here.

Currently I am using default Auto Split policy for my hbase tables.

Let me know how do i determine this or any other way to handle region distributions?

Thanks

2
hadoophbasebulk-load
Are the regions splitting but residing on that single region server? If that's the case, the balancer might have gotten disabled. To balance by hand, you can go into the hbase shell and run balancer. To enable and disable the balancer, you can issue balance_switch true or balance_switch false. – brandon.bell
@brandon: Yes . In my case multiple regions are getting created once it reaches to configured region size but it is residing on single region server.I am not too sure about balancer at this moment? Do i need to manually switch on the balancer? – Shashi
HBase balancer is usually enabled by default, however I've had occasions where it gets switched off for one reason or another. The balancer, when enabled, will run every 5 minutes (by default). You can initiate it manually in the Hbase shell as well as check if it's enabled or disabled. See hbase.apache.org/0.94/book/… for information regarding the balancer. – brandon.bell
Thank you brandon. I will check status of balancer.In case it is off then I will try to run job again by making it on. – Shashi
can you approve answer ? – halil

2 Answers

3
votes

I assume you created your table with one split, then it is auto-splitting based on your data size. Hbase balance table splits according to number of split per region server, so if you want to have multiple region servers serving multiple splits, you should pre-split your table based on some logic, or click split in user interface when use have some data in table.

You can read this good detailed article http://hortonworks.com/blog/apache-hbase-region-splitting-and-merging/

If any one know how to distribute splits based on load of that table, it would be good.

-2
votes

Let me have a guess.

  1. You created a table without presplitting.
  2. bulk-loaded huge data into the region.

Now you got the a problem. try the operations as follows:

while (exist region bigger than 10G ) do 1. split the region which is bigger than 10G in hbase shell. 2. major compact the daughter regions. done

tips.

presplit the table before bulk-loading.