Hbase rowkey design for time range query

4
votes

I need to get page view of product from start date to end date. So i design Hbase rowkey like this.

ProductId(md5)_productId_dateKey(yyyymmdd)  d:pageview.

Example: productId is 1180560. The rowkey from 1st August to 30th August will be:

0f0e02e9e24c29ad2c2b28e37e4f250b_1180560_20150801

.....

0f0e02e9e24c29ad2c2b28e37e4f250b_1180560_20150830

So i will easily get product pageview for time range by scanning rowkey with start row, stop row

My questions are:

Does this design avoid hotspotting ?

The length of rowkey seem too long. Will it effect the performance ?.

Is there a way to shorten the rowkey and still ensure avoid hotspotting, collision.

Is there better design for this case ?

2
hbase

2 Answers

1
votes

Hotspotting : It is always advisable not to use sequential row keys, even though you get better scan results. More info here.

Length of row key - For each cell, rowkey details, column family, and qualifier details are stored. So it is always advisable to keep them as shot as possible, mainly because the same information is repeated on large scale.

So whats next - salt usage and its prefixing will help to distribute the rows among region servers. This can help you

0
votes

You don't need to md5 your productId. Just compute remainder according to no of region server you would like. i.e. key split.

So, if you want 0 to 4 (total 5 split) then you need to build you key as 

Bytes(StringValue(productID%5)) + Bytes(productId) + Bytes(timestamp)

For example for product id 1180560

1180560%5=0
Bytes.toBytes(1180560)
Bytes.toBytes(timestamp)

Now you can query your data by using prefix scan.

Example code in Java to get 5 days of data for same product:

final Calendar cal = Calendar.getInstance();
cal.add(Calendar.DATE, -2); // deduct 2 days
byte[] STARTROW = Bytes.add(Bytes.toBytes( productID%5),Bytes.toBytes(productID),Bytes.toBytes(cal.getTime().getTime()));
cal.add(Calendar.DATE, +3); //add 3 days
byte[] ENDTROW = Bytes.add(Bytes.toBytes( productID%5),Bytes.toBytes(productID),Bytes.toBytes(cal.getTime().getTime()));
byte[] prefix = Bytes.add(Bytes.toBytes(String.valueOf(remainder)), 
Scan scan = new Scan(STARTROW,ENDTROW);
scan.setFilter(prefixFilter);