Efficient way to delete multiple rows in HBase

Question

Is there an efficient way to delete multiple rows in HBase or does my use case smell like not suitable for HBase?

There is a table say 'chart', which contains items that are in charts. Row keys are in the following format: chart|date_reversed|ranked_attribute_value_reversed|content_id

Sometimes I want to regenerate chart for a given date, so I want to delete all rows starting from 'chart|date_reversed_1' till 'chart|date_reversed_2'. Is there a better way than to issue a Delete for each row found by a Scan? All the rows to be deleted are going to be close to each other.

I need to delete the rows, because I don't want one item (one content_id) to have multiple entries which it will have if its ranked_attribute_value had been changed (its change is the reason why chart needs to be regenerated).

Being a HBase beginner, so perhaps I might be misusing rows for something that columns would be better -- if you have a design suggestions, cool! Or, maybe the charts are better generated in a file (e.g. no HBase for output)? I'm using MapReduce.

Answer 1

Firstly, coming to the point of range delete there is no range delete yet in HBase, AFAIK. But there is a way to delete more than one rows at a time in the HTableInterface API. For this simply form a Delete object with row keys from scan and put them in a List and use the API, done! To make scan faster do not include any column family in the scan result as all you need is the row key for deleting whole rows.

Secondly, about the design. First my understanding of the requirement is, there are contents with content id and each content has charts generated against them and those data are stored; there can be multiple charts per content via dates and depends on the rank. In addition we want the last generated content's chart to show at the top of the table.

For my assumption of the requirement I would suggest using three tables - auto_id, content_charts and generated_order. The row key for content_charts would be its content id and the row key for generated_order would be a long, which would auto-decremented using HTableInterface API. For decrementing use '-1' as the amount to offset and initialize the value Long.MAX_VALUE in the auto_id table at the first start up of the app or manually. So now if you want to delete the chart data simply clean the column family using delete and then put back the new data and then make put in the generated_order table. This way the latest insertion will also be at the top in the latest insertion table which will hold the content id as a cell value. If you want to ensure generated_order has only one entry per content save the generated_order id first and take the value and save it into content_charts when putting and before deleting the column family first delete the row from generated_order. This way you could lookup and charts for a content using 2 gets at max and no scan required for the charts.

I hope this is helpful.

Answer 2

You can use the BulkDeleteProtocol which uses a Scan that defines the relevant range (start row, end row, filters).

See here

Answer 3

I ran into your situation and this is my code to implement what you want

Scan scan = new Scan();
    scan.addFamily("Family");
    scan.setStartRow(structuredKeyMaker.key(starDate));
    scan.setStopRow(structuredKeyMaker.key(endDate + 1));
try {
    ResultScanner scanner = table.getScanner(scan);


    Iterator<Entity> cdrIterator = new EntityIteratorWrapper(scanner.iterator(), EntityMapper.create(); // this is a simple iterator that maps rows to exact entity of mine, not so important ! 

    List<Delete> deletes = new ArrayList<Delete>();
    int bufferSize = 10000000; // this is needed so I don't run out of memory as I have a huge amount of data ! so this is a simple in memory buffer
    int counter = 0;
    while (entityIterator.hasNext()) {
        if (counter < bufferSize) {
                            // key maker is used to extract key as byte[] from my entity 
            deletes.add(new Delete(KeyMaker.key(entityIterator.next())));
            counter++;

        } else {
            table.delete(deletes);
            deletes.clear();
            counter = 0;
        }
    }

    if (deletes.size() > 0) {
        table.delete(deletes);
        deletes.clear();
    }

} catch (IOException e) {
    e.printStackTrace();
}