Google Cloud Bigtable coprocessor support

8
votes

Google Cloud BigTable doesn't support coprocessors:

Coprocessors are not supported. You cannot create classes that implement the interface org.apache.hadoop.hbase.coprocessor.

https://cloud.google.com/bigtable/docs/hbase-differences

I can understand that coprocessors require deployment of customer code (jars) on each Tablet (RS) node. Still, Endpoint coprocessors are vital to HBase applications to ensure data locality in some scenarios. HBase extensions such as Apache Phoenix rely on Observer coprocessors for maintaining secondary indexes, so lack of coprocessor support looks like a major area of incompatibility to me.

Is Coprocessor support possible in the future and what workarounds exist for executing custom Java "stored procedures" on BigTable Tablets?

UPDATE 1: List of Apache Phoenix coprosessors:

  • GroupedAggregateRegionObserver
  • Indexer
  • MetaDataEndpointImpl
  • MetaDataRegionObserver
  • ScanRegionObserver
  • SequenceRegionObserver
  • ServerCachingEndpointImpl
  • UngroupedAggregateRegionObserver
1
hbasephoenixgoogle-cloud-bigtable
Why a downvote on this question? The policy says "Use your downvotes whenever you encounter an egregiously sloppy, no-effort-expended post". There's really not much hosted BigTable information out there. – Sergei Rodionov
This is a good question for the [email protected] list. But I will forward it to the team. – Les Vogel - Google DevRel
@Les Thank you. I signed up for that list a day ago so I'm watching it too. Aside from Phoenix (SQL layer on top of HBase), our own co-processors mostly perform a set of additional deletions and compactions (lossless and lossy compression). We're leaning toward developing MapReduce jobs that perform the same task so we could run M/R jobs on BigTable cluster as described in documentation. – Sergei Rodionov
As I understand it, we are unlikely to support custom co-processors, but it may be possible for us to support specific general functionality -- though no promises as to when. The place to express your needs is the discussion list. – Les Vogel - Google DevRel

1 Answers

2
votes

Custom co-processors are not supported as of June 1, 2015.