Bigquery to Bigtable data transfer using google dataflow in python

1
votes

We are using the dataflow pipeline written in JAVA for transfering data from bigquery to bigtable. e.g. https://github.com/GoogleCloudPlatform/cloud-bigtable-examples/blob/master/java/dataflow-connector-examples/src/main/java/com/google/cloud/bigtable/dataflow/example/BigQueryBigtableTransfer.java

I am trying to write same code in python. But I am not able to get the bigtable dataflow connector for python. Any clue how it can be done?

1
pythongoogle-bigquerygoogle-cloud-dataflowdataflowbigtable
The Python SDK doesn't have support for Bigtable yet. beam.apache.org/documentation/io/built-in - Graham Polley
@GrahamPolley Thanks. Do you have any idea when google guys are planning to introduce it? - MANISH ZOPE
The Cloud Bigtable client is actively being changed to be able to support Beam. For now, we'll see if we ought to create a sample of how you can do the Bigtable side of things. - Solomon Duskis
There is BigTable Python/batch Connector available, only for write. github.com/apache/beam/blob/master/sdks/python/apache_beam/io/… - Julio Oliveira

1 Answers

1
votes

Just not to leave the question unanswered, as Graham Polley commented: "The Python SDK doesn't have support for Bigtable yet.". I see that engineer from Bigtable is already involved, but if you want, you can also create a feature request in Public Issue Tracker.