BigQuery Data Transfer Services - python client library

1
votes

Currently, we have lots of requests in scheduling a BigQuery data transfer for DoubleClick Campaign Manager. So we follow the instruction here to use Data Transfer V2.0. It requires manual set up and permission with user account.

However, I notice that Data Transfer Service support REST-API

But it looks like the REST-API only supports version=v1. My questions:

  • What are the differences between REST-API v1.0 and Data Transfer V2.0?
  • Can I use the REST-API with Data Transfer V2.0?

I am looking for a python client library of "bigquery data transfer services", specially for how to launch the bigquery job to run or schedule a backfill to load DCM data to BigQuery.

How could I do the above works to set up "bigquery data transfer" automatically with service account instead of manual set up with user account?

Thanks!

1
pythongoogle-bigquerydata-transfer
just as fyi - you can do all this in magnus with rest api - most likely you know that :o) - Mikhail Berlyant
Sure, thanks Mikhail. Let me try that instead. - Tuan Vu
i think i already have an example there for using https://storagetransfer.googleapis.com/v1/transferJobs for setting and scheduling transfer from S3 to GCS. so it should be similar (at least conceptually) - Mikhail Berlyant

1 Answers

0
votes

The Data Transfer Service API version is currently v1. It does support DoubleClick Campaign Manager "Data Transfer v2". They are two different sets of APIs: Data Transfer Service vs. DoubleClick Campaign Manager Data Transfer.