Loading data automatically from Oracle DB to Google BigQuery

4
votes

Good day,

I have an Oracle DB and I need to load some tables so I can query them in BigQuery.

¿Is there a way of loading the data automatically, every 24 h, to Google BigQuery?

Any way would work. It could be loading into Data Storage and creating the tables from there, or loading into Google drive from the server.

I really need some ideas, I have read a lot of articles with no luck.

1
oraclegoogle-bigquery
I'm sure that are other good options as well, but the BigQuery Partners page lists some recommendations if you're hoping to find a provider for this kind of service. - Elliott Brossard

1 Answers

4
votes

Check this tutorial by Progress:

In this tutorial the main goal will be to connect to an On-Premises Oracle database, read the data, apply a simple transformation and write it to BigQuery. The code for this project has been uploaded to GitHub for your reference.

This solution uses Dataflow and Progress' Hybrid Data Pipeline tool:

Google Cloud Dataflow is a data processing service for both batch and real-time data streams. Dataflow allows you to build pipes to ingest data, then transform and process according to your needs before making that data available to analysis tools. DataDirect Hybrid Data Pipeline can be used to ingest both on-premises and cloud data with Google Cloud Dataflow.