Is there a way through the BigQuery Java library to load a table from a local CSV file? When I tried to pass in a local file URI to JobConfigurationLoad.setSourceUris() I received the error "Source URI must be a Google-Storage location".
Related, if I need to upload the files to Google Cloud Storage first, what do I get billed for if I delete the file from cloud storage right after I load it into BigQuery?
Use the Bigquery insert method:
public Insert insert(String projectId, com.google.api.services.bigquery.model.Job content,
com.google.api.client.http.AbstractInputStreamContent mediaContent)
You have 2 implems of AbstractInputStreamContent: FileContent, InputStreamContent
For example:
val content = FileContent("application/octet-stream", new File("csv","data.csv")
val insertReq = bigquery.jobs().insert(PROJECT_ID, insertJob, content))
You can upload the file directly by using a multi-part mime request... see the bigquery import docs here. There is no example in java, but there is one in python that should be relatively easy to adapt. If you are dealing with very large files, however, you will probably be better off uploading the files to Google Cloud Storage first.
If you stage the files to Google Cloud Storage and then delete them, you may get billed for up to 24 hours of storage (I'm not certain of this, but it will certainly not be more than 24 hours of storage).
