0
votes

CSThis is supposed to read in a and then write it to bigquery. When it runs, however, nothing is written, and there are no errors logged. I read that I need to write a csv and then turn it into an Octet Stream. I am not sure whether or not this is compatible with google bigquery. I also tried removing the setSourceUris, but nothing showed up as an error still.

function test(){
try{
 var tableReference = BigQuery.newTableReference();
 tableReference.setProjectId(PROJECT_ID);
 tableReference.setDatasetId(datasetId);
 tableReference.setTableId(tableId);
 var schema = "CUSTOMER:string, CLASSNUM:integer, CLASSDESC:string, CSR:string, CSR2:string,     INSURANCE:string, REFERRALGENERAL:string, REFERRALSPECIFIC:string, NOTES:string, INMIN:integer, INHR:integer, OUTMIN:integer, OUTHR:integer, WAITMIN:integer, WAITHR:integer, DATETIMESTAMP:float, DATEYR:integer,DATEMONTH:integer, DATEDAY:integer";
 var load = BigQuery.newJobConfigurationLoad();
 load.setDestinationTable(tableReference);
 load.setSourceUris(URIs);
 load.setSourceFormat('CSV');
 load.setSchema(schema);
 load.setMaxBadRecords(0);
 load.setWriteDisposition('WRITE_TRUNCATE');

 var configuration = BigQuery.newJobConfiguration();
 configuration.setLoad(load);

 var newJob = BigQuery.newJob();
 newJob.setConfiguration(configuration);


 var loadr = DriveApp.getFilesByName("test.csv");
 var x = loadr.next().getBlob();
 Logger.log(x.getDataAsString());

 var d = DriveApp.getFilesByName("test.csv");
 var id = d.next().getId();
 Logger.log(id);
 var data = DocsList.getFileById(id).getBlob().getDataAsString();
 var mediaData = Utilities.newBlob(data, 'application/octet-stream');

 BigQuery.Jobs.insert(newJob, PROJECT_ID, mediaData) 
} 
catch(error){Logger.log("A" + error.message);} 
}
1
Why you have NEWLINE_DELIMITED_JSON source format for a CSV file?Pentium10
I change it to CSV, but its asking for a URI which I cant find. Currently, I am doing the gs://project_nameCharles Wallis

1 Answers

0
votes

The Apps Script documentation contains some sample code showing how to load a CSV into bigquery: https://developers.google.com/apps-script/advanced/bigquery#load_csv_data