GoogleApiException: Google.Apis.Requests.RequestError Backend Error [500] when streaming to BigQuery

Question

I'm streaming data to BigQuery for the past year or so from a service in Azure written in c# and recently started to get increasing amount of the following errors (most of the requests succeed):

Message: [GoogleApiException: Google.Apis.Requests.RequestError An internal error occurred and the request could not be completed. [500] Errors [ Message[An internal error occurred and the request could not be completed.] Location[ - ] Reason[internalError] Domain[global] ] ]

This is the code I'm using in my service:

public async Task<TableDataInsertAllResponse> Update(List<TableDataInsertAllRequest.RowsData> rows, string tableSuffix)
        {
            var request = new TableDataInsertAllRequest {Rows = rows, TemplateSuffix = tableSuffix};
            var insertRequest = mBigqueryService.Tabledata.InsertAll(request, ProjectId, mDatasetId, mTableId);

            return await insertRequest.ExecuteAsync();
        }

Graham Polley Graham Polley · Accepted Answer · 2017-02-19T22:51:12

Just like any other cloud service, BigQuery doesn't offer a 100% uptime SLA (it's actually 99.9%), so it's not uncommon to encounter transient errors like these. We also receive them frequently in our applications.

You need to build exponential backoff-and-retry logic into your application(s) to handle such errors. A good way of doing this is to use a queue to stream your data to BigQuery. This is what we do and it works very well for us.

Some more info:

GoogleApiException: Google.Apis.Requests.RequestError Backend Error [500] when streaming to BigQuery

1 Answers