500 Backend Error on streaming API

2
votes

so I have a scheduler running which pushes 500 rows to bigquery at every 2 seconds using "insertAll" streaming API. It inserts successfully for initial few batches but after sometime it receives following exception
com.google.api.client.googleapis.json.GoogleJsonResponseException: 500 Internal Server Error
{ "code" : 500, "errors" : [ { "domain" : "global", "message" : "Backend Error", "reason" : "backendError" } ], "message" : "Backend Error" }

Any idea what could be the reason?

Thanks, Ved

1
google-bigquery
Do you have a job id? - Pentium10
Alas, insertAll doesn't create a job id, so insertAll issues can be much more difficult to track down. Can you send your project id and an approximate time that the last error happened? Also, error 500s are bigquery bugs, so you can also report them here: code.google.com/p/google-bigquery - Jordan Tigani
Hi, Thanks for the swift response. It happened just now with the above mentioned scenario. It gave this error after pushing approx 4000 rows at the rate of 500 rows at every 2 seconds. My Project-Id is "deft-virtue-628". And this happened just now at 10:30 IST (12:00 AM EST). Will report at above mentioned link as well, but if possible please let us know if we can get rid of it as in our scenario it requires to push the data at above rate by 24x7 and this can be show stopper for us. Thanks. - ved

1 Answers

0
votes

Yes, we have the same problem. As I assume it'll take a while before this bug being solved, I'm wondering what's the suggested workaround we can do for now? Right now I put the data back into my request queue and resent them until I got a success. Will this cause data duplication?