400 Error while reading data, error message: CSV table encountered too many errors, giving up. Rows: 29274; errors: 1

1
votes

I want to load data to bigQuery using python, but I get the following error:

400 Error while reading data, error message: CSV table encountered too many errors, giving up. Rows: 29274; errors: 1. Please look into the errors[] collection for more details.

I have tried step like this link CSV Table Encountered Too Many Errors, Giving Up but I have not permission to view feature Transfer in google cloud platform.

I have no idea where the error happens and how to solve it?

As a reference, I find the solution in this link stackoverflow but can not understand.

1
pythoncsvgoogle-bigquery
What is in the errors[] collection? - CDJB
that is not show where error[] collection, how can i know ? - hinafaya
You can try to load the data via the webUI and check the error[] object from there or check the Job History for more information from the left side top menu - Tamir Klein
Agree with @TamirKlein here, instead of writing to BigQuery writ your data out to a .csv then upload this manually. Checking the logs should guide you to the problem, which is likely a null value or some unexpected character around row 29274. - Ben P
If you are using Python code then you have something like load_job = client.load_table_from_uri.... , you should be able to access the errors[] collection like load_job.errors. - ECris

1 Answers

0
votes

It may be necessary for you to open a ticket directly with Google Cloud Platform Support. Before that you can follow this article to try and inspect the error collection.

But in order to do this you need the appropriate permissions for the Transfer service(otherwise one of your admins must do this).

You may be able to assign the needed permissions, depending on the roles already assigned to your account. The role needed is "BigQuery Data Transfer Service Agent". This document explains how to assign roles to a member.