How to list columns using COPY command with Redshift?

Question

I am trying to load a table using the COPY command and specifying the Column list as per the redshift documentation https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-column-mapping.html#copy-column-mapping-jsonpaths.

The file content of s3://mybucket/data/listing/data.csv is header followed by the file content. File content below:

c1, c2, c3, c4. c5, c6, c7
1,2,3,4,5,6,7
11,11,11,11,11,11,11
21,21,21,21,21,21,21
31,31,31,31,31,31,31
.........................
.........................
.........................

And I am using the following command to load the listing table which only has three columns c1, c2, c3.

copy listing(c1, c2, c3)
from 's3://mybucket/data/listing/data.csv' 
iam_role 'arn:aws:iam::0123456789012:role/MyRedshiftRole';
CSV

However, Redshift is not allowing the copy and failing with the following error:

1202    Extra column(s) found

Why is it? I am specifying the selected columns with the same name. What I am missing here?

Joe Harris Joe Harris · Accepted Answer · 2020-12-09T13:56:19

You cannot currently limit the columns in a COPY statement this way. You can either load all columns to a temporary table and then INSERT them into your target table or you can define the file(s) to be loaded as an external table and then INSERT directly to your target using SELECT from the external table.

https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html

How to list columns using COPY command with Redshift?

1 Answers