How to import a PostgreSQL data file into Amazon redshift

3
votes

I am trying to import a PostgreSQL data file into Amazon redshift using my command line. I did import the schema file but can not import data file. It seems that data insertion in amazon redshift is a bit different.

I want to know all kinds of way of importing data file into redshift using command line.

UPDATE

My data file looks like :

COPY actor (actor_id, first_name, last_name, last_update) FROM stdin; 
0 Chad Murazik 2014-12-03 10:54:44
1 Nelle Sauer 2014-12-03 10:54:44
2 Damien Ritchie 2014-12-03 10:54:44
3 Casimer Wiza 2014-12-03 10:54:44
4 Dana Crist 2014-12-03 10:54:44
....

I typed the following command from CLI:

PGPASSWORD=**** psql -h testredshift.cudmvpnjzyyy.us-west-2.redshift.amazonaws.com -p 5439 -U abcd -d pagila -f /home/jamy/Desktop/pag_data.sql`

And then got error like :

ERROR: LOAD source is not supported. (Hint: only S3 or DynamoDB or EMR based load is allowed

2
postgresqlamazon-web-servicesimportcommand-line-interfaceamazon-redshift
What do you mean by "data file"? Is that a file with SQL INSERT statements? CSV? Output from pg_dump? - Patrick
output from pg_dump for example: COPY actor (actor_id, first_name, last_name, last_update) FROM stdin; 0 Chad Murazik 2014-12-03 10:54:44 1 Nelle Sauer 2014-12-03 10:54:44 2 Damien Ritchie 2014-12-03 10:54:44 3 Casimer Wiza 2014-12-03 10:54:44 4 Dana Crist 2014-12-03 10:54:44 - Zahidul Islam Jamy
"all kind of way[s]"? There are endless ways. But I think it sounds like you want to just feed the dump into psql. Other options include pg_restore, the COPY command, or using an ETL tool like Talend Studio, Pentaho Kettle or CloverETL. Anyway, what's the error message? - Craig Ringer
command: PGPASSWORD=**** psql -h testredshift.cudmvpnjzyyy.us-west-2.redshift.amazonaws.com -p 5439 -U abcd -d pagila -f /home/jamy/Desktop/pag_data.sql ERROR: LOAD source is not supported. (Hint: only S3 or DynamoDB or EMR based load is allowed) @CraigRinger - Zahidul Islam Jamy

2 Answers

3
votes

Dump your table using a CSV format:

\copy <your_table_name> TO 'dump_fulename.csv' csv header NULL AS '\N'

Upload it to S3, and read it from redshift using:

COPY schema.table FROM 's3:/...' WITH CREDENTIALS '...' CSV;

Source: Importing Data into Redshift from MySQL and Postgres

1
votes

You can't use pg_dump: unload all your data to s3 and use a copy command to load it into Redshift. This is a common mistake.