Moving data from Teradata to Snowflake

0
votes

Trying to move data from Teradata to Snowflake. Have created a process to run TPT scripts for each table to generate files for each table.

Files are also split to achieve concurrency while running COPY INTO in snowflake.

Need to understand what is the best way to move those Files from On Prem Linux Machine to Azure ADLS. Considering files in Terabyte size.

Does Azure provide any mechanism to move these files or can we directly create files on ADLS from Teradata?

1
azureteradatasnowflake-cloud-data-platform
Considering files in Terabyte size. Snowflake recommends 100-250 MB files and recommends against files larger than 100 GB. This helps parallelize file loads. – Greg Pavlik
what file format are you exporting from Teradata csv / other ? see this post: stackoverflow.com/questions/50062474/… – john.da.costa

1 Answers

0
votes

The best approach to load data to snowflake via external table if you have the Azure Blob Storage or ADLS Gen2. Load data to blob storage and create external table and then load data data to snowflake.