Snowflake COPY command directory read

Question

COPY INTO SALES.SF_DEV.CUSTOMER_PARQUET_TGT FROM @EXTERNAL_STAGE_PARQUET_DEMO/partioned/customer/Country=INDIA/ MATCH_BY_COLUMN_NAME = CASE_SENSITIVE FILE_FORMAT =  (TYPE = PARQUET);

The above command will load all the files recursively from dir Country=INDIA and all its sub-directories.

Que: Do we have an option to control recursive file loading, what I mean to load only files from Country=INDIA not from its sub-directories?

P.S. This image is with the pattern suggested by @gokhan

Gokhan Atil Gokhan Atil · Accepted Answer · 2021-03-25T12:30:32

Can you try to use the pattern option?

COPY INTO SALES.SF_DEV.CUSTOMER_PARQUET_TGT 
FROM @EXTERNAL_STAGE_PARQUET_DEMO/partioned/customer/Country=INDIA/
MATCH_BY_COLUMN_NAME = CASE_SENSITIVE FILE_FORMAT =  (TYPE = PARQUET)
PATTERN = '[^\/]*';

The pattern is to prevent any sub-directories (files containing the slash characters).

My test environment:

ls @my_stage/test/Country=INDIA/; 

+--------------------------------------------------------------------------+
|                                   name                                   |
+--------------------------------------------------------------------------+
| s3://mysecretbucketname/test/test/Country=INDIA/data_0_0_0.csv.gz        |
| s3://mysecretbucketname/test/test/Country=INDIA/hododo/data_0_0_0.csv.gz |
+--------------------------------------------------------------------------+

ls @my_stage/test/Country=INDIA/ pattern = '[^\/]*';

+-------------------------------------------------------------------+
|                               name                                |
+-------------------------------------------------------------------+
| s3://mysecretbucketname/test/test/Country=INDIA/data_0_0_0.csv.gz |
+-------------------------------------------------------------------+

Snowflake COPY command directory read

1 Answers