Continuous data for Snowpipe set up billing question Rest api vs Snowflake UI DDL?

0
votes

I am interested in automating the data job we run weekly. I have engaged with snowsql and just started to see what Snowpipe can do.

On the note that Snowflake Internal does not have an option yet to automate data loads with cloud messaging via How Does Snowpipe Work? Note I went with trying the Snowpipe REST Endpoints.

Per the recommendation to separate the files that I copied with the copy command and where I set up Snowpipe, I made sure they were in different tables.

However, with the Snowpipe DDL and the python apks, will both the endpoints and the pipe created in the user interface appear in the Snowpipe billing section?

1
snowflake-cloud-data-platform
You can load data in an automated way using Snowpipe. I'm not sure what you mean when you say "On the note that Snowflake Internal does not have an option yet to automate data loads with cloud messaging". Snowpipe uses SQS + S3 event notifications to automate loads into tables. – Simon D
I was just mentioning and acknowledging the limitation, I would like to use snowflake stage with snowpipe to keep it all in one place. – Pumpkincloud
Wait can you use a Snowflake internal stage to ingest data? docs.snowflake.net/manuals/sql-reference/sql/… the first example looks like it does that exactly. – Pumpkincloud

1 Answers

0
votes

Yes - for example.

This is an example for Python.

Then CREATE PIPE in a worksheet using the DDL.

Do they both show up?

Try creating a pipe in both your connections and view them in the Billing & Ussage section in the Snowpipe warehouse. Make sure you are using the ACCOUNTADMIN to view the usage information. See this.

Hope that helps.