Why Apache server throws syntax error with snowflake ingest module in python

Question

I have a flask application running on apache server in the Linux box(ec2 instance), which calls the python script that does the work of moving the files from s3 to snowflake. The script works fine if executed directly. It throws the below error for the snowflake's ingest python module when trying to host it on apache using virtualhost and mod_wsgi. The module is installed in the default site packages.

Apache server version: 2.4.41

Python Version: 3.7

snowflake: snowflake_connector_python-2.1.3-py3.7-nspkg.pth

OS: AMAZON linux 2

Below is the apache error log, when i try to restart:

[Fri Jan 10 15:47:53.703083 2020] [:error] [pid 19755] [remote 67.79.202.36:20] mod_wsgi (pid=19755): Target WSGI script '/var/www/FLASKAPPS/snowflakeingestapp/snowflakeingestapp.wsgi' cannot be loaded as Python module.

[Fri Jan 10 15:47:53.703113 2020] [:error] [pid 19755] [remote 67.79.202.36:20] mod_wsgi (pid=19755): Exception occurred processing WSGI script '/var/www/FLASKAPPS/snowflakeingestapp/snowflakeingestapp.wsgi'.

[Fri Jan 10 15:47:53.703129 2020] [:error] [pid 19755] [remote 67.79.202.36:20] Traceback (most recent call last):

[Fri Jan 10 15:47:53.703142 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/var/www/FLASKAPPS/snowflakeingestapp/snowflakeingestapp.wsgi", line 4, in

[Fri Jan 10 15:47:53.703176 2020] [:error] [pid 19755] [remote 67.79.202.36:20] from snowflakeingestapp import app as application

[Fri Jan 10 15:47:53.703181 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/var/www/FLASKAPPS/snowflakeingestapp/init.py", line 6, in

[Fri Jan 10 15:47:53.703206 2020] [:error] [pid 19755] [remote 67.79.202.36:20] import snowflake_ingest

[Fri Jan 10 15:47:53.703210 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/var/www/FLASKAPPS/snowflakeingestapp/snowflake_ingest/init.py", line 1, in

[Fri Jan 10 15:47:53.703225 2020] [:error] [pid 19755] [remote 67.79.202.36:20] import ingest_ecomm_json_snowpipe

[Fri Jan 10 15:47:53.703230 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/var/www/FLASKAPPS/snowflakeingestapp/snowflake_ingest/ingest_ecomm_json_snowpipe.py", line 20, in

[Fri Jan 10 15:47:53.703296 2020] [:error] [pid 19755] [remote 67.79.202.36:20] from snowflake.ingest import SimpleIngestManager

[Fri Jan 10 15:47:53.703300 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/usr/local/lib/python3.7/site-packages/snowflake/ingest/init.py", line 1, in

[Fri Jan 10 15:47:53.703315 2020] [:error] [pid 19755] [remote 67.79.202.36:20] from .simple_ingest_manager import SimpleIngestManager, StagedFile

[Fri Jan 10 15:47:53.703342 2020] [:error] [pid 19755] [remote 67.79.202.36:20] File "/usr/local/lib/python3.7/site-packages/snowflake/ingest/simple_ingest_manager.py", line 65

[Fri Jan 10 15:47:53.703345 2020] [:error] [pid 19755] [remote 67.79.202.36:20] def init(self, account: Text, user: Text, pipe: Text, private_key: Text,

[Fri Jan 10 15:47:53.703347 2020] [:error] [pid 19755] [remote 67.79.202.36:20] ^

[Fri Jan 10 15:47:53.703350 2020] [:error] [pid 19755] [remote 67.79.202.36:20] SyntaxError: invalid syntax

d.hoeffer d.hoeffer · Accepted Answer · 2020-01-12T08:20:32

It looks like when you're executing this from mod_wsgi, you're using Python 2 instead of Python 3. The error message you see is happening because Snowflake's simple ingest manager uses type hints (via the typing module), which are not available in Python 2.

Here's a simple fragment that fails with the same error in Python 2 but succeeds in Python 3:

from typing import Text
def foo(bar: Text):
    print(bar)
foo('huh')

Why Apache server throws syntax error with snowflake ingest module in python

1 Answers