psycopg2.errors.ConnectionFailure: SSL SYSCALL error: EOF detected

Question

I have a Jenkins job(runs every 3 hours) which ports data from a remote server to my data ware house, usually it will take about 2 hours for it to complete and recently i'm getting the below error.

The Jenkins job connect the remote server via SSH and calls a PostgreSQL procedure (load_foreign_schema_postgresql) via Python script(sql_runner_procedure).

i have created another Jenkins job(runs every 2 minute) for porting a specific table as we want that table to be updated very frequently,So the call for that job is also same and the above job is running successively few times and failing with the above mentioned error on certain schedules, So anyone can help me to figure out why this is happening and how can i resolve it?

Error:

+ python -u sql_runner_procedure.py /home/etl/bi/datawrapper/data_loader.sql
##################Executing############################## 

CALL load_foreign_schema_postgresql('odoo_server', 'public', 'dl', 'ft', log_table_schema_name := 'log',
                                    log_table_name := 'fr_table'
--     ,recreate_indices := FALSE
    );
Traceback (most recent call last):
  File "sql_runner_procedure.py", line 23, in <module>
    pg_cur.execute(sql)
psycopg2.errors.ConnectionFailure: SSL SYSCALL error: EOF detected
CONTEXT:  remote SQL command: START TRANSACTION ISOLATION LEVEL REPEATABLE READ
SQL statement "CREATE TABLE dl.account_account_type AS
                    SELECT *
                    FROM ft.account_account_type;"
PL/pgSQL function load_foreign_schema_postgresql(text,text,text,text,boolean,boolean,text,text) line 151 at EXECUTE

Process leaked file descriptors. See https://jenkins.io/redirect/troubleshooting/process-leaked-file-descriptors for more information
Build step 'Execute shell' marked build as failure
Finished: FAILURE

i'm running the Jenkins Job and the build script inside is this,

First connect the server with SSH then,

        export PYTHONPATH=/home/etl/bi/
        cd /home/etl/bi/
            python -u sql_runner_procedure.py /home/etl/bi/datawrapper/data_loader.sql

        kill $(ps -ef | grep [m]ysql_tunnel | awk '{print $2}')

Sandeep Sandeep · Accepted Answer · 2019-08-22T13:53:34

Mostly as per the description, i guess it must be due to the overlapping of the jobs.You should check whether the Job's are running with same ports , If you are using port forwarding.You should change the port or create a schedule such as these jobs won't overlap.

psycopg2.errors.ConnectionFailure: SSL SYSCALL error: EOF detected

1 Answers