Python: decryption failed or bad record mac when calling from Thread

4
votes

I'm getting a "decryption failed or bad record mac" error in this code-fragment:

conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()

This is called in the run() method of a Thread, several times in a while(True) loop. I'm just opening a connection to my PostgreSQL database using the psycopg2 driver.

Any idea of how safe is opening db connections into Threads in Python? I don't know what is raising this error.

2
pythonmultithreadingpostgresqlencryptiondatabase-connection
Is the exception raised on every iteration of the loop? Which line is the one that actually raises it? - Wieland
Hello, it's raised at cursor.execute("SELECT id, ip FROM schema.table;"). It's not being thrown in every single iteration, it does few iterations right, then raises the error once and stop the execution. - Iván
Ok, looks like I've fixed the problem. I was creating too many connections and seems I was running out of memory or something. conn = None after conn.close() does the trick. Thanks for your response. - Iván
You can post your comment as an answer and even accept it after a while. In that case people can see from the question that the issue has been resolved. You will get points if people click the +1 on question or answer (like this). - Maarten Bodewes
I was facing a similar problem while trying to run Multiprocessing and Postgresql in Python. I noticed that I was using the same connection in all spawned process and hence it was failing. I passed engine to different processes and then created new connections in their respective processes. It worked! - p699

2 Answers

6
votes

Ok, looks like I've fixed the problem. I was creating too many connections and seems I was running out of memory or something. I gathered all the queries and do cursor.execute(...) once with a huge query, instead performing hundreds of small queries/connections.

conn = psycopg2.connect(...)
cursor = conn.cursor()
cursor.execute("SELECT id, ip FROM schema.table;")
rows = cursor.fetchall()
cursor.close()
conn.commit()
conn.close()
conn = None
0
votes

The cause of this issue could be, there were too many processes(multi) were trying to access PostGres, it was not able to handle that. I was using Django & PostGres in BeanStalk.

Adding 'OPTIONS': {'sslmode': 'disable'} in the database config helped.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': ...
        'USER': ....
        'PASSWORD': ...
        'HOST': ....
        'PORT': '5432',
        'OPTIONS': {
           'sslmode': 'disable',
        }
    }
}