Is there a way to show the SQL that Django is running while performing a query?
19 Answers
463
votes
See the docs FAQ: "How can I see the raw SQL queries Django is running?"
django.db.connection.queries contains a list of the SQL queries:
from django.db import connection
print(connection.queries)
Querysets also have a query attribute containing the query to be executed:
print(MyModel.objects.filter(name="my name").query)
Note that the output of the query is not valid SQL, because:
"Django never actually interpolates the parameters: it sends the query and the parameters separately to the database adapter, which performs the appropriate operations."
From Django bug report #17741.
Because of that, you should not send query output directly to a database.
If you need to reset the queries to, for example, see how many queries are running in a given period, you can use reset_queries from django.db:
from django.db import reset_queries
reset_queries()
print(connection.queries)
>>> []
87
votes
Django-extensions have a command shell_plus with a parameter print-sql
./manage.py shell_plus --print-sql
In django-shell all executed queries will be printed
Ex.:
User.objects.get(pk=1)
SELECT "auth_user"."id",
"auth_user"."password",
"auth_user"."last_login",
"auth_user"."is_superuser",
"auth_user"."username",
"auth_user"."first_name",
"auth_user"."last_name",
"auth_user"."email",
"auth_user"."is_staff",
"auth_user"."is_active",
"auth_user"."date_joined"
FROM "auth_user"
WHERE "auth_user"."id" = 1
Execution time: 0.002466s [Database: default]
<User: username>
60
votes
Take a look at debug_toolbar, it's very useful for debugging.
Documentation and source is available at http://django-debug-toolbar.readthedocs.io/.
31
votes
30
votes
No other answer covers this method, so:
I find by far the most useful, simple, and reliable method is to ask your database. For example on Linux for Postgres you might do:
sudo su postgres
tail -f /var/log/postgresql/postgresql-8.4-main.log
Each database will have slightly different procedure. In the database logs you'll see not only the raw SQL, but any connection setup or transaction overhead django is placing on the system.
18
votes
Though you can do it with with the code supplied, I find that using the debug toolbar app is a great tool to show queries. You can download it from github here.
This gives you the option to show all the queries ran on a given page along with the time to query took. It also sums up the number of queries on a page along with total time for a quick review. This is a great tool, when you want to look at what the Django ORM does behind the scenes. It also have a lot of other nice features, that you can use if you like.
16
votes
Another option, see logging options in settings.py described by this post
http://dabapps.com/blog/logging-sql-queries-django-13/
debug_toolbar slows down each page load on your dev server, logging does not so it's faster. Outputs can be dumped to console or file, so the UI is not as nice. But for views with lots of SQLs, it can take a long time to debug and optimize the SQLs through debug_toolbar since each page load is so slow.
10
votes
If you make sure your settings.py file has:
django.core.context_processors.debuglisted inCONTEXT_PROCESSORSDEBUG=True- your
IPin theINTERNAL_IPStuple
Then you should have access to the sql_queries variable. I append a footer to each page that looks like this:
{%if sql_queries %}
<div class="footNav">
<h2>Queries</h2>
<p>
{{ sql_queries|length }} Quer{{ sql_queries|pluralize:"y,ies" }}, {{sql_time_sum}} Time
{% ifnotequal sql_queries|length 0 %}
(<span style="cursor: pointer;" onclick="var s=document.getElementById('debugQueryTable').style;s.disp\
lay=s.display=='none'?'':'none';this.innerHTML=this.innerHTML=='Show'?'Hide':'Show';">Show</span>)
{% endifnotequal %}
</p>
<table id="debugQueryTable" style="display: none;">
<col width="1"></col>
<col></col>
<col width="1"></col>
<thead>
<tr>
<th scope="col">#</th>
<th scope="col">SQL</th>
<th scope="col">Time</th>
</tr>
</thead>
<tbody>
{% for query in sql_queries %}
<tr class="{% cycle odd,even %}">
<td>{{ forloop.counter }}</td>
<td>{{ query.sql|escape }}</td>
<td>{{ query.time }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
I got the variable sql_time_sum by adding the line
context_extras['sql_time_sum'] = sum([float(q['time']) for q in connection.queries])
to the debug function in django_src/django/core/context_processors.py.
9
votes
I developed an extension for this purpose, so you can easily put a decorator on your view function and see how many queries are executed.
To install:
$ pip install django-print-sql
To use as context manager:
from django_print_sql import print_sql
# set `count_only` to `True` will print the number of executed SQL statements only
with print_sql(count_only=False):
# write the code you want to analyze in here,
# e.g. some complex foreign key lookup,
# or analyzing a DRF serializer's performance
for user in User.objects.all()[:10]:
user.groups.first()
To use as decorator:
from django_print_sql import print_sql_decorator
@print_sql_decorator(count_only=False) # this works on class-based views as well
def get(request):
# your view code here
6
votes
3
votes
I believe this ought to work if you are using PostgreSQL:
from django.db import connections
from app_name import models
from django.utils import timezone
# Generate a queryset, use your favorite filter, QS objects, and whatnot.
qs=models.ThisDataModel.objects.filter(user='bob',date__lte=timezone.now())
# Get a cursor tied to the default database
cursor=connections['default'].cursor()
# Get the query SQL and parameters to be passed into psycopg2, then pass
# those into mogrify to get the query that would have been sent to the backend
# and print it out. Note F-strings require python 3.6 or later.
print(f'{cursor.mogrify(*qs.query.sql_with_params())}')
3
votes
I put this function in a util file in one of the apps in my project:
import logging
import re
from django.db import connection
logger = logging.getLogger(__name__)
def sql_logger():
logger.debug('TOTAL QUERIES: ' + str(len(connection.queries)))
logger.debug('TOTAL TIME: ' + str(sum([float(q['time']) for q in connection.queries])))
logger.debug('INDIVIDUAL QUERIES:')
for i, query in enumerate(connection.queries):
sql = re.split(r'(SELECT|FROM|WHERE|GROUP BY|ORDER BY|INNER JOIN|LIMIT)', query['sql'])
if not sql[0]: sql = sql[1:]
sql = [(' ' if i % 2 else '') + x for i, x in enumerate(sql)]
logger.debug('\n### {} ({} seconds)\n\n{};\n'.format(i, query['time'], '\n'.join(sql)))
Then, when needed, I just import it and call it from whatever context (usually a view) is necessary, e.g.:
# ... other imports
from .utils import sql_logger
class IngredientListApiView(generics.ListAPIView):
# ... class variables and such
# Main function that gets called when view is accessed
def list(self, request, *args, **kwargs):
response = super(IngredientListApiView, self).list(request, *args, **kwargs)
# Call our function
sql_logger()
return response
It's nice to do this outside the template because then if you have API views (usually Django Rest Framework), it's applicable there too.
3
votes
Django SQL Sniffer is another alternative for viewing (and seeing the stats of) raw executed queries coming out of any process utilising Django ORM. I've built it to satisfy a particular use-case that I had, which I haven't seen covered anywhere, namely:
- no changes to the source code that the target process is executing (no need to register a new app in django settings, import decorators all over the place etc.)
- no changes to logging configuration (e.g. because I'm interested in one particular process, and not the entire process fleet that the configuration applies to)
- no restarting of target process needed (e.g. because it's a vital component, and restarts may incur some downtime)
Therefore, Django SQL Sniffer can be used ad-hoc, and attached to an already running process. The tool then "sniffs" the executed queries and prints them to console as they are executed. When the tool is stopped a statistical summary is displayed with outlier queries based on some possible metric (count, max duration and total combined duration).
Here's a screenshot of an example where I attached to a Python shell
You can check out the live demo and more details on the github page.
2
votes
The following returns the query as valid SQL, based on https://code.djangoproject.com/ticket/17741:
def str_query(qs):
"""
qs.query returns something that isn't valid SQL, this returns the actual
valid SQL that's executed: https://code.djangoproject.com/ticket/17741
"""
cursor = connections[qs.db].cursor()
query, params = qs.query.sql_with_params()
cursor.execute('EXPLAIN ' + query, params)
res = str(cursor.db.ops.last_executed_query(cursor, query, params))
assert res.startswith('EXPLAIN ')
return res[len('EXPLAIN '):]
2
votes
I've made a small snippet you can use:
from django.conf import settings
from django.db import connection
def sql_echo(method, *args, **kwargs):
settings.DEBUG = True
result = method(*args, **kwargs)
for query in connection.queries:
print(query)
return result
# HOW TO USE EXAMPLE:
#
# result = sql_echo(my_method, 'whatever', show=True)
It takes as parameters function (contains sql queryies) to inspect and args, kwargs needed to call that function. As the result it returns what function returns and prints SQL queries in a console.
1
votes
To get result query from django to database(with correct parameter substitution) you could use this function:
from django.db import connection
def print_database_query_formatted(query):
sql, params = query.sql_with_params()
cursor = connection.cursor()
cursor.execute('EXPLAIN ' + sql, params)
db_query = cursor.db.ops.last_executed_query(cursor, sql, params).replace('EXPLAIN ', '')
parts = '{}'.format(db_query).split('FROM')
print(parts[0])
if len(parts) > 1:
parts = parts[1].split('WHERE')
print('FROM{}'.format(parts[0]))
if len(parts) > 1:
parts = parts[1].split('ORDER BY')
print('WHERE{}'.format(parts[0]))
if len(parts) > 1:
print('ORDER BY{}'.format(parts[1]))
# USAGE
users = User.objects.filter(email='[email protected]').order_by('-id')
print_database_query_formatted(users.query)
Output example
SELECT "users_user"."password", "users_user"."last_login", "users_user"."is_superuser", "users_user"."deleted", "users_user"."id", "users_user"."phone", "users_user"."username", "users_user"."userlastname", "users_user"."email", "users_user"."is_staff", "users_user"."is_active", "users_user"."date_joined", "users_user"."latitude", "users_user"."longitude", "users_user"."point"::bytea, "users_user"."default_search_radius", "users_user"."notifications", "users_user"."admin_theme", "users_user"."address", "users_user"."is_notify_when_buildings_in_radius", "users_user"."active_campaign_id", "users_user"."is_unsubscribed", "users_user"."sf_contact_id", "users_user"."is_agree_terms_of_service", "users_user"."is_facebook_signup", "users_user"."type_signup"
FROM "users_user"
WHERE "users_user"."email" = '[email protected]'
ORDER BY "users_user"."id" DESC
It based on this ticket comment: https://code.djangoproject.com/ticket/17741#comment:4
0
votes
0
votes
For Django 2.2:
As most of the answers did not helped me much when using ./manage.py shell. Finally i found the answer. Hope this helps to someone.
To view all the queries:
from django.db import connection
connection.queries
To view query for a single query:
q=Query.objects.all()
q.query.__str__()
q.query just displaying the object for me.
Using the __str__()(String representation) displayed the full query.
0
votes
To generate SQL for CREATE / UPDATE commands, which are immediate in Django [tested on 1.8]
def generate_update_sql(queryset, update_kwargs):
"""Converts queryset with update_kwargs
like if was: queryset.update(**update_kwargs)
qs = Event.objects.exclude(app='some')
update_kwargs = dict(description='test', action='action')
generate_update_sql(qs, update_kwargs) will return
UPDATE `api_event` SET `description` = test, `action` = action WHERE NOT (`api_event`.`app` = some)
"""
from django.db.models import sql
query = queryset.query.clone(sql.UpdateQuery)
query.add_update_values(update_kwargs)
compiler = query.get_compiler(queryset.db)
sql, params = compiler.as_sql()
return sql % params
def generate_create_sql(model, model_data):
"""Converts queryset with create_kwargs
like if was: queryset.create(**create_kwargs)
generate_create_sql(Event, dict(slug='a', app='b', model='c', action='e')) will return
"INSERT INTO `api_event` (`slug`, `app`, `model`, `action`, `action_type`) VALUES (a, b, c, e, )"
"""
from django.db.models import sql
not_saved_instance = model(**model_data)
not_saved_instance._for_write = True
query = sql.InsertQuery(model)
fields = [f for f in model._meta.local_concrete_fields if not isinstance(f, AutoField)]
query.insert_values(fields, [not_saved_instance], raw=False)
compiler = query.get_compiler(model.objects.db)
sql, params = compiler.as_sql()[0]
return sql % params
Tests & usage
def test_generate_update_sql_with_F(self):
qs = Event.objects.all()
update_kwargs = dict(description=F('slug'))
result = generate_update_sql(qs, update_kwargs)
sql = "UPDATE `api_event` SET `description` = `api_event`.`slug`"
self.assertEqual(sql, result)
def test_generate_create_sql(self):
result = generate_create_sql(Event, dict(slug='a', app='b', model='c', action='e'))
sql = "INSERT INTO `api_event` (`slug`, `app`, `model`, `action`, `action_type`, `description`) VALUES (a, b, c, e, , )"
self.assertEqual(sql, result)
