Wagtail 1.8.1 migration issue

Question

Trying to upgrade Wagtail from 1.7 to 1.8.1 but we're getting the following error when running migrations. We're using Postgres db.

Applying wagtailimages.0016_deprecate_rendition_filter_relation...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 204, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 495, in alter_field
    old_db_params, new_db_params, strict)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql/schema.py", line 117, in _alter_field
    new_db_params, strict,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 649, in _alter_field
    params,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: could not create unique index "wagtailimages_rendition_image_id_742f4fe4119535f1_uniq"
DETAIL:  Key (image_id, filter_id, focal_point_key)=(507, 4, 2617-1240-3472x35) is duplicated.

Here is the output from the psql command

Table "public.wagtailimages_rendition"
     Column      |          Type          |                              Modifiers
-----------------+------------------------+-------------------------- --------------------------------------------
 id              | integer                | not null default   nextval('wagtailimages_rendition_id_seq'::regclass)
 filter_id       | integer                |
 file            | character varying(100) | not null
 width           | integer                | not null
 height          | integer                | not null
 image_id        | integer                | not null
 focal_point_key | character varying(255) | not null
 filter_spec     | character varying(255) | not null
Indexes:
    "wagtailimages_rendition_pkey" PRIMARY KEY, btree (id)
    "wagtailimages_rendition_image_id_323c8fe0_uniq" UNIQUE CONSTRAINT, btree (image_id, filter_spec, focal_point_key)
    "wagtailimages_rendition_58c64917" btree (filter_spec)
    "wagtailimages_rendition_filter_id" btree (filter_id)
    "wagtailimages_rendition_filter_spec_1cba3201_like" btree (filter_spec varchar_pattern_ops)
    "wagtailimages_rendition_image_id" btree (image_id)
Foreign-key constraints:

The errors from sql import are psql:foxtail.sql:68823:

ERROR:  constraint    "wagtailcore__content_type_id_c28424df_fk_django_content_type_id" for  relation "wagtailcore_page" already exists
psql:foxtail.sql:68831: ERROR:  constraint   "wagtailcore_collection_id_5423575a_fk_wagtailcore_collection_id" for   relation "wagtailcore_groupcollectionpermission" already exists
ALTER TABLE
psql:foxtail.sql:68847: ERROR:  constraint   "wagtailcore_groupc_permission_id_1b626275_fk_auth_permission_id" for   relation "wagtailcore_groupcollectionpermission" already exists
psql:foxtail.sql:68855: ERROR:  constraint   "wagtailcore_groupcollectionp_group_id_05d61460_fk_auth_group_id" for   relation "wagtailcore_groupcollectionpermission" already exists
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
psql:foxtail.sql:68903: ERROR:  constraint  "wagtaildocs_collection_id_23881625_fk_wagtailcore_collection_id" for  relation "wagtaildocs_document" already exists
psql:foxtail.sql:68911: ERROR:  constraint  "wagtaildocs_docume_uploaded_by_user_id_17258b41_fk_auth_user_id" for   relation "wagtaildocs_document" already exists
ALTER TABLE
psql:foxtail.sql:68927: ERROR:  constraint   "wagtailimag_collection_id_c2f8af7e_fk_wagtailcore_collection_id" for    relation "wagtailimages_image" already exists

Output after fresh import but before running migrations

Table "public.wagtailimages_rendition"
Column      |          Type          |                                 Modifiers
-----------------+------------------------+--------------------------   --------------------------------------------
id              | integer                | not null default   nextval('wagtailimages_rendition_id_seq'::regclass)
filter_id       | integer                | not null
file            | character varying(100) | not null
width           | integer                | not null
height          | integer                | not null
image_id        | integer                | not null
focal_point_key | character varying(255) | not null
Indexes:
"wagtailimages_rendition_pkey" PRIMARY KEY, btree (id)
"wagtailimages_rendition_image_id_742f4fe4119535f1_uniq" UNIQUE  CONSTRAINT, btree (image_id, filter_id, focal_point_key)
"wagtailimages_rendition_filter_id" btree (filter_id)
"wagtailimages_rendition_image_id" btree (image_id)
Foreign-key constraints:
"filter_id_refs_id_6909da8c" FOREIGN KEY (filter_id) REFERENCES   wagtailimages_filter(id) DEFERRABLE INITIALLY DEFERRED
"image_id_refs_id_e221c01d" FOREIGN KEY (image_id) REFERENCES   wagtailimages_image(id) DEFERRA:

I'm slightly puzzled by that error, since the migration in question is supposed to be removing the offending unique index, not creating it... Please can you run \d wagtailimages_rendition on the Postgresql command line and paste the output here, so I can see what state the database table is currently in? — gasman
We're using docker for local dev, when I tried it again after removing the containers, the migrations ran ok. When I tried to import some sql to populate the db with some data, I get errors and all I have in my admin is the default welcome to your wagtail site page. Though it has imported users etc. It seems if there is data already there the migrations fail. — joss
It looks like your SQL import file includes some schema alteration / creation commands, which certainly won't play well with migrations because it relies on starting from one specific initial database state. I'm guessing it's intended to be run starting from an empty database - does that complete without errors? If so, what happens if you run the Wagtail 1.8.1 migrations after that? — gasman
It completes without errors import from scratch, but then the migrations error happens — joss
Please can you show the output of \d wagtailimages_rendition after the import has run, but before running the migrations? (I'm assuming the output shown above is the result of successfully running the migrations on a fresh database, right?) — gasman

gasman gasman · Accepted Answer · 2017-02-21T16:51:05

Since I can't make sense of how the situation above is arising, I can only suggest the "nuclear option": delete the contents of the wagtailimages_rendition table by running

DELETE FROM wagtailimages_rendition;

on the Postgresql command line. This should allow the remaining migrations to complete without errors. This will cause no lasting data loss, since Wagtail will regenerate the renditions as and when it needs them - although page requests will be slower for a time as it has to re-render each image where the rendition entry is missing.

UPDATE: Rather than deleting everything, a more conservative approach is to find and delete the duplicate records:

DELETE FROM wagtailimages_rendition
WHERE image_id || '-' || filter_id || '-' || focal_point_key IN (
    SELECT image_id || '-' || filter_id || '-' || focal_point_key
    FROM (
        SELECT image_id, filter_id, focal_point_key, COUNT(*) AS count
        FROM wagtailimages_rendition GROUP BY image_id, filter_id, focal_point_key
    ) AS renditions
    WHERE count > 1
);

Wagtail 1.8.1 migration issue

2 Answers