How to sync postgres database with neo4j in realtime while using Django-admin?

Question

So currently I am working on a Django project where I have two different databases. one is PostgreSQL and the second is Neo4j. what I want is real-time sync between both databases. I am using the Django-admin panel for crud operation in the Postgres database. now I want every crud operation update in the neo4j database also. but I don't know how to do it.

Steven Montalbano Steven Montalbano · Accepted Answer · 2021-06-09T20:54:03

There are a few routes you could go with this. I think you are going to want to leave Postgres as your main Django database and use Neo4j as an accessory DB, not managed by the Django framework. You can do this by using the Neo4j client library similar to this post for interfacing with the graph DB yourself.

To keep Neo4j in sync with the Postgres managed by Django, I think signals are going to be your best friend, specifically the post_save signal. Essentially, anytime that you save a Django model, this signal will fire a function that you can write to use the Neo4j client API to reflect the changes in the graph database.

However, depending on how complex your models are, this is where it can get tricky determining which fields were changed to only update those. I had a similar issue a while back and used this custom Mixin (I cant find the original author but kept this in my notes):

from django.forms import model_to_dict
import logging

logger = logging.getLogger(__name__)


class ModelDiffMixin(object):
    """
    A model mixin that tracks model fields' values and provide some useful api
    to know what fields have been changed.
    """

    def __init__(self, *args, **kwargs):
        super(ModelDiffMixin, self).__init__(*args, **kwargs)
        self.__initial = self._dict

    @property
    def diff(self):
        d1 = self.__initial
        d2 = self._dict
        diffs = [(k, (v, d2[k])) for k, v in d1.items() if v != d2[k]]
        return dict(diffs)

    @property
    def has_changed(self):
        return bool(self.diff)

    @property
    def changed_fields(self):
        return self.diff.keys()

    def get_field_diff(self, field_name):
        """
        Returns a diff for field if it's changed and None otherwise.
        """
        return self.diff.get(field_name, None)

    def save(self, *args, **kwargs):
        """
        Saves model and set initial state.
        """
        super(ModelDiffMixin, self).save(*args, **kwargs)
        self.__initial = self._dict

    @property
    def _dict(self):
        return model_to_dict(self, fields=[field.name for field in
                             self._meta.fields])

If you implement this Mixin to your models you with to sync with Neo4j, then inside the post_save signal handler you can call model.diff or model.changed_fields to see which attributes need to be updated on the Neo4j node instead of deleting the entire Node and recreating it and all relationships.

How to sync postgres database with neo4j in realtime while using Django-admin?

1 Answers