PouchDB delete document upon successful replication to CouchDB

Question

I am trying to implement a process as described below:

Create a sale_transaction document in the device.
Put the sale_transaction document in Pouch.
Since there's a live replication between Pouch & Couch, let the sale_transaction document flow to Couch.
Upon successful replication of the sale_transaction document to Couch, delete the document in Pouch.
Don't let the deleted sale_transaction document in Pouch, to flow through Couch.

Currently, I have implemented a two-way sync from both databases, where I'm filtering each document that is coming from Couch to Pouch, and vice versa.

For the replication from Couch to Pouch, I didn't want to let sale_transaction documents to go through, since I could just get these documents from Couch.

PouchDb.replicate(remoteDb, localDb, {
   // Replicate from Couch to Pouch
   live: true,
   retry: true,
   filter: (doc) => {
      return doc.doc_type!=="sale_transaction";
   }
})

While for the replication from Pouch to Couch, I put in a filter not to let deleted sale_transaction documents to go through.

PouchDb.replicate(localDb, remoteDb, {
   // Replicate from Pouch to Couch
   live: true,
   retry: true,
   filter: (doc) => {
      if(doc.doc_type==="sale_transaction" && doc._deleted) {
         // These are deleted transactions which I dont want to replicate to Couch
         return false;
      }
      return true;
    }
}).on("change", (change) => {
   // Handle change
   replicateOutChangeHandler(change)
});

I also implemented a change handler to delete the sale_transaction documents in Pouch, after being written to Couch.

function replicateOutChangeHandler(change) {
   for(let doc of change.docs) {
      if(doc.doc_type==="sale_transaction" && !doc._deleted) {
         localDb.upsert(doc._id, function(prevDoc) {
            if(!prevDoc._deleted) {
               prevDoc._deleted = true;
            }
            return prevDoc;
         }).then((res)=>{
            console.log("Deleted Document After Replication",res);
         }).catch((err)=>{
            console.error("Deleted Document After Replication (ERROR): ",err);
         })
      }
   }
}

The flow of the data seems to be working at first, but when I get the sale_transaction document from Couch, then do some editing, I would then have to repeat the process of writing the document in Pouch, then let it flow to Couch, then delete it in Pouch. But, after some editing with the same document, the document in Couch, has also been deleted.

I am fairly new with Pouch & Couch, specifically in NoSQL, and was wondering if I'm doing something wrong in the process.

Why don't you simply edit/create the document in memory and them push them directly to CouchDB? — Alexis Côté
@AlexisCôté I was thinking of the instance where the device doesn't have an access to the internet. So if I push the document to Pouch, and there's no internet connection, I'm relying on Pouch's native replication to sync the document to Couch. — aamiel16

StickByAtlas StickByAtlas · Accepted Answer · 2017-04-27T19:22:10

For a situation like the one you've described above, I'd suggest tweaking your approach as follows:

Create a PouchDB database as a replication target from CouchDB, but treat this database as a read-only mirror of the CouchDB database, applying whatever transforms you need in order to strip certain document types from the local store. For the sake of this example, let's call this database mirror. The mirror database only gets updated one-way, from the canonical CouchDB database via transform replication.

Create a separate PouchDB database to store all your sales transactions. For the sake our this example, let's call this database user-data.

When the user creates a new sale transaction, this document is written to user-data. Listen for changes on user-data, and when a document is created, use the change handler to create and write the document directly to CouchDB.

At this point, CouchDB is recieving sales transactions from user-data, but your transform replication is preventing them from polluting mirror. You could leave it at that, in which case user-data will have local copies of all sales transactions. On logout, you can just delete the user-data database. Alternatively, you could add some more complex logic in the change handler to delete the document once CouchDB has recieved it.

If you really wanted to get fancy, you could do something even more elaborate. Leave the sales transactions in user-data after it's written to CouchDB, and in your transform replication from CouchDB to mirror, look for these newly-created sales transactions documents. Instead of removing them, just strip them of anything but their _id and _rev fields, and use these as 'receipts'. When one of these IDs match an ID in user-data, that document can be safely deleted.

Whichever method you choose, I suggest you think about your local PouchDB's _changes feed as a worker queue, instead of putting all of this elaborate logic in replication filters. The methods above should all survive offline cases without introducing conflicts, and recover nicely when connectivity is restored. I'd recommend the last solution, though it might be a bit more work than the others. Hope this helps.

PouchDB delete document upon successful replication to CouchDB

2 Answers