Validate documents during CouchDB replication process

Question

During replication, I need to validate documents that a client tries to write to my CouchDB instance. Ideally, I should reject just "invalid" documents allowing all others to pass through. Other possible outcome might be to reject the whole replication process and do not accept any documents altogether. I cannot use the validate_doc_update function because it lacks all the necessary information to make a decision.

I thought about serving all endpoints needed for replication behind the service and validate docs on an application level. For example, take all docs from POST /_bulk_docs and send back 400 error response if some docs are invalid.

Do I understand it right that such an approach stops the replication process, and the database might be left with partially replicated documents? It's because documents are uploaded in chunks during replication, and therefore there are might be couple POST /_bulk_docs calls where the first one has all valid docs and second invalid.

Is there another way, how can I discard only invalid docs?

Thanks for your help!

xpqz xpqz · Accepted Answer · 2020-09-24T08:19:08

You can apply a Cloudant Query selector via your replication document to specify which documents are valid. The canonical example is to ditch tombstones:

{
    "_id": "repl01",
    "source" : "https://.../<source>",
    "target" : "https://.../<target>",
    "selector": {
        "_deleted": {
            "$exists": false
        }
    }
}

See https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-replication-api#the-selector-field for more details.

Validate documents during CouchDB replication process

1 Answers