Why couchDB does full replication instead of filtered?

Question

I am having this problem with couchdb replication: I am using the replicator db with this setup document:

{
   "_id": "source_to_target",
   "_rev": "1-2a6510e28c2cc7caf0d58a85d705d2b8",
   "source": "http://xxxx:xxxx@localhost:5984/sourcedb",
   "target": "targetdb",
   "create_target": true,
   "continuous": true,
   "filter": "sourcedb/repl_filter",
   "query_params": {
       "someproperty": "somevalue"
   },
   "user_ctx": {
       "name": "someadmin",
       "roles": [
           "_admin"
       ]
   }
}

The problem is that the filter function gets ignored. I am sure it is correct because if I invoke:

curl -X GET "http://localhost:5984/sourcedb/_changes?filter=sourcedb/repl_filter&someproperty=somevalue&feed=continuous&style=all_docs&since=0"

the results are correctly filtered.

I perform the test as follows:

I erase the target db completely. Then I erase the above replication and recreate it from scratch. As soon as the replication document shown above gets commited, the replication starts and instead of the filtered docs, the target db gets ALL the docs, which constitutes my problem. The logs give a hint as to why:

[Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] Replication records differ. Scanning histories to find a common ancestor.
[Fri, 20 Jul 2012 17:43:38 GMT] [info] [<0.5860.17>] no common ancestry -- performing full replication

, which makes me scratch my head,because since I completely erase the target db before starting the replication, why search for common ancsestors with a not yet created db? Apparently I am missing something, but I cant figure it out. Any suggestions?

Just for kicks, try a new target DB name that you've never used before. — smathy
I know It works that way, but it doesnt solve the problem (I cant change the name every time I erase the db), and I cant tell for sure if I would call it a bug or if it is justified desired behaviour that I cant understand — Paralife
I wasn't suggesting that as a solution to your problem, simply as a troubleshooting step. So, when you said: "I erase the target DB completely" what do you mean? — smathy
@smathy I delete the target database from futon so that the replicator will recreate it as per the "create_target": true property in the replication document — Paralife

Paralife Paralife · Accepted Answer · 2012-07-21T08:52:55

Ok, here is the solution:

As smathy suggested, trying another fresh target database with a new name works. So this made me search a bit better in the logs and i found that although i was erasing the replication document in each test, somewhere among those deletions one of them failed to delete its underlying replication process which continued to be up and running. So this zombie one, made me think that the new ones didnt work. In fact they worked, filtering all the docs out, but the zombie replication kept going on doing full replication.

It was difficult to distinguish in the logs that in fact there always were 2 replications running because the zombie performed full replication and appended a lot to the logs, where the normal one didnt append almost anything because it filters out all the docs, making me thing there is just one, that ignores the filter function. I should have looked more carefully.

Anyway, since I always started the replications in futon by erasing/inserting the replication doc and I never started a replication from command-line or any other way, I assume deleting the replication document is not guaranteed to cancel the underlying replication process. I will search more on this.

Why couchDB does full replication instead of filtered?

1 Answers