Regenerating CouchDB views

3
votes

Contrived example:

{
  productName: 'Lost Series 67 DVD',
  availableFrom: '19/May/2011',
  availableTo: '19/Sep/2011'
}

View storeFront/currentlyAvailableProducts basically checks if current datetime is within availableFrom - availableTo and emits the doc.

I would like to force a view to regenerate at 1am every night, i.e. process/map all docs.

At first I had a simple python script scheduled via crontab that touched each document hence causing a new revision and the view to update,however since couchdb is append only this wasnt very efficient - i.e. loads of unnecessary IO and disk space usage followed by compaction, very resource wasteful on all fronts.

Second solution was to push the view definition again via couchapp push however this meant the view was unavailable (or partially unavailable) for several minutes which was also unacceptable.

Is there any other solutions?

2
nosqlcouchdb
Why would you need to do this? The entire point behind the incremental updates is to prevent this sort of thing... - Dominic Barnes
Because the result of view will be different based on the current day - user53791

2 Answers

7
votes

Will's answer is great; but just to get the consensus viewpoint represented here:

Keep one view, and query it differently every day

Determine your time-slice size, for example one day.

Next, for each document, you emit once for every time slice (day) that it is available. So if a document is available from 19 May to 21 May (inclusive), your emit keys would be:

"2011-05-19"
"2011-05-20"
"2011-05-21"

Once that is computed for every document, to find docs available on a certain day, just query the view with (e.g. today) ?key="2011-05-18".

You never have to update or re-run your views.

If you must never change your query URL for some reason, you might be able to use a _show function to 302 (temporary) redirect to today's correct query.

2
votes

So your view is not being updated automatically I take it?

New and changed documents are not being added on the fly?

Oh I see, you're cheating. You're using "out of document" information (i.e. the current date) during view creation.

There's no view renaming, but if you were desperate you could use url rewriting.

Simply create a design document "each day": /db/_design/today05172011

Then use some url rewriting to change: GET /db/_design/today/_view/yourview

to: GET /db/_design/today051711/_view/yourview

Create the view at 11pm server time (tweak it so that "now" is "tomorrow", or whatever).

Then add some more clean up code to later delete the older views.

This way your view builds each night as you like.

Obviously you'll need to front Couch with some other web server/proxy to pull this off.

It's elegant, and inelegant, at the same time.