How does Datomic support correction?

Question

In Rich Hickey's talk "The Value of Values", he contends that to have true information, we should not replace old facts with new facts, but keep both with a timestamp.

For instance, if a user changes their email address, we shouldn't overwrite the old address with a new one, but instead remember "the address was [email protected] as of [timestamp], and then it was [email protected] as of [timestamp]".

This makes sense as a principle of data integrity, and I believe it's the principle upon which Datomic is built. However, if that's true, how does Datomic handle mistakes? For example, if the user mistyped their email address, we don't want to report that it used to be "[email protected]"; we want to forget that incorrect info.

How is this done?

Answer 1

To answer your question, datomic supports correction with new transactions. This is the same way that git, svn, cvs, etc support correction. It is still valuable to keep the old incorrect data because it's valuable to know that (and when!) you were incorrect.

However, you do raise an important point about time. You are "complecting" two notions of time: the time a fact is true and the time a fact was recorded. Datomic's time is the time a fact was recorded not when it was valid. In many applications these are the same thing, but sometimes (especially in finance) these are very much distinct.

Datomic's creators are rather insistent that the time the database is managing is time of record, not anything else. If you have another type of time that you need to model you can reify your fact and assert time on it or you can add the other types of time to the transaction record.

This is discussed in much greater detail on this thread (and others) about the two different kinds of time on the Datomic google group. There are some attempts illustrated on that thread to recover some of the conveniences of datomic's recorded-time functions (e.g. fast sorted access to tx logs) while using application-domain notions of time (i.e., not "time of record"). It's worth a look.

Answer 2

Currently, there's no way to do it. The closest thing is to purge/remove old data using excision. http://blog.datomic.com/2013/05/excision.html. Even with excision, there will be a record of what you deleted (e.g "values of attributes X were excised").

EDIT: There's a useful comment relating this to Git. To extend the analogy, Git does let you perform destructive changes (i.e. anytime you rebase) which creates a completely different history path. The problem is that any other person working off of that repo wouldn't be able to automatically reconcile those changes.

You can imagine your Datomic Peers to be those entities working off of your Database. If you changed the history (i.e. corrected facts), any cached Datoms inside the Transactor, Memcached and the Peers' cache referencing the affected attributes would need to be invalidated.