Clojure: future, agent or core.async for IO

Question

Say that there are 4 components:

1. interactive data collection via http (survey)
2. atom, which accumulates survey stages
3. cpu-heavy computation
4. database writer

I try to achieve last two operations be asyncronous, like placing accumulated data somewhere in a queue and start collecting other survey, while something processes data and something else does IO with any of previously prepared data.

The quesion is what feature of language to use? In Clojure Programming book there are examples of using agents as components to perform IO operations, but doesn't futures offer same "fire and forget" facilities? Or agent is still an identity like atom or ref and not an actor at all?

Answer 1

An agent is a container for mutable state, just like atoms and refs. You send an action to an agent, which is "fire and forget," with the guarantee that each action sent to that agent will be executed only once (unlike functions swap!ped on an atom, and alter!s for refs, which can be retried multiple times potentially). Multiple actions sent to the agent will be completed serially, not concurrently.

Thus, I/O can safely be performed within an action sent to an agent. So, an agent may be a good candidate for your step #4, as you want each database write to occur only once (never retried).

You would have a single agent, representing your database state. In each future/thread you create in step #3, after you perform your computation, you send the action to the lone agent, which will serialize the actions, even if they arrive at the same time from your threads.