cqrs without event sourcing and LMAX style single thread design review

Question

Just a review of what I'm planning and to see if it can work.

Im using a RDBMS and planning to use CQRS without event sourcing. I think event sourcing is a bit advanced for a first attempt and I'm forced to use an RDBMS.

The Task based UI is made up of many commands most of which do not need a response.

Architecture is basically

            /---- RDBMS ( EF ) 
     IO  ops                 \
            |                 \
  Single threaded Domains   Fascade(DTO) Queries
            |                /
  Event/Command Dispatcher  /
             \             /
                MVC client 

Single threaded domains don't talk to each other and they talk to the outside world via disruptors (basically ring buffers).

The Command dispatcher copies external events and commands to disk and reloads them in case of crash. Completion is explicitly marked (by IO Ops).

Commands will basically be persisted (with a transactional scope of the command), the IO Ops layer will grab all the events and process them and marks the command completion in 1 transaction. (note the command is persisted via a journal services not the domain but it talks to IO opps which matches it up with work for the command). If the command fails and its marked persist ( not all will be ) then it can be replayed. (It only persists the command when it has the command and has received a DoTransation Message.)

The Command dispatcher connects to the domains via a disruptor.

Questions

1. Should I load the entire domain into memory (about 300 - 500 Meg) and run of that?Obviously domain would only update after the DB is updated.

2. Is it OK to mix external events back into the command dispatcher (so it gets picked up by the single thread and processed) . eg the event becomes a command.

3. It looks simple to code the domains and user code (and I get a nice rich domain), at least from a prototype I'm working on. Is it?

4. When the domains do IO they send a message to a disruptor and can either

   • specify the command guid and the command gets matched up with a transaction
   • assume completion ( shoot and pray)
   • provide a call back command which is passed to the command dispatcher and then reappears in the domain. After an IO message is created the system can continue on the current command or complete it and receive the next command.
     Is this good enough?

5. The system runs in one process and shared memory, but the domains resources are only accessed by themselves and 1 thread . Is this ok?

It's a bit of an experimental mish mesh. Any thoughts?

Answer 1

First of all, this architecture is quite complicated. Double-check that the projected longevity of the project will be worth all the initial investment in architecture. Bottom line: will this project still be paying the bills in 5 or 10 years? Will the customer allow you to spend months on architecture without providing business value?

Problems

You didn't mention what hosts your command dispatcher. Whatever this piece is, it will most likely be the bottleneck in your application. Unless it's pulling messages off of a very high performance queue (e.g. ZeroMQ), I don't think you are going to need a Ring Buffer. A Queue will do just fine in most scenarios, and it is a lot simpler.

Your Questions

I'm assuming IO Ops means events / event handlers. There are probably other nuances I didn't catch.

1. I doubt this really makes sense for a web application. Loading the domain in memory is beneficial if your database is a performance bottleneck. Depending on the performance profile of your application, it might not be, and the effort towards managing all those threads could be a waste of developer time. (Mainly because you have to ensure that you stop the threads when your app shuts down, or the app will never shut down.) The part about updating the domain after the DB is updated doesn't make sense to me. You only want to load the domain on startup. Saving the domain's state change is only so it can be reloaded again on the next startup. I'd think you'd need to change the domain's state before you could persist it. Also, if the DB were that much of a bottleneck, I would probably save state asynchronously.

2. I think you are missing some steps here. Where you might want to take an event and send it as a command sounds like the place where a Process Manager (or Saga if you prefer) should go. Rarely are business processes as simple as always converting an event into a command. E.g. what happens when the subsequent command can't complete? Or when more than one event needs to happen before the command can be issued. (e.g. OrderPlaced and PaymentReceived must happen before ShipOrder is sent)

3. The hard part of domains is usually figuring out the most appropriate way to model them. If your domain model doesn't fit the actual domain, then the code becomes more awkward and complicated. Aside from that, it really depends on your domain. If your programming a calculus solver, you're probably in for it. But often business logic, once understood, can be codified reasonably well.

4. I'm really not sure what you are trying to do with the transaction bit. Are you trying to wrap multiple commands into a transaction? That probably represents where the domain could be modeled better or you are doing batching. Since hopefully not that many commands are batched, seems a waste to always use one. If all your operations are batched and each operation in the batch is just setting individual fields, then you are really missing the point of using DDD and task-based UI.

5. See also #1. Provided this level of performance is needed, this should be fine. Your domain will most likely be hosted in memory as singletons on the MVC application that receives the commands. I assume you are using proper locking techniques when accessing shared memory. Note that this site is a great resource for such topics. Particularly, don't be fooled by the volatile keyword!

Just remember everything is a trade-off. Don't waste a lot of time developing architecture that doesn't really benefit your app.