What are the design patterns to share same functionality in elixir?

Question

For example, I have an app that "takes" a structure, "transforms" structure to different format and "pushes" it to some storage. I have 3 structures that go thru this pipeline with 3 transformation rules and 3 storages.

the project uses gen_stage package and it has following structure:

(book)   |producer| -> |transformer| -> |indexer|
(article)|producer| -> |transformer| -> |indexer|
(post)   |producer| -> |transformer| -> |indexer|

Each stage is a separate module, i.e. Book.Producer, Book.Transformer, Book.Indexer.

Stages on the same vertical line doing the same thing but with different entities. I.e. Book.Producer taking books from database, Article.Producer taking articles from database, etc.

The "Taking from database" piece is fairly general and can be re-used across all pipelines, i.e.

alias Experimental.GenStage

defmodule Books.Producer do
  use GenStage

  def start_link do
    GenStage.start_link(__MODULE__, [], name: __MODULE__)
  end

  def init([]) do
    {:producer, []}
  end

  def handle_demand(demand, processed) when demand > 0 do
    events = Repo.all Book
    {:noreply, events, processed ++ events}
  end
end

The transformer runs a function on an entity produced from database. The indexer pushes the transformer records into another storage.

In a naive approach with a lot of duplication, I could have 9 (3 for Book, 3 for Article, 3 for Post) modules that's doing all these steps.

What options do I have to extract bits of similar functionality and share it between modules that use it.

Perhaps I can configure it via passing params at initialization stage, or just still have 9 modules, but refactor functions into another module. What is the best practice?

This is a little hard to answer without knowing what exactly is common and what is specific to the structure of the data between the 9 functions/modules. Can you include some actual code? — Dogbert
@Dogbert I tried to be as explicit as I could, there's not much real code, but I've added example to illustrate what is "common". Thank you! — Alex Kurkin

Thiago Campezzi Thiago Campezzi · Accepted Answer · 2016-08-26T10:10:52

A possible option for the case you mentioned is to pass the schema module you're trying to fetch from (i.e. Book, Post, etc.) as an option when starting the process (from a supervision tree, for example). You would receive those options in the call to start_link, and could then pass them as the second parameter when calling GenStage.start_link.

That would cause those options to be passed as an argument to init when the GenStage is starting. At this point, you can put that module in the state (along with the processed list in a tuple, for example) so it's passed in on every handle_demand call. There, you can finally use it as an argument to the call to Repo.all.

Modules can be passed around like any other value, and sometimes you can use that to your advantage to reuse code - this is one example where it may come in handy!

You can use a similar approach for the other parts of your flow - just figure out what varies and specify that as an option when starting the process. The transform function, for example, would be a good candidate for an option on your second step. There are a few ways you can accomplish that:

pass in a module that implements a certain function - check out the documentation on Behaviours to get some guarantee that the function actually exists in the module;
pass in a captured function in an existing module, for example &BookTransformer.transform/1;
pass in an anonymous function: fn data -> transform_all_the(data) end.

Like modules, functions are first class citizens in Elixir. They can be passed around like values. This is a common code reuse pattern in functional programming languages.

What are the design patterns to share same functionality in elixir?

1 Answers