What is the correct way to read dependency tree?

Question

I want to parse app deps tree and generate a map.
I'm currently reading top-level deps like:

top_level_deps = Mix.Dep.loaded([]) |> Enum.filter(& &1.top_level)

I use Enum.reduce in recursive method for top_level_deps and then for dep.deps.
Everything is good until 4th level where is no deps ...
For example:
App A require B that requires C that requires D - I see empty list when inspect C deps (also no hex packages here).
When I modify app A or B to require D and C not require D then I see all deps in my map.
What is the correct way to read dependency tree?

Steps to reproduce:
1 Create a tmp dir for apps
2 Go to tmp dir and create 4 apps like:

mix new a
mix new b
mix new c
mix new d

3 Add deps for a, b and c apps like:

defp deps do
    [{:b, path: "../b"}] # deps for A app
end

4 Add this task to lib/mix/tasks/ to A project:

defmodule Mix.Tasks.Reproduce do
    use Mix.Task
    def run(_) do
        app_atom = Mix.Project.config[:app]
        top_level_deps = Mix.Dep.loaded([]) |> Enum.filter(& &1.top_level)
        result = reproduce top_level_deps, app_atom
        IO.inspect Map.put_new result, app_atom, Atom.to_string(app_atom)
    end

    defp reproduce deps, prefix, result \\ Map.new do
        Enum.reduce deps, result, fn(dep, result) ->
            if dep.scm != Hex.SCM do # filter Hex packages here
                new_prefix = "#{prefix}_#{dep.app}"
                new_result = reproduce dep.deps, new_prefix, result
                if dep.app == :c do
                    IO.puts "No deps here !!!"
                    IO.inspect dep.deps
                end
                Map.put_new new_result, dep.app, new_prefix
            else
                result
            end
        end
    end
end

5 Run mix reproduce
Current results:

%{a: "a", b: "a_b", c: "a_b_c"}

Expected results:

%{a: "a", b: "a_b", c: "a_b_c", d: "a_b_c_d"}

Not sure why the returned list does not include nested dependencies, but I was trying out a different approach to get your expected result, and have a question: what output do you expected if b also depends on d (if the dependencies are a -> b, b -> c, b -> d, and c -> d). — Dogbert
@Dogbert: I will catch a collision and do one of operations: first result, last result(override previous) or fallback like: "a_multi_d" - depends of configuration. — Eiji
I need to add a disclaimer that Mix.Dep is not a public API. You are not supposed to be calling it and there is no guarantee the module nor the function will exist in future Elixir versions. :) — José Valim

Dogbert Dogbert · Accepted Answer · 2016-09-01T15:00:17

I do not know why Mix.Dep.loaded([]) does not contain nested deps after a certain level, but since all recursive dependencies are present directly in that list, we can build our own lookup Map and use that. Here's an implementation that returns your expected output:

defmodule Mix.Tasks.Deps.Map do
  use Mix.Task

  def run(_) do
    app = Mix.Project.config[:app]
    deps = for %{app: app, deps: deps} <- Mix.Dep.loaded([]), into: %{} do
      {app, deps}
    end |> Map.put(app, Enum.filter(Mix.Dep.loaded([]), &(&1.top_level)))
    recur(deps, app, "") |> Map.put(app, "#{app}") |> IO.inspect
  end

  def recur(deps, app, prefix, result \\ Map.new) do
    Enum.reduce(deps[app], result, fn(dep, result) ->
      if dep.scm != Hex.SCM do
        recur(deps, dep.app, "#{prefix}#{app}_", result)
        |> Map.put_new(dep.app, "#{prefix}#{app}_#{dep.app}")
      else
        result
      end
    end)
  end
end

Output with 4 packages, a, b, c, and d:

%{a: "a", b: "a_b", c: "a_b_c", d: "a_b_c_d"}

Output with the above plus a new package e which d depends on:

%{a: "a", b: "a_b", c: "a_b_c", d: "a_b_c_d", e: "a_b_c_d_e"}

What is the correct way to read dependency tree?

1 Answers