Shapeless not working in generic context

Question

I am still trying to get my head around Shapeless (and, to a lesser extent, Scala!) and I have been writing some simple code to generate random instance data for case classes - predominantly based on the guides here: http://enear.github.io/2016/09/27/bits-of-shapeless-2/ (the example covers a JSON Writer implementation)

I have created a Generator[A] trait and created implicit implementations for simple types, and as per the example in the above link, I have also created implicit implementations to handle HList, HNil, Coproduct and CNil:

  implicit def hnilGenerator = new Generator[HNil] {
    override def generate(a: HNil) = HNil
  }

  implicit def hconsGenerator[H, T <: HList](implicit headGen: Generator[H], tailGen: Generator[T]) =
    new Generator[H :: T] {
      override def generate(a: H :: T) = headGen.generate(a.head) :: tailGen.generate(a.tail)
    }

  implicit def cnilGenerator: Generator[CNil] =
    new Generator[CNil] {
      override def generate(a: CNil): CNil = throw new RuntimeException("Invalid candidate configuration")
    }

  implicit def cconsGenerator[H, T <: Coproduct] =
    new Generator[H :+: T] {
      override def generate(a: H :+: T) = throw new RuntimeException("Invalid candidate configuration")
    }

I can now use this code to generate a random instance based on a case class or a sealed trait:

    it("should work with a case class to hlist") {
      case class Test(x: IntGene, y: DoubleGene, z: BooleanGene)
      val c = Generic[Test].to(Test(IntGene(), DoubleGene(), BooleanGene()))
      generate(c)
    }
    it("what happens with sealed traits") {
      sealed trait Shape
      case class Square(width: Int, height: Int) extends Shape
      case class Circle(radius: Int) extends Shape

      val c = Generic[Shape].to(Circle(1))
      generate(c)
    }

Both of the above work no problem, however, if I try to make this a generic (as in parameter types) I get compilation errors not being able to find the necessary implicts:

it("should handle generics") {
  case class GenericTest[A: Generic](x: A) {
    def convert() = {
      val c = Generic[A].to(x)
      generate(c)
    }
  }
}

So from my understanding, because I have used the Generic context bound A, the compiler knows that is going to be available, so c must be some possible return from the call to(x) - Am I missing something in the implementation to handle that return type from the Generic shapeless call? Or have I wildly misunderstood something?

I am hoping this is possible and I have just missed something - is it that the compiler doesn't know what will be passed in (Im assuming not), or is there another possible type that needs to be handled implicitly from that to(x) call?

---

EDIT

Compile error added below - I'm really just trying to understand: Is it that there is some return case from the to(x) shapeless call that I have not catered for, or is it because the compiler doesn't have any idea what will be passed in and there are some types not catered for (e.g. I haven't added a Date generator implicit - and a case class could potentially have any type included? I was hoping that was not the case, and as the compiler knows nothing is actually being passed to the class/method it knows there are no issues?)

GeneratorSpec.scala:44: could not find implicit value for parameter gen: io.github.robhinds.genotype.Generator[GenericTest.this.evidence$1.Repr]
          generate(c)

And my generate method is just a simple helper method that gets given the implicit Generator :

def generate[A](a: A)(implicit gen: Generator[A]) = gen.generate(a)

Answer 1

Your problem comes from the fact that Generic only converts a case class to a HList and a sealed trait to a Coproduct (not recursively).

So if you have a generic Generic, you have no information on what HList or Coproduct you are given, so, you cannot use your product and coproduct rules to find the wanted implicit. In some explicit case, you might have the same problem, so I will give you this as an example:

Let's say you have a case class architecture

case class Bottom(value: String)
case class Top(bot: Bottom, count: Int)

The implicit Generic[Top] will have type MyHList = Bottom :: Int :: HNil as output type, so you'll be asking for an implicit Generator[MyHList]. But since you don't have an implicit Generator[Bottom] in scope, you won't be able to use hconsgenerator.

In the generic case, it's even worse. The compiler can only infer HList for the output type of Generic[A] (and that's assuming you forget about Coproduct), so you need an implicit Generator[HList], which you cannot provide.

The solution is to give an implicit for constructs that have a generic that can itself be generated:

implicit def generic2generate[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Generator[L]): Generator[T] = new Generator[T] {
  def generate(c: T) = generic.from(lGen.generate(generic.to(c)))
}

EDIT

You can now follow the implicit resolution for our Top type:

• We can have a Generator[Top] using last rule, if we have a Generic.Aux[Top, L] for some L, and a Generator[L].

• The only Generic.Aux[Top, _] that exists implicitly is a Generic.Aux[Top, Bottom :: Int :: HNil], so we are reduced to finding a Generator[Top, Bottom :: Int :: HNil]

• using the hcons rule three times, we are reduced to finding a Generator[Bottom], a Generator[Int] and a Generator[HNil].

• Generator[Int] is given (I assume) and Generator[HNil] is the first rule, so we are reduced to finding a Generator[Bottom]

• the only rule that can provide one, is once again the 3rd rule, so we must find a Generator[String :: HNil], since the only Generic available is a Generic.Aux[Bottom, String :: HNil].

• using the hcons rule, we are down to finding a Generator[String], which can easily be provided.

This example shows different points:

• first that it may take a long time when compiling to solve all these implicits (I only gave the main points of the proof, but the compiler has to try all possible branches)

• second, that this resolution can only be done for a specific Generic, it cannot be inferred generically (although this might seem counter-intuitive); even if the human mind is able to tell that it will work for every Generic, the compiler cannot process it as such.

Answer 2

I think you are missing the Generator type bound.

it("should handle generics") {
  case class GenericTest[A: Generic : Generator](x: A) {
    def convert() = {
      val c = Generic[A].to(x)
      generate(c)
    }
  }
}