When calling a scala function with compile-time macro, how to failover smoothly when it causes compilation errors?

Question

Assuming that I intend to use the singleton/literal type feature in a scala program, this feature is provided in shapeless library in scala 2.12 (scala 2.13 supports native literal type but let's use shapeless as an example)

In shapeless, literal type is represented as a path-dependent inner type of Witness object, which can be implicitly converted from a scala literal/const:


import com.tribbloids.spike.BaseSpec
import shapeless.Witness

import scala.util.Random

    val w: Witness.Lt[Int] = 3

    val w2: Witness.Lt[Int] = Random.nextInt(3) // this doesn't compile

The second line cause compilation to throw an exception:


[Error] .../WitnessSuite.scala:14: Expression scala.util.Random.nextInt(3) does not evaluate to a constant or a stable reference value
one error found

Now, assuming that I want to write something like Option[Witness.Lt[Int]] that can be converted from an Int if it is a literal or not. In scala type class convention I should write something like this:

    trait MayHaveWitness {

      type Lit
    }

    trait MayHaveWitness_Implicits0 {

      class Some(val w: Witness.Lt[Int]) extends MayHaveWitness {

        type Lit = w.T
      }
      object None extends MayHaveWitness {

        type Lit = Nothing
      }

      implicit def fromNonLit(v: Int): None.type = None
    }

    object MayHaveWitness extends MayHaveWitness_Implicits0 {

      implicit def fromLit[T](literal: T)(implicit proof: T => Witness.Lt[Int]): MayHaveWitness.Some = new Some(literal)
    }

    val v1: MayHaveWitness = 3
    println(v1.getClass)

    val v2: MayHaveWitness = Random.nextInt(3)
    println(v2.getClass)

MayHaveWitness_Implicits0 is of lower level and theoretically should be overshadowed by fromLit if the Witness implicit conversion is successful. Unfortunately when I execute this code all I got was:

class com.tribbloids.spike.shapeless_spike.WitnessSuite$MayHaveWitness_Implicits0$1$None$
class com.tribbloids.spike.shapeless_spike.WitnessSuite$MayHaveWitness_Implicits0$1$None$

The Witness implicit conversion never happens. My questions are:

why implicit proof: T => Witness.Lt[Int] is not a successful summoner of the following shapeless macro?

  implicit def apply[T](t: T): Witness.Lt[T] = macro SingletonTypeMacros.convertImpl

how do I use type classes & other scala features to implement this smooth fallback of type-level deduction? preferrably:
- NOT using macro
- If not possible, NOT using whitebox macro
- If also not impossible, NOT using macro that will be discarded by dotty

Dmytro Mitin Dmytro Mitin · Accepted Answer · 2020-06-06T17:09:24

Shapeless defines implicit instance of type Witness.Aux[T]

implicit def apply[T]: Witness.Aux[T] = macro SingletonTypeMacros.materializeImpl[T]

and implicit conversion from type T to Witness.Lt[T]

implicit def apply[T](t: T): Witness.Lt[T] = macro SingletonTypeMacros.convertImpl

Implicit instance Witness.Aux[T] is resolved or not based on type T only (whether T is a singleton type or nor) like implicit instances of ordinary type classes. But implicit conversion T => Witness.Lt[T] is not like ordinary implicit conversions. Ordinary implicit conversions are resolved or not based on type of a value to be conversed. But T => Witness.Lt[T] is resolved or not based not only on the type T but also on the value t itself (whether t is constant/stable or not).

If you switch on scalacOptions ++= Seq("-Ymacro-debug-lite", "-Xlog-implicits") you'll see that in

val w: Witness.Lt[Int] = 3 //compiles
//Warning:scalac: performing macro expansion shapeless.this.Witness.apply[Int](3) at source-/media/data/Projects/macrosdemo213/core/src/main/scala/App114_2.scala,line-9,offset=205
//Warning:scalac: _root_.shapeless.Witness.mkWitness[Int(3)](3.asInstanceOf[Int(3)])  

val w2: Witness.Lt[Int] = Random.nextInt(3) //doesn't compile
//Warning:scalac: performing macro expansion shapeless.this.Witness.apply[Int](scala.util.Random.nextInt(3)) at source-/media/data/Projects/macrosdemo213/core/src/main/scala/App114_2.scala,line-10,offset=249
//Warning:scalac: macro expansion has failed: Expression scala.util.Random.nextInt(3) does not evaluate to a constant or a stable reference value
//Error: Expression scala.util.Random.nextInt(3) does not evaluate to a constant or a stable reference value

only implicit def apply[T](t: T): Witness.Lt[T] was checked (and worked in w but didn't work in w2).

Also in

val v1: MayHaveWitness = 3 // compiles but gives None
//Warning:scalac: macro expansion is delayed: shapeless.this.Witness.apply[T]
//Warning:scalac: performing macro expansion shapeless.this.Witness.apply[T]
//Warning:scalac: macro expansion has failed: Type argument T is not a singleton type
//Information: shapeless.this.Witness.apply is not a valid implicit value for Int => shapeless.Witness.Lt[Int] because:
//hasMatchingSymbol reported error: polymorphic expression cannot be instantiated to expected type;
// found   : [T]shapeless.Witness.Aux[T]
//    (which expands to)  [T]shapeless.Witness{type T = T}
// required: Int => shapeless.Witness.Lt[Int]
//    (which expands to)  Int => shapeless.Witness{type T <: Int}
//Information: App.this.MayHaveWitness.fromLit is not a valid implicit value for Int(3) => App.MayHaveWitness because:
//No implicit view available from Int => shapeless.Witness.Lt[Int].

and in

val v2: MayHaveWitness = Random.nextInt(3) // compiles but gives None
//Warning:scalac: macro expansion is delayed: shapeless.this.Witness.apply[T]
//Warning:scalac: performing macro expansion shapeless.this.Witness.apply[T]
//Warning:scalac: macro expansion has failed: Type argument T is not a singleton type
//Warning:scalac: performing macro expansion shapeless.this.Witness.apply[T]
//Information: App.this.MayHaveWitness.fromLit is not a valid implicit value for Int => App.MayHaveWitness because:
//No implicit view available from Int => shapeless.Witness.Lt[Int].
//Information: shapeless.this.Witness.apply is not a valid implicit value for Int => shapeless.Witness.Lt[Int] because:
//hasMatchingSymbol reported error: polymorphic expression cannot be instantiated to expected type;
// found   : [T]shapeless.Witness.Aux[T]
//    (which expands to)  [T]shapeless.Witness{type T = T}
// required: Int => shapeless.Witness.Lt[Int]
//    (which expands to)  Int => shapeless.Witness{type T <: Int}
//Information: App.this.MayHaveWitness.fromLit is not a valid implicit value for Int => App.MayHaveWitness because:
//No implicit view available from Int => shapeless.Witness.Lt[Int].

both implicit def apply[T]: Witness.Aux[T] and implicit def apply[T](t: T): Witness.Lt[T] were checked and none of them worked.

why implicit proof: T => Witness.Lt[Int] is not a successful summoner of the following shapeless macro?

Compiler treats implicits of functional types A => B differently than implicits of other types. It can treat them as implicit conversions (views). But whether it actually treats them as conversions or just implicit instances of type A => B (like other types) depends on boolean flag isView.

When you do

val w: Witness.Lt[Int] = 3 //compiles
val w2: Witness.Lt[Int] = Random.nextInt(3) //doesn't compile
val v1: MayHaveWitness = 3 //compiles
val v2: MayHaveWitness = Random.nextInt(3) //compiles

isView is true. But when you do

implicitly[Int => Witness.Lt[Int]] //doesn't compile
implicitly[3 => Witness.Lt[Int]] //doesn't compile
implicitly[Int => MayHaveWitness] //doesn't compile
implicitly[3 => MayHaveWitness] //doesn't compile

or here

implicit def fromLit... (implicit proof: T => Witness.Lt[Int]) ...
                        ______________________________________

isView is false.

In simple cases existence of implicit A => B and implicit conversion from A to B are the same

class A
class B
// implicit val aToB: A => B = null // this one
implicit def aToB(a: A): B = null   // or this one
implicitly[A => B] //compiles
val b: B = new A //compiles

but not in our case. There is implicit conversion 3 => Witness.Lt[3] but not an instance of this type

val w: Witness.Lt[3] = 3.asInstanceOf[3] //compiles

implicitly[3 => Witness.Lt[3]] // doesn't compile
//Information: shapeless.this.Witness.apply is not a valid implicit value for 3 => shapeless.Witness.Lt[3] because:
//hasMatchingSymbol reported error: polymorphic expression cannot be instantiated to expected type;
// found   : [T]shapeless.Witness.Aux[T]
//    (which expands to)  [T]shapeless.Witness{type T = T}
// required: 3 => shapeless.Witness.Lt[3]
//    (which expands to)  3 => shapeless.Witness{type T <: 3}
//Error: No implicit view available from 3 => shapeless.Witness.Lt[3].

so it checks implicit def apply[T]: Witness.Aux[T] but not implicit def apply[T](t: T): Witness.Lt[T]. I didn't debug implicit resolution deeply but I suspect that some type is not inferred before implicit is resolved.

There is no standard way to switch on isView in order to completely emulate behavior of implicit conversion while resolving proof in ... def fromLit... (implicit proof: T => Witness.Lt[Int]) .... We can switch on isView with macros if we use c.inferImplicitView rather than c.inferImplicitValue

import scala.language.experimental.macros
import scala.reflect.macros.whitebox

trait ImplicitView[A, B] {
  def instance: A => B
}
object ImplicitView {
  implicit def mkImplicitView[A, B]: ImplicitView[A, B] = macro mkImplicitViewImpl[A, B]
  def mkImplicitViewImpl[A: c.WeakTypeTag, B: c.WeakTypeTag](c: whitebox.Context): c.Tree = {
    import c.universe._

    val tpA = weakTypeOf[A]
    val tpB = weakTypeOf[B]

    val x = TermName(c.freshName("x"))
    val conversion = c.inferImplicitView(tree = EmptyTree, from = tpA, to = tpB, silent = false)

    q"""new ImplicitView[$tpA, $tpB] {
      def instance: $tpA => $tpB = ($x: $tpA) => $conversion($x)
    }"""
  }

Let's replace

implicit def fromLit[T](literal: T)(implicit proof: T => Witness.Lt[Int]): MayHaveWitness.Some = new Some(literal)

with

implicit def fromLit[T](literal: T)(implicit proof: ImplicitView[T, Witness.Lt[Int]]): MayHaveWitness.Some = new Some(proof.instance(literal))

Also we have to modify

implicit def fromNonLit(v: Int): None.type = None

because it's ambiguous with fromLit. Reasons are similar to those. The easiest fix is to replace it with

implicit def fromNonLit[T](v: T): None.type = None

Now both

val v1: MayHaveWitness = 3
println(v1.getClass)

val v2: MayHaveWitness = Random.nextInt(3)
println(v2.getClass)

give Some (I suspect that's not what you wanted). That's understandable. Random.nextInt(3) is Int. And we were resolving MayHaveWitness based only on types. And there is implicit conversion Int => Witness.Lt[Int]. So it's Some.

So it seems if we want v1 to give Some and v2 to give None then we can't do that based only on types. So approach with type classes will not work and we'll have to use macros.

trait MayHaveWitness {
  type Lit
}

object MayHaveWitness  {
  class Some(val w: Witness.Lt[Int]) extends MayHaveWitness {
    type Lit = w.T
  }
  object None extends MayHaveWitness {
    type Lit = Nothing
  }

  implicit def fromLit[T](literal: T): MayHaveWitness = macro fromLitImpl[T]
  def fromLitImpl[T: c.WeakTypeTag](c: whitebox.Context)(literal: c.Tree): c.Tree = {
    import c.universe._
    val conversion = c.inferImplicitView(tree = literal, from = weakTypeOf[T], to = typeOf[Witness.Lt[Int]], silent = false)
    util.Try(c.typecheck(q"new MayHaveWitness.Some($conversion($literal))"))
      .getOrElse(q"MayHaveWitness.None")
  }
}

Here we replaced (implicit proof: T => Witness.Lt[Int]) with c.inferImplicitView... and we explored not only type of literal but also literal itself.

Now in

val v1: MayHaveWitness = 3
println(v1.getClass)

val v2: MayHaveWitness = Random.nextInt(3)
println(v2.getClass)

v1 gves Some and v2 gives None.

If you make fromLit blackbox it will still work but will return MayHaveWitness instead of MayHaveWitness.Some and MayHaveWitness.None.

When calling a scala function with compile-time macro, how to failover smoothly when it causes compilation errors?

1 Answers