Is it possible to avoid this error with variable speed (produced during macro expansion)?

Question

I am developing a DSL and I am getting a “free term” rejection when expanding a macro. I would like to know if this can be avoided. I simplified the problem in the following situation.

Suppose we have this expression:

val list = join { 0 1 2 3 } println(list) 

where join is a macro whose implementation:

 def join(c: Ctx)(a: c.Expr[Int]): c.Expr[List[Int]] = { import c.mirror._ a.tree match { case Block(list, ret) => // c.reify(List(new c.Expr(list(0)).eval, // new c.Expr(list(1)).eval, // new c.Expr(list(2)).eval) :+ new c.Expr(ret).eval) c.reify((for (expr <- list) yield new c.Expr(expr).eval) :+ new c.Expr(ret).eval) } } 

The purpose of the macro is to combine all the elements in the argument block and return them to a single list. Since the contents of the block may be variable, I cannot use the commented reify (which works well). Uncommented - with an understanding that generates free terms - it displays a message:

"The macro distribution contains a list of variable free variables defined by the union in Macros.scala: 48: 18. Have you forgotten to use eval when splicing this variable in reifee? If you have problems tracking free variables, consider using -Xlog -free terms "

Is there a way to present for understanding (or an iterator or something else) without getting this error? By the way, I'm using 2.10-M3.

Answer 1

The problem is that your code mixes the concepts of compile time and runtime.

The variable "list" you use is a compile-time value (that is, it is assumed to be repeated at compile time), and you ask reify to save it until run time (by splicing the derived values). This cross-reference puzzle leads to the creation of the so-called free term.

In short, free terms are stubs that refer to meanings from earlier steps. For example, the following snippet:

 val x = 2 reify(x) 

It will be compiled as follows:

 val free$x1 = newFreeTerm("x", staticClass("scala.Int").asTypeConstructor, x); Ident(free$x1) 

Clever, huh? As a result, the fact that x is an identifier is preserved, it retains its type (compilation time characteristics), but, nevertheless, applies to its value (runtime characteristic). This is made possible thanks to lexical reach.

But if you try to return this tree from the macro extension (which is built into the macro invocation site), everything will explode. The macro invocation site most likely does not have x in the lexical domain, so it will not be able to refer to the value of x.

Worse yet. If the above snippet is written inside the macro, then x exists only at compile time, i.e. in the JVM that starts the compiler. But when the compiler ends, it is gone.

However, macro decomposition results containing a reference to x must be run at run time (most likely in another JVM). To understand this, you will need multi-stage perseverance, i.e. The ability to somehow serialize arbitrary compile-time values ​​and deserialize them at runtime. I do not know how to do this in a compiled language like Scala.

---

Please note that in some cases a cross stage is possible. For example, if x was a field of a static object:

 object Foo { val x = 2 } import Foo._ reify(x) 

Then it will not end as a free term, but will be confirmed in a simple way:

 Select(Ident(staticModule("Foo")), newTermName("x")) 

This is an interesting concept that was also discussed in SPJ conversations at Scala Days 2012: http://skillsmatter.com/podcast/scala/haskell-cloud .

To make sure that any expression does not contain free terms, in Haskell they add a new built-in primitive to the compiler, a constructor of the Static type. With macros, we can do this naturally using reify (which in itself is just a macro). See the discussion here: https://groups.google.com/forum/#!topic/scala-internals/-42PWNkQJNA .

---

Ok, now we saw what exactly is the problem with the source code, and how do we do it?

Unfortunately, we will have to go back to the AST manual design because reify has a hard time expressing dynamic trees. The ideal use case for macro validation is a static template with hole types known during macro compilation. Take a step to the side - and you have to resort to building trees manually.

In the end, you should go with the following (works with the recently released 2.10.0-M4, see the migration guide for the scala language to find out what exactly has changed: http://groups.google.com/group/scala-language / browse_thread / thread / bf079865ad42249c ):

 import scala.reflect.makro.Context object Macros { def join_impl(c: Context)(a: c.Expr[Int]): c.Expr[List[Int]] = { import c.universe._ import definitions._ a.tree match { case Block(list, ret) => c.Expr((list :+ ret).foldRight(Ident(NilModule): Tree)((el, acc) => Apply(Select(acc, newTermName("$colon$colon")), List(el)))) } } def join(a: Int): List[Int] = macro join_impl }