Can I use Scala REPL for benchmarks?

Question

Scala REPL is a great platform for interactively testing specific pieces of code. I recently made some performance comparisons using REPL for multiple operations and benchmarking wall clocks.

Here is an example I recently created to help answer the question SO [1] [2]:

// Figure out the perfomance difference between direct method invocation and reflection-based method.invoke

def invoke1[T,U](obj:Any, method:Method)(param:T):U = method.invoke(obj,Seq(param.asInstanceOf[java.lang.Object]):_*) match { 
    case x: java.lang.Object if x==null => null.asInstanceOf[U]
    case x => x.asInstanceOf[U]
}

def time[T](b: => T):(T, Long) = {
    val t0 = System.nanoTime()
    val res = b
    val t = System.nanoTime() - t0
    (res,t )
}

class Test {
  def op(l:Long): Long = (2 until math.sqrt(l).toInt).filter(x=>l%x==0).sum
}

val t0 = new Test

val method = classOf[Test].getMethods.find(_.getName=="op").get

def timeDiff = {
  val (timeDirectCall,res) = time { (0 to 1000000).map(x=>t0.op(x)) }
  val (timeInvoke, res2) = time { (0 to 1000000).map(x=>{val res:Long=invoke1(t0,method)(x);res}) }
  (timeInvoke-timeDirectCall).toDouble/timeDirectCall.toDouble
}


//scala> timeDiff
//res60: Double = 2.1428745665357445
//scala> timeDiff
//res61: Double = 2.1604176409796683

In another case, I generated MM random data points to compare concurrency models for an open source project. REPL did a great job with various configurations without a code compilation cycle.

I know about common test errors, such as JIT optimization and the need for warm-ups.

My questions:

• - REPL, -?

• ? : A , B?

• jit ?

• , ?

[1] Scala :

[2] https://gist.github.com/maasg/6808879

Answer 1

. , - .

, , , REPL scala -lang.org faq . .

: , REPL , .

, : REPL , . " ", .

REPL -. , . , , , JVM, ; . REPL.

, . .

, , .

, , REPL . Microbenchmarking - , . .

. , REPL, ( ), , ( inlining by scalac).

:

-Yrepl-class-based , .

, .