Actors mailbox overflow. Scala

Question

I am currently working with two members in scala. One, the producer, produces some data and sends it to the parser. The manufacturer sends HashMap[String,HashMap[Object,List[Int]]] via the message (along with this to mark the sender):

 parcer ! (this,data) 

The parser is constantly waiting for such messages:

 def act(){ loop{ react{ case (producer, data)=> parse(data); } } } 

The program works fine under normal conditions. The problem is associated with large amounts of data and a lot of messages sent (the hash has about 10 ^ 4 elements, the internal hash has about 100 elements, and the list has 100), the program crashes. It does not show errors or exceptions. He just stops.

The problem is that my producer works much faster than the parser (and at the moment I do not want more than one parser).

After reading scala, the mailbox size limit I wonder if my mailbox parser reaches this limit. The post also offers some solutions, but first I need to make sure this is a problem. How can I check this?

Is there a way to find out the limits of an actor’s memory? How to read used / free memory in a mailbox?

Any suggestions for the workflow that were not published in this link are also welcome.

Thanks,

Answer 1

First, you do not need to explicitly pass the sender, since the sender is tracked by the Scala side frame. You can always access the sender of the message using the sender method.

As you can see here: scala.actors.MQueue , the actor’s mailbox is implemented as a linked list and, therefore, only limited by the size of the heap.

However, if you are concerned that the manufacturer is very fast and the consumer is very slow, I suggest you study the throttling mechanism. But I would not recommend the approach from the accepted answer to the scala question of mailbox size limitation .

Trying to send overload messages when the system is heavily loaded is usually not a good idea. What if your system is too busy to check for overload? What if the overload message receiver is too busy to act on it? Also, dropping messages is not very good for me. I would think that you want all your work items to be handled reliably.

In addition, I did not rely on mailboxSize to determine the load. You cannot distinguish between different types of messages, and you can only check inside the consumer himself, and not from the manufacturer.

I suggest using an approach where the consumer asks for more work, when he knows that he can handle it.

The following is a simple example of how this can be implemented.

 import scala.actors._ import Actor._ object ConsumerProducer { def main(args: Array[String]) { val producer = new Producer(Iterator.range(0, 10000)) val consumer = new Consumer(producer) } } case class Produce(count: Int) case object Finished class Producer[T](source: Iterator[T]) extends Actor { start def act() { loopWhile(source.hasNext) { react { case Produce(n: Int) => produce(n) } } } def produce(n: Int) { println("producing " + n) var remaining = n source takeWhile(_ => remaining > 0) foreach { x => sender ! x; remaining -= 1 } if(!source.hasNext) sender ! Finished } } class Consumer(producer: Actor) extends Actor { start private var remaining = 0 def act() { requestWork() consume() } def consume(): Nothing = react { case Finished => println("Finished") case n: Int => work(n); requestWork(); consume() } def requestWork() = if(remaining < 5) { remaining += 10; producer ! Produce(10) } def work(n: Int) = { println(n + ": " + (0 until 10000).foldLeft(0) { (acc, x) => acc + x * n }) remaining -= 1 } }