Akka: "Trying to deserialize a serialized ActorRef without an ActorSystem error in the scope"

Question

I integrate Akka and Spark usage as follows: when a task is distributed among Spark nodes when processing these tasks, each node also periodically sends indicator data to another data collection process, which is located somewhere else on the network using Akka (connecting to the remote process via akka-remote).

The actor’s send / receive metrics functionality works very well when used offline, but when it is integrated into the Spark task, the following error occurs:

java.lang.IllegalStateException: Trying to deserialize a serialized ActorRef without an ActorSystem in scope. Use 'akka.serialization.Serialization.currentSystem.withValue(system) { ... }' at akka.actor.SerializedActorRef.readResolve(ActorRef.scala:407) ~[akka-actor_2.10-2.3.11.jar:na] 

If I understood correctly, the source of the problem is the Spark node, which is unable to deserialize ActorRef, because it does not have the complete information necessary for this. I understand that installing ActorSystem in the scope will fix it, but I'm not sure how to use the suggested akka.serialization.Serialization.currentSystem.withValue(system) { ... }

Akka white papers are very good with regard to all the topics that they touch. Unfortunately, the chapter on Serialization can be improved IMHO.

Note: there is a similar SO question here, but the decision made is too specific and therefore not very useful in the general case

Answer 1

An ActorSystem is responsible for all functions associated with ActorRef objects.

When you program something like

 actorRef ! message 

In fact, you call a bunch of work on the ActorSystem, not the ActorRef, to put the message in the correct mailbox, raise the actor to start the method of receiving in the thread pool, etc. From the documentation :

The actor system manages the resources that it is configured to use in order to run the actors that it contains. There may be millions of actors in one such system, after all the mantras will consider them as they are weighed with an overhead of only about 300 bytes per copy. Naturally, the exact order in which messages are processed on large systems is not controlled by the application.

That's why your code works fine "standalone", but not in Spark. Each of your Spark nodes lacks an ActorSystem mechanism, so even if you can de-serialize ActorRef into a node, there will be no ActorSystem to handle ! in your node function.

You can set the ActorSystem in each node and use (i) remoting to send messages to your ActorRef in the "master" ActorSystem via the actorSelection or (ii) the serialization method that you mentioned, where each node of the ActorSystem will be the system in the example that you are quoted .