Geek Answers Handbook

Apache Spark - connection refused to employee

Hi, I was new to apache sparks and I was trying to recognize it.

When creating a new autonomous cluster, I met this error.

I started my wizard and it is active in port 7077, I see that in ui (port 8080)

When starting the server using the command

 ./bin/spark-class org.apache.spark.deploy.worker.Worker spark://192.168.0.56:7077

I encounter a communication failure error

14/07/22 13:18:30 ERROR EndpointWriter: AssociationError [akka.tcp://sparkWorker@node-        physical:55124] -> [akka.tcp://sparkMaster@192.168.0.56:7077]: Error [Association failed     with [akka.tcp://sparkMaster@192.168.0.56:7077]] [
akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkMaster@192.168.0.56:7077]
Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2:     Connection refused: /192.168.0.56:7077

Please help me with the error that I have broken here for a long time.

I hope there is enough information. Please, help

+5
source share
8 answers

In my case, I went to / etc / hosts and:

  • deleted the line with 127.0.1.1 and it worked.
  • wrote "MASTER_IP MACHINE_NAME"
+4
source

"./sbin/start-master -h". , IP-.

+4

, , :

$ sudo service iptables stop

:

$ sudo chkconfig iptables off
+1

, IP . . , .

- :

SPARK_MASTER_IP=YOUR_SPARK_MASTER_IP ${SPARK_HOME}/sbin/start-master.sh

,

${SPARK_HOME}/sbin/start-slave.sh spark://YOUR_SPARK_MASTER_IP:PORT

, !

+1

SPARK_MASTER_HOST = <ip> spark-env.sh .

, grep , ,

Java -cp/spark/conf/: /spark/jars/* -Xmx1g org.apache.spark.deploy.master.Master --host < > --port 7077 --webui-port 8080

Java -cp/spark/conf/: /spark/jars/* -Xmx1g org.apache.spark.deploy.master.Master --host <HOST IP> --port 7077 - - 8080

+1

- etc/hosts, .

0

Windows: org.apache.spark.deploy.master.Master -h [IP- ]

0

I had a similar problem in the dock container, I solved it by setting the IP for master and driver as localhost, namely:

    set('spark.master.hostname' ,'localhost')
    set('spark.driver.hostname', 'localhost')
0
source

More articles:


All Articles