If I use spark in standalone applications, I donโt think I need a connection to the server (of course). But why am I getting this network error message?
[ERROR] Error while sending or receiving.
Traceback (most recent call last):
File "/Users/chlee021690/anaconda/lib/python2.7/site-packages/py4j/java_gateway.py",
line 473, in send_command
answer = smart_decode(self.stream.readline()[:-1])
File "/Users/chlee021690/anaconda/python.app/Contents/lib/python2.7/socket.py", line 430, in readline
data = recv(1)
timeout: timed out
....
Py4JNetworkError: An error occurred while trying to connect to the Java server.
My code is as follows:
from pyspark import *
import pyspark.mllib.recommendation as spark_rec
filename = "./yahoo music/train_0.txt"
sc = SparkContext('local', 'spark_rec')
aData = sc.textFile(filename).cache() (this part was successful, but the next following lines were failtures)
ratings = aData.map(lambda line: np.array([float(x) for x in line.split('\t')]))
rank = 10
numIterations = 20
aModel = spark_rec.ALS.train(ratings, rank, numIterations)
source
share