Spark streaming 2.0.0 - freezes after a few days under load

Question

We are running AWS EMR 5.0.0 with Spark 2.0.0. Consumption of 125 fragments of Kinesis. Submission of 19k events / s using 2 message producers, each message about 1k in size. Consumption using a cluster of 20 cars. The code has flatMap (), groupByKey (), persist (StorageLevel.MEMORY_AND_DISK_SER_2 ()) and redistribution (19); Then save to s3 using foreachRDD (); Using Back Pressure and Kryo:

sparkConf.set("spark.streaming.backpressure.enabled", "true"); sparkConf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"); 

During operation, Ganglia shows a steady increase in memory usage without a GC. At some point, when there is no free memory to allocate, Spark stops processing micropackets, and the incoming queue grows. The fact that the freezing point is sparking cannot be restored. In our case, Spark froze after 3.5 days of work under pressure.

Problem: we need to stream for at least a week (preferably more) without restarting.

Spark Configuration:

 spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.driver.extraJavaOptions -Dspark.driver.log.level=INFO -XX:+UseConcMarkSweepGC -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.master yarn-cluster spark.executor.instances 19 spark.executor.cores 7 spark.executor.memory 7500M spark.driver.memory 7500M spark.default.parallelism 133 spark.yarn.executor.memoryOverhead 2950 spark.yarn.driver.memoryOverhead 2950 spark.eventLog.enabled false spark.eventLog.dir hdfs:///spark-logs/ -XX: + PrintGCDateStamps -XX: + UseConcMarkSweepGC -XX: CMSInitiatingOccupancyFraction = spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.driver.extraJavaOptions -Dspark.driver.log.level=INFO -XX:+UseConcMarkSweepGC -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.master yarn-cluster spark.executor.instances 19 spark.executor.cores 7 spark.executor.memory 7500M spark.driver.memory 7500M spark.default.parallelism 133 spark.yarn.executor.memoryOverhead 2950 spark.yarn.driver.memoryOverhead 2950 spark.eventLog.enabled false spark.eventLog.dir hdfs:///spark-logs/ = 256M -XX spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.driver.extraJavaOptions -Dspark.driver.log.level=INFO -XX:+UseConcMarkSweepGC -XX:PermSize=256M -XX:MaxPermSize=256M -XX:OnOutOfMemoryError='kill -9 %p' spark.master yarn-cluster spark.executor.instances 19 spark.executor.cores 7 spark.executor.memory 7500M spark.driver.memory 7500M spark.default.parallelism 133 spark.yarn.executor.memoryOverhead 2950 spark.yarn.driver.memoryOverhead 2950 spark.eventLog.enabled false spark.eventLog.dir hdfs:///spark-logs/ 

Thanks in advance.