Weblogic.socket.Muxer uses a 100% processor

Recently, we started working with deployments in Weblogic 12c using a utility weblogic.Deployer. We can deploy the EAR, but whenever we try to deploy this application with a managed server, it will still start using 100% of our processor (4-core Xeon, bare-metal).

After some mastering and countless streams, we could isolate the problem on 4 stuck threads. Each of them consumed 100% on the core. The average value of the load spasmodically jumped down from 0.10 to 4.00 in 5 minutes.

These are threads that seem to be stuck:

"ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'" daemon prio=10 tid=0x00007fb52801c800 nid=0x6bf0 runnable [0x00007fb58a0ad000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
        - locked <0x00000000e18c66d0> (a sun.nio.ch.Util$2)
        - locked <0x00000000e18c66c0> (a java.util.Collections$UnmodifiableSet)
        - locked <0x00000000e18c6598> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
        at weblogic.socket.NIOSocketMuxer.selectFrom(NIOSocketMuxer.java:541)
        at weblogic.socket.NIOSocketMuxer.processSockets(NIOSocketMuxer.java:470)
        at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:30)
        at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:43)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)

It seems to me that many people have the same problem (not with Weblogic):

https://github.com/netty/netty/issues/327

https://issues.jboss.org/browse/XNIO-172

select() ?

, - JDK. java -version :

java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

, . WL , ?

!

+4
3

, , , , .

: fooobar.com/questions/126124/...

shebang, GC ( ) . .

-XX:+UseConcMarkSweepGC 
-XX:+UseParNewGC 
-XX:ParallelCMSThreads=2 
-XX:+CMSParallelRemarkEnabled 
-XX:+CMSIncrementalMode 
-XX:+CMSIncrementalPacing 
-XX:CMSFullGCsBeforeCompaction=1 
-XX:+CMSClassUnloadingEnabled 
-XX:CMSInitiatingOccupancyFraction=80

- , .

.

+3

. , :

1. posix-:

set('MuxerClass', 'weblogic.socket.PosixSocketMuxer')

. -

2. :

-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider -DUseSunHttpHandler=true
  • sun.nio.ch.PollSelectorProvider epoll_wait

  • -DUseSunHttpHandler = true weblogic http socket

+4

This is a known issue with Weblogic 12c and is published as the following Oracle support document:

Performance error due to using weblogic.socket.NIOSocketMuxer in WLS 12.1.2+ (Doc ID 2128032.1) ( link )

A workaround is to switch to using the native Muxer class, as described in Omar MEBARKI's answer.

The article does not address any or other workarounds mentioned in the other answers here.

+1
source

All Articles