Failed to deploy the CXF3.x JAX-RS service in Weblogic 12c (12.2.1)

Question

After updating the JAX-RS service built using CXF3.x, from weblogic 12.1.3 - 12.2.1. I ran into the following strange problem.

<03-ago-2017, 3:22:38,789 PM CEST> <Error> <HTTP> <BEA-101216> <Servlet: "JAX-RS/Jersey#1" failed to preload on startup in Web application: "sife". A MultiException has 1 exceptions. They are: 1. java.lang.LinkageError: ClassCastException: attempting to castzip:C:/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/sife/845176/war/WEB-INF/lib/javax.ws.rs-api-2.0.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/C:/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.ws.rs.javax.ws.rs-api.jar!/javax/ws/rs/ext/RuntimeDelegate.class at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:1007) at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:962) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1054) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1046) at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:696) Truncated. see log file for complete stacktrace 

Note that I'm using CXF, not Jersey, but it seems that Weblogic is trying to use it (this does not happen with 12.1.3). I disabled it in the configuration files, following these recommendations: https://jerometambo.imtqy.com/blog/2016/12/13/How-to-use-CXF-3x-implementation-of-JAX-RS-20-REST -with-Weblogic-12c.html

In the default configuration, Weblogic wants to use its internal implementation (JAX-RS 2.0 and serialization) to deploy REST web services (Error 500):

META-INF / WebLogic-application.xml

 <weblogic-application xmlns:wls="http://xmlns.oracle.com/weblogic/weblogic-application" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/weblogic/weblogic-application http://xmlns.oracle.com/weblogic/weblogic-application/1.8/weblogic-application.xsd"> <prefer-application-packages> <package-name>javax.ws.rs.*</package-name> <package-name>com.fasterxml.jackson.*</package-name> <package-name>weblogic.jaxrs.api.client.*</package-name> <package-name>weblogic.jaxrs.internal.api.*</package-name> <package-name>weblogic.jaxrs.dispatch.*</package-name> <package-name>weblogic.jaxrs.monitoring.util.*</package-name> </prefer-application-packages> </weblogic-application> 

WEB-INF / weblogic.xml

 <weblogic-web-app xmlns="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd"> <container-descriptor> <prefer-application-packages> <package-name>javax.ws.rs.*</package-name> <package-name>com.fasterxml.jackson.*</package-name> </prefer-application-packages> </container-descriptor> </weblogic-web-app> 

I do not know if this is a specific problem of my project or if it is a weblog issue. I found similar problems like this and this , and I tried to use them.

These are my dependencies (IVY syntax)

 <dependency org="org.apache.cxf" name="cxf-rt-frontend-jaxrs" rev="3.1.10" conf="default" /> <dependency org="org.apache.cxf" name="cxf-rt-rs-extension-providers" rev="3.1.10" conf="default" /> <dependency org="org.apache.cxf" name="cxf-rt-transports-http" rev="3.1.10" conf="default" /> <dependency org="com.fasterxml.jackson.jaxrs" name="jackson-jaxrs-json-provider" rev="2.6.6" conf="default" /> 

And full stack trace

 <03-ago-2017, 3:32:03,645 PM CEST> <Warning> <JAXRSIntegration> <BEA-2192510> <Cannot add Jersey servlet for application class org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet$ApplicationImpl because ApplicationPath annotation is not set on it.> <03-ago-2017, 3:32:03,737 PM CEST> <Warning> <JAXRSIntegration> <BEA-2192511> <The list of resource packages: com.fasterxml.jackson.jaxrs.base;org.apache.cxf.jaxrs.provider;com.fasterxml.jackson.jaxrs.json;org.apache.cxf.jaxrs.provider.aegis;es.aragon.sife.server.filter;org.apache.cxf.jaxrs.provider.jsrjsonp;es.aragon.sife.keystore.api;es.aragon.sife.server.account.api;org.apache.cxf.jaxrs.provider.xmlbeans;org.apache.cxf.jaxrs.validation;es.aragon.sife.server.task.api;org.apache.cxf.jaxrs.provider.json;org.apache.cxf.jaxrs.provider.atom;es.aragon.sife.server.config.api> ago 03, 2017 3:32:16 PM es.gob.afirma.signers.xml.Utils installXmlDSigProvider <03-ago-2017, 3:32:20,581 PM CEST> <Error> <HTTP> <BEA-101216> <Servlet: "JAX-RS/Jersey#1" failed to preload on startup in Web application: "sife". A MultiException has 1 exceptions. They are: 1. java.lang.LinkageError: ClassCastException: attempting to castzip:C:/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/sife/845176/war/WEB-INF/lib/javax.ws.rs-api-2.0.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/C:/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.ws.rs.javax.ws.rs-api.jar!/javax/ws/rs/ext/RuntimeDelegate.class at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:1007) at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:962) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1054) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1046) at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:696) Truncated. see log file for complete stacktrace Caused By: java.lang.LinkageError: ClassCastException: attempting to castzip:C:/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/sife/845176/war/WEB-INF/lib/javax.ws.rs-api-2.0.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/C:/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.ws.rs.javax.ws.rs-api.jar!/javax/ws/rs/ext/RuntimeDelegate.class at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:146) at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120) at javax.ws.rs.core.MediaType.valueOf(MediaType.java:179) at org.glassfish.jersey.media.sse.SseFeature.<clinit>(SseFeature.java:62) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) Truncated. see log file for complete stacktrace > <03-ago-2017, 3:32:20,696 PM CEST> <Error> <Deployer> <BEA-149231> <Unable to set the activation state to true for the application "sife". weblogic.application.ModuleException: java.lang.LinkageError: ClassCastException: attempting to castzip:C:/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/sife/845176/war/WEB-INF/lib/javax.ws.rs-api-2.0.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/C:/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.ws.rs.javax.ws.rs-api.jar!/javax/ws/rs/ext/RuntimeDelegate.class at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:140) at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:233) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:228) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:45) Truncated. see log file for complete stacktrace Caused By: java.lang.LinkageError: ClassCastException: attempting to castzip:C:/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/sife/845176/war/WEB-INF/lib/javax.ws.rs-api-2.0.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/C:/Oracle/Middleware/Oracle_Home/oracle_common/modules/javax.ws.rs.javax.ws.rs-api.jar!/javax/ws/rs/ext/RuntimeDelegate.class at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:146) at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120) at javax.ws.rs.core.MediaType.valueOf(MediaType.java:179) at org.glassfish.jersey.media.sse.SseFeature.<clinit>(SseFeature.java:62) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) Truncated. see log file for complete stacktrace > <03-ago-2017, 3:32:20,762 PM CEST> <Notice> <Log Management> <BEA-170027> <The server has successfully established a connection with the Domain level Diagnostic Service.> <03-ago-2017, 3:32:20,931 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to ADMIN.> <03-ago-2017, 3:32:21,46 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RESUMING.> <03-ago-2017, 3:32:21,174 PM CEST> <Notice> <WebLogicServer> <BEA-000360> <The server started in RUNNING mode.> <03-ago-2017, 3:32:21,186 PM CEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING.> <03-ago-2017, 3:32:22,710 PM CEST> <Error> <HTTP> <BEA-101216> <Servlet: "weblogic.management.rest.Application" failed to preload on startup in Web application: "wls-management-services.war". A MultiException has 1 exceptions. They are: 1. java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature at org.jvnet.hk2.internal.Utilities.justCreate(Utilities.java:1007) at org.jvnet.hk2.internal.ServiceLocatorImpl.create(ServiceLocatorImpl.java:962) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1054) at org.jvnet.hk2.internal.ServiceLocatorImpl.createAndInitialize(ServiceLocatorImpl.java:1046) at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:696) Truncated. see log file for complete stacktrace Caused By: java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.glassfish.hk2.utilities.reflection.ReflectionHelper.makeMe(ReflectionHelper.java:1350) Truncated. see log file for complete stacktrace > <03-ago-2017, 3:32:22,747 PM CEST> <Error> <J2EE> <BEA-160001> <Error deploying Internal Application: weblogic.application.ModuleException: java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature weblogic.application.utils.StateChangeException: weblogic.application.ModuleException: java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:58) at weblogic.application.services.BackgroundDeploymentManagerService$BackgroundDeployAction.run(BackgroundDeploymentManagerService.java:396) at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:666) at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:348) at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:333) Truncated. see log file for complete stacktrace Caused By: weblogic.application.ModuleException: java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:140) at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:233) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:228) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:45) Truncated. see log file for complete stacktrace Caused By: java.lang.NoClassDefFoundError: Could not initialize class org.glassfish.jersey.media.sse.SseFeature at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.glassfish.hk2.utilities.reflection.ReflectionHelper.makeMe(ReflectionHelper.java:1350) Truncated. see log file for complete stacktrace 

---

Temporary solution

It works if I deploy another CXF module earlier using the default JAX-RS implementation and removing WEB-INF/lib/javax.ws.rs-api-2.0.1.jar . Perhaps weblogic is doing some internal initialization.

  myapp.ear init.war (JAX-RS with CXF and removing javax.ws.rs-api-2.0.1.jar) myapp.war (JAX-RS with CXF and weblogic configuration) 

Answer 1

@pedrofb, this may not fully answer your question, however I had a similar problem with Tomcat. Symptoms are similar, the problem was not due to the fact that different banks have different classes, i.e. Bank shading. This was due to the shadowing of the class loader. In Tomcat, class loading is actually inverted, rather than the usual ask-your-parent-class-loader-first model, if the application loader class can find the class (where it loads only from WEB-INF / lib or classes) then classloader loads the class. Now, in your case, I think, since the JAX-RS banks are present in your WEB-INF / lib, it loads from there at the web application level. However, when the same object is passed to the container, it tries to pass it to the same class, but at the container level loaded by another class loader. Classes loaded by different class loaders are considered different; this will cause a ClassCastException.

This also explains why ClassCastException does not occur when removing the JAX-RS jar from WEB-INF / lib. The workaround here is limited, or you remove the JAX-RS jar from your WEb-INF / lib as you did. (This can be potentially problematic, but you can experiment on an unproductive server), replace the JAX-RS container with the one you want.

As I said, this probably will not solve your problem completely, but it may indicate the direction in which the problem occurs. [As I noticed a few years ago, Weblogic uses tomcat as the main core of the web container (not sure if it's the same) with changed package names like com.bea.weblogic.org.apache.tomcat ... ].

Hope this helps.