Jackson throws mistakes

Trying to convert XML to JSON in STORM Trident My code worked fine in storm 0.9.1 and now we have updated to the latest version of Storm 0.10.0 and we will get errors below if you have any ideas about let me know ?

11752 [Thread-56-b-1] ERROR bsutil - Async loop died! java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findDefaultDeserializer(BasicDeserializerFactory.java:1522) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.findStdDeserializer(BeanDeserializerFactory.java:167) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:132) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:403) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:352) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:461) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3804) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3698) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2714) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.tda.trident.bolt.HdfsStateConvertor$HdfsFileOptions.execute(HdfsStateConvertor.java:278) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.tda.trident.bolt.HdfsStateConvertor.updateState(HdfsStateConvertor.java:462) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:15) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at com.tda.trident.bolt.HdfsUpdater.updateState(HdfsUpdater.java:9) ~[StormTridentXmlNew-0.0.1-SNAPSHOT.jar:?] at storm.trident.planner.processor.PartitionPersistProcessor.finishBatch(PartitionPersistProcessor.java:98) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at storm.trident.planner.SubtopologyBolt.finishBatch(SubtopologyBolt.java:152) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at storm.trident.topology.TridentBoltExecutor.finishBatch(TridentBoltExecutor.java:253) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at storm.trident.topology.TridentBoltExecutor.checkFinish(TridentBoltExecutor.java:286) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:360) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.daemon.executor$fn__6214$tuple_action_fn__6216.invoke(executor.clj:670) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.daemon.executor$mk_task_receiver$fn__6137.invoke(executor.clj:426) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.disruptor$clojure_handler$reify__5713.onEvent(disruptor.clj:58) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.daemon.executor$fn__6214$fn__6227$fn__6278.invoke(executor.clj:808) ~[storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at backtype.storm.util$async_loop$fn__543.invoke(util.clj:475) [storm-core-0.10.0.2.3.2.0-2950.jar:0.10.0.2.3.2.0-2950] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?] at java.lang.Thread.run(Thread.java:744) [?:1.7.0_45] 

My maven shadow output when building POM is shown below.

 [INFO] --- maven-shade-plugin:1.4:shade (default) @ StormTridentXmlNew --- [INFO] Including org.apache.hadoop:hadoop-auth:jar:2.7.1.2.3.2.0-2950 in the shaded jar. [INFO] Including org.apache.curator:curator-framework:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.storm:storm-hdfs:jar:0.10.0.2.3.2.0-2950 in the shaded jar. [INFO] Including org.yaml:snakeyaml:jar:1.11 in the shaded jar. [INFO] Including com.hmsonline:storm-jms:jar:0.2.2 in the shaded jar. [INFO] Including org.apache.geronimo.specs:geronimo-jms_1.1_spec:jar:1.1.1 in the shaded jar. [INFO] Including com.tibco.ems:tibjms:jar:5.1 in the shaded jar. [INFO] Including org.springframework:spring-beans:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including org.springframework:spring-core:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including commons-logging:commons-logging:jar:1.2 in the shaded jar. [INFO] Including org.springframework:spring-context:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including org.springframework:spring-aop:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar. [INFO] Including org.springframework:spring-expression:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including org.springframework:spring-jms:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including org.springframework:spring-messaging:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including org.springframework:spring-tx:jar:4.1.3.RELEASE in the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-core:jar:2.6.5 in the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5 in the shaded jar. [INFO] Including com.fasterxml.jackson.core:jackson-databind:jar:2.6.5 in the shaded jar. [INFO] Including com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.6.5 in the shaded jar. [INFO] Including com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.6.5 in the shaded jar. [INFO] Including org.codehaus.woodstox:stax2-api:jar:3.1.4 in the shaded jar. [INFO] Including org.codehaus.woodstox:woodstox-core-asl:jar:4.4.1 in the shaded jar. [INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-client:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.1 in the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-client:jar:2.7.1 from the shaded jar. [INFO] Including org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1 in the shaded jar. [INFO] Excluding org.apache.hadoop:hadoop-yarn-common:jar:2.7.1 from the shaded jar. [INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar. [INFO] Including javax.activation:activation:jar:1.1 in the shaded jar. [INFO] Including com.sun.jersey:jersey-client:jar:1.9 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-annotations:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-common:jar:2.7.1 in the shaded jar. [INFO] Including com.google.guava:guava:jar:11.0.2 in the shaded jar. [INFO] Including commons-cli:commons-cli:jar:1.2 in the shaded jar. [INFO] Including org.apache.commons:commons-math3:jar:3.1.1 in the shaded jar. [INFO] Including xmlenc:xmlenc:jar:0.52 in the shaded jar. [INFO] Including commons-httpclient:commons-httpclient:jar:3.1 in the shaded jar. [INFO] Including commons-codec:commons-codec:jar:1.4 in the shaded jar. [INFO] Including commons-io:commons-io:jar:2.4 in the shaded jar. [INFO] Including commons-net:commons-net:jar:3.1 in the shaded jar. [INFO] Including commons-collections:commons-collections:jar:3.2.1 in the shaded jar. [INFO] Including javax.servlet:servlet-api:jar:2.5 in the shaded jar. [INFO] Including org.mortbay.jetty:jetty:jar:6.1.26 in the shaded jar. [INFO] Including org.mortbay.jetty:jetty-util:jar:6.1.26 in the shaded jar. [INFO] Including javax.servlet.jsp:jsp-api:jar:2.1 in the shaded jar. [INFO] Including com.sun.jersey:jersey-core:jar:1.9 in the shaded jar. [INFO] Including com.sun.jersey:jersey-json:jar:1.9 in the shaded jar. [INFO] Including org.codehaus.jettison:jettison:jar:1.1 in the shaded jar. [INFO] Including com.sun.xml.bind:jaxb-impl:jar:2.2.3-1 in the shaded jar. [INFO] Including com.sun.jersey:jersey-server:jar:1.9 in the shaded jar. [INFO] Including asm:asm:jar:3.1 in the shaded jar. [INFO] Including log4j:log4j:jar:1.2.17 in the shaded jar. [INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar. [INFO] Including org.apache.httpcomponents:httpclient:jar:4.1.2 in the shaded jar. [INFO] Including org.apache.httpcomponents:httpcore:jar:4.1.2 in the shaded jar. [INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar. [INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar. [INFO] Including commons-configuration:commons-configuration:jar:1.6 in the shaded jar. [INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar. [INFO] Excluding commons-beanutils:commons-beanutils:jar:1.7.0 from the shaded jar. [INFO] Excluding commons-beanutils:commons-beanutils-core:jar:1.8.0 from the shaded jar. [INFO] Excluding org.slf4j:slf4j-api:jar:1.7.10 from the shaded jar. [INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.10 from the shaded jar. [INFO] Including org.apache.avro:avro:jar:1.7.4 in the shaded jar. [INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.3 in the shaded jar. [INFO] Including org.xerial.snappy:snappy-java:jar:1.0.4.1 in the shaded jar. [INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar. [INFO] Including com.google.code.gson:gson:jar:2.2.4 in the shaded jar. [INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar. [INFO] Including org.apache.curator:curator-client:jar:2.7.1 in the shaded jar. [INFO] Including org.apache.curator:curator-recipes:jar:2.7.1 in the shaded jar. [INFO] Including com.google.code.findbugs:jsr305:jar:3.0.0 in the shaded jar. [INFO] Including org.apache.htrace:htrace-core:jar:3.1.0-incubating in the shaded jar. [INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.6 in the shaded jar. [INFO] Including io.netty:netty:jar:3.7.0.Final in the shaded jar. [INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded jar. [INFO] Including org.tukaani:xz:jar:1.0 in the shaded jar. [INFO] Including org.apache.hadoop:hadoop-hdfs:jar:2.7.1 in the shaded jar. [INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar. [INFO] Including io.netty:netty-all:jar:4.0.23.Final in the shaded jar. [INFO] Including xerces:xercesImpl:jar:2.9.1 in the shaded jar. [INFO] Including org.fusesource.leveldbjni:leveldbjni-all:jar:1.8 in the shaded jar. [INFO] Replacing original artifact with shaded artifact. 
+7
json xml jackson hadoop apache-storm
source share
3 answers

The latest version of the storm has jackson-core-2.3.1.jar in its lib folder, and my thick jar that I create using maven has a different version of Jackson dependencies. That's why when I send my jar to a storm cluster, it throws these no such method errors at runtime.

I changed my POM to use the same version of Jackson dependencies that the storm uses, and everything works fine.

0
source share

It looks like you have many versions of jsonson jars, and your class path references json Jackson to 2.6 at runtime.

The ResolvedType class represents isReferenceType () since version 2.6. See the link below:

http://googleweblight.com/?lite_url=http://fasterxml.imtqy.com/jackson-core/javadoc/2.6/com/fasterxml/jackson/core/type/ResolvedType.html&ei=ochB8Ipc&lc=en-IN&geid=10&s = 1 & m = 71 & ts = 1454581842 & sig = ALL1Aj7I2EAu3AoKmOu6FKbtFTTuKU02Bg # a-isReferenceType ()

Please make sure you specify the correct version or higher at runtime. You can remove / remove old Jackson cans if this is not required.

Hope this helps.

+5
source share

This problem seems to come from several versions of jackson classes in the classpath. You can remove unnecessary versions. At the time of this writing. The current version for jackson isis 2.7.1.

0
source share

All Articles