WSO2 ESB Unable to convert full JSON data to XML

Question

I am creating a POC. And I created a pass through a proxy service for Google Plus. Without using any proxy service, I get this my output:

{ "kind":"plus#person", "etag":"\"ExituU7aUpmkkfyD52VulzptThw/4J1clegrhxYC2fsJOu2XWCs1Ewg\"", "id":"117488614303967062311", "displayName":"Abhi NeoN", "name":{ "familyName":"NeoN", "givenName":"Abhi" }, "tagline":"hey guys ! ssup!! check out ma recnt videos... uploaded", "gender":"male", "aboutMe":"\u003cb\u003ehie, abhishek - ma full name \u003c/b\u003e\u003cdiv\u003e\u003cb\u003em a DANCER ,\u003c/b\u003e\u003c/div\u003e\u003cdiv\u003e\u003cb\u003ei luv ma dancing .\u003c/b\u003e\u003c/div\u003e\u003cdiv\u003e\u003cb\u003ei care ma dancing ,\u003c/b\u003e\u003c/div\u003e\u003cdiv\u003e\u003cb\u003ei jus hv a gr8 thng in me dats ma dancing.\u003c/b\u003e\u003c/div\u003e", "relationshipStatus":"single", "url":"https://plus.google.com/117488614303967062311", "image":{ "url":"https://lh6.googleusercontent.com/-tF-ip0tUxD4/AAAAAAAAAAI/AAAAAAAAAAA/WKI3USUh_DA/photo.jpg?sz=50" }, "urls":[ { "value":"https://plus.google.com/117488614303967062311", "type":"profile" }, { "value":"https://www.googleapis.com/plus/v1/people/117488614303967062311", "type":"json" } ], "organizations":[ { "name":"our lady of nazareth high school", "title":"science", "type":"school" }, { "name":"", "title":"BLUEBYTES", "type":"work" } ] } 

But when I try to do the same with a simple pass through the service, I get only:

 { "kind":"plus#person" } 

I read on wso2esb that they had an error, and the explanation given to fix the error was that the received json data was not in the correct format. But now how to solve the problem. I mean, they can somehow manipulate json data before esb converts it to json data.

Answer 1

We solved this problem in the latest version of ESB (version 4.5.0). By default, it comes with a JSONMessageFormatter / JSONBuilder that can handle JSON payload with multiple keys.

We also developed another solution for processing message flows, which include various types of JSON ↔ XML conversions (or JSON ↔ JSON). JSONStreamBuilder and JSONStreamFormatter can be used to implement such scripts using a script broker. See Example No. 441 in ESB 4.5.0.

To run sample # 441;

• Add JSONStreamBuilder and JSONStreamFormatter as the developer and formatter for JSON in the repository file / conf / axis 2 / axis2.xml
• SimpleStockQuoteService Deployment
• Run the axis2server sample
• Run the JSON client using < ant newjsonclient

Answer 2

This is one of the limitations of the current axis2 JSON creator / formatter. We are currently working on a new builder / formatter pair for JSON that does not convert JSON ↔ XML. Instead, it (the builder) saves the JSON message as a stream, and a script broker can be used to create a JSON object from this stream. For example, if we send {"a": "x", "b": "y"} as a request to the ESB, we can manipulate this request as a JSON object with javascript.

 var a = mc.getJSON().a.toString(); var b = mc.getJSON().b.toString(); mc.setPayloadXML( <m:A xmlns:m="http://example.json"> <m:a>{a}</m:a> <m:b>{b}</m:b> </m:A>); 

Similarly, the mc.setJSON() method can be used to set arbitrary JSON objects.

Answer 3

The only way to reliably convert json to xml and back is to use type hints in xml. the default converter does not. This 1. throws everything after the first property 2. mixes lists of individual elements with properties when moving from xml to json

I redefined the transconversion classes using the json-util library, which converts json-xml-containing type hints as element attributes so that there is no ambiguity.

this way we can use an intelligent proxy server (i.e. content route and mediate during transportation and payload) for ALL json-based support services via WSO2 without problems

This solves the problem (I think the camel does this by default).

Here is the pom file and code:

put the jar in / repository / components / lib

you must update the messageformatter and messagebuilder mappings for the content type "application / json" in axis2.xml

---

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <name>wso2 json/xml converter</name> <groupId>xyz</groupId> <artifactId>wso2converter</artifactId> <version>1.0.0-SNAPSHOT</version> <packaging>jar</packaging> <properties> <jdk.version>1.6</jdk.version> </properties> <build> <finalName>wso2converter</finalName> <resources> <resource> <filtering>false</filtering> <directory>src/main/resources</directory> </resource> </resources> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>${jdk.version}</source> <target>${jdk.version}</target> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0.1</version> <executions> <execution> <id>enforce-jdk</id> <phase>validate</phase> <goals> <goal>display-info</goal> <goal>enforce</goal> </goals> <configuration> <rules> <requireJavaVersion> <version>[${jdk.version},)</version> </requireJavaVersion> </rules> </configuration> </execution> </executions> </plugin> </plugins> </build> <dependencies> <dependency> <groupId>net.sf.json-lib</groupId> <artifactId>json-lib</artifactId> <version>2.3</version> <classifier>jdk15</classifier> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-io</artifactId> <version>1.3.2</version> <type>jar</type> <scope>compile</scope> </dependency> <dependency> <groupId>org.apache.ws.commons.axiom</groupId> <artifactId>axiom-api</artifactId> <version>1.2.13</version> </dependency> <dependency> <groupId>org.apache.axis2</groupId> <artifactId>axis2-kernel</artifactId> <version>1.6.2</version> </dependency> <dependency> <groupId>xom</groupId> <artifactId>xom</artifactId> <version>1.1</version> </dependency> <dependency> <groupId>org.apache.synapse</groupId> <artifactId>synapse-core</artifactId> <version>2.1.0</version> </dependency> <dependency> <groupId>com.sun.jersey</groupId> <artifactId>jersey-json</artifactId> <version>1.1.5</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.8.2</version> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.13</version> <!--scope>provided</scope--> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.4</version> <scope>test</scope> </dependency> </dependencies> </project> 

---

 package abwso2; import java.io.InputStream; import net.sf.json.JSON; import net.sf.json.JSONSerializer; import net.sf.json.xml.XMLSerializer; import org.apache.axiom.om.OMElement; import org.apache.axiom.om.util.AXIOMUtil; import org.apache.axis2.AxisFault; import org.apache.axis2.builder.Builder; import org.apache.axis2.context.MessageContext; import org.apache.commons.io.IOUtils; import org.apache.log4j.BasicConfigurator; import org.apache.log4j.Logger; public class WsoJtoX implements Builder { Logger logger = Logger.getLogger("abwso2converter"); @Override public OMElement processDocument(InputStream is, String contentType, MessageContext messageContext) throws AxisFault { String jsonData = ""; try { jsonData = IOUtils.toString(is,"UTF-8"); String output = process(jsonData); OMElement e = AXIOMUtil.stringToOM(output); return e; } catch (Exception e) { logger.error("error converting json string " + jsonData, e); if (e instanceof AxisFault) { throw (AxisFault) e; } throw new AxisFault("(B"+counter+") error converting json to xml", e); } } static int counter=0; public String process(String jsonData) throws AxisFault { try { String tran = "__ns__"; jsonData=jsonData.replace("\r", "").trim(); //jsonData=jsonData.replace("\n", ""); String decoded = (jsonData.replaceAll("\"([a-zA-Z0-9_]*)\\:([a-zA-Z0-9]*)\"(\\s*)(:)", "\"$1" + tran + "$2\"$3:")); counter++; if (logger.isDebugEnabled()) { logger.debug("\n>>>>> (B"+counter+") converting json\n " + jsonData + "\n===="); } XMLSerializer serializer = new XMLSerializer(); JSON json = JSONSerializer.toJSON(decoded); String xml = serializer.write(json); //add in the soap stuff StringBuilder sb = new StringBuilder(); sb.append("<soap:Envelope xmlns:soap=\"http://www.w3.org/2001/12/soap-envelope\" soap:encodingStyle=\"http://www.w3.org/2001/12/soap-encoding\"> <soap:Body>"); sb.append(xml.replace("<?xml version=\"1.0\" encoding=\"UTF-8\"?>", "")); sb.append("</soap:Body></soap:Envelope>"); if (logger.isDebugEnabled()) { logger.debug("\n==== (B"+counter+") to xml\n" + sb.toString()+"\n<<<<<"); } return sb.toString(); } catch (Exception e) { throw new AxisFault("(B"+counter+") error transforming json to xml", e); } } } 

---

 package abwso2; import java.io.IOException; import java.io.OutputStream; import java.io.UnsupportedEncodingException; import java.net.URL; import net.sf.json.JSON; import net.sf.json.xml.XMLSerializer; import org.apache.axiom.om.OMElement; import org.apache.axiom.om.OMOutputFormat; import org.apache.axiom.om.util.AXIOMUtil; import org.apache.axis2.AxisFault; import org.apache.axis2.Constants; import org.apache.axis2.context.MessageContext; import org.apache.axis2.transport.MessageFormatter; import org.apache.log4j.BasicConfigurator; import org.apache.log4j.Logger; public class WsoXtoJ implements MessageFormatter { Logger logger = Logger.getLogger("abwso2converter"); private static int counter=0; public String convert(String xData) { counter++; if (logger.isDebugEnabled()) { logger.debug("\n]]]]] (A"+counter+") converting xml\n " + xData + "\n-----"); } try { String tran = "__ns__"; XMLSerializer serializer = new XMLSerializer(); OMElement e = AXIOMUtil.stringToOM(xData); OMElement b = (OMElement) e.getChildrenWithLocalName("Body").next(); b = (OMElement) b.getChildElements().next(); String xfrag = b.toStringWithConsume(); String str = ""; JSON j = serializer.read(xfrag); str = j.toString(); String nstr = str.replaceAll("\"([a-zA-Z0-9_]+)" + tran + "([a-zA-Z0-9]+)\"(\\s*)(:)", "\"$1:$2\"$3:"); //", "\"$1:$2\""); if (logger.isDebugEnabled()) { logger.debug("\n----- (A"+counter+") to json\n" + nstr+"\n[[[[["); } return nstr; } catch (Exception e) { throw new RuntimeException(e); } } @Override public String formatSOAPAction(MessageContext msgCtxt, OMOutputFormat format, String soapActionString) { return null; } @Override public byte[] getBytes(MessageContext ctx, OMOutputFormat format) throws AxisFault { String env=""; try { OMElement element = ctx.getEnvelope().getBody().getFirstElement(); String payload = this.convert(element.toString()); return payload.getBytes(format.getCharSetEncoding()); } catch (UnsupportedEncodingException e) { logger.error("(A"+counter+") error converting xml to json "+ctx.getEnvelope().toString()); throw AxisFault.makeFault(e); } } @Override public String getContentType(MessageContext msgCtxt, OMOutputFormat format, String soapActionString) { String contentType = (String) msgCtxt.getProperty(Constants.Configuration.CONTENT_TYPE); String encoding = format.getCharSetEncoding(); if (contentType == null) { contentType = (String) msgCtxt.getProperty(Constants.Configuration.MESSAGE_TYPE); } if (encoding != null) { contentType += "; charset=" + encoding; } return contentType; } @Override public URL getTargetAddress(MessageContext msgCtxt, OMOutputFormat format, URL targetURL) throws AxisFault { return targetURL; } @Override public void writeTo(MessageContext msgCtxt, OMOutputFormat format, OutputStream out, boolean preserve) throws AxisFault { try { out.write(this.getBytes(msgCtxt, format)); out.flush(); } catch (IOException e) { throw AxisFault.makeFault(e); } } } 

Answer 4

I had the same problem.

In my experience, the JSON parser for the WSO2 ESB (based on Axis2-json) only supports a subset of JSON:

• JSON must begin with "{", that is, there cannot be a JSONArray in the root.

• Only the first key-value pair will be considered. This is because JSON maps to an XML-like data structure, and XML must have a root, so the first key-value pair is considered root.

• The value of the first key-value pair should not be an array. This is because the converter needs to know which XML tag should be used for each value:

  eg: ... {"key": ["val1", "val2", ...]} → <key> val1 </key> <key> val2 </key> ....

I have the same problem and I want to find a fix for this. My thoughts are to create a new JSONBuilder (a parser that builds the internal construction of the SOAP message) and a JSONFormatter (serializer) to use a virtual root (for example, {" root ": ...}) to fake the parser.