JAXB/Moxy Unmarshalling assigns all field values to Map<String,Object> rather than the specific field provided for it

Question

In short, I would like to perform the unmarshalling as mentioned here but along with Map I will have one more @XmlElement. So one field is annotated with (Map field) @XmlPath(".") and another field with (String field) @XmlElement and then I would like to perform unmarshalling.

My main goal of the application is to convert XML->JSON and JSON->XML using the JAXB/Moxy and Jackson library. I am trying to unmarshal the XML and map it to the Java POJO. My XML can have some dedicated elements and some user-defined elements which can appear random so I would like to store them in Map<String, Object>. Hence, I am making use of XMLAdapter. I am following the blog article to do so. I am not doing exactly the same but a bit different.

The problem I am facing is during unmarshalling the dedicated fields are not taken into consideration at all. All the values are unmarshalled to Map<String.Object>. As per my understanding it's happening because of the annotation @XmlPath(".") and usage of XMLAdapter but If I remove this annotation then it won't work as expected. Can someone please help me with this issue? The marshaling works fine with both @XmlPath(".") and XMLAdapter. The problem is arising only during unmarshalling.

Following is my XML that I would like to convert to JSON: (Note: Name and Age are dedicated fields and others is the user-defined field.)

<Customer xmlns:google="https://google.com">
  <name>BATMAN</name>
  <age>2008</age>
  <google:main>
    <google:sub>bye</google:sub>
  </google:main>
</Customer>

Following is my Customer class used for marshaling, unmarshalling by Moxy and Jackson: (Note: Name and Age are dedicated fields and others is the user-defined field. I want others to store only the values that cannot be mapped directly to POJO such as google:main and its children from above XML)

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "others"})
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer {
  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  private Map<String, Object> others;
  //Getter, Setter and other constructors
}

Following is my TestAdapter class which will be used for the Userdefined fields:

class TestAdapter extends XmlAdapter<Wrapper, Map<String, Object>> {

  @Override
  public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("INSIDE UNMARSHALLING METHOD TEST");
    final Map<String, Object> others = new HashMap<>();

    for (Object obj : value.getElements()) {
      final Element element = (Element) obj;
      final NodeList children = element.getChildNodes();

      //Check if its direct String value field or complex
      if (children.getLength() == 1) {
        others.put(element.getNodeName(), element.getTextContent());
      } else {
        List<Object> child = new ArrayList<>();
        for (int i = 0; i < children.getLength(); i++) {
          final Node n = children.item(i);
          if (n.getNodeType() == Node.ELEMENT_NODE) {
            Wrapper wrapper = new Wrapper();
            List childElements = new ArrayList();
            childElements.add(n);
            wrapper.elements = childElements;
            child.add(unmarshal(wrapper));
          }
        }
        others.put(element.getNodeName(), child);
      }
    }

    return others;
  }

  @Override
  public Wrapper marshal(Map<String, Object> v) throws Exception {
    Wrapper wrapper = new Wrapper();
    List elements = new ArrayList();
    for (Map.Entry<String, Object> property : v.entrySet()) {
      if (property.getValue() instanceof Map) {
        elements.add(new JAXBElement<Wrapper>(new QName(property.getKey()), Wrapper.class, marshal((Map) property.getValue())));
      } else {
        elements.add(new JAXBElement<String>(new QName(property.getKey()), String.class, property.getValue().toString()));
      }
    }
    wrapper.elements = elements;
    return wrapper;
  }
}

@Getter
class Wrapper {

  @XmlAnyElement
  List elements;
}

And finally, my Main class will be used for marshaling and unmarshalling. Also, to convert to JSON and XML.

class Main {

  public static void main(String[] args) throws JAXBException, XMLStreamException, JsonProcessingException {

    //XML to JSON
    JAXBContext jaxbContext = JAXBContext.newInstance(Customer.class);
    Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    InputStream inputStream = Main.class.getClassLoader().getResourceAsStream("Customer.xml");
    final XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
    final XMLStreamReader streamReader = xmlInputFactory.createXMLStreamReader(inputStream);
    final Customer customer = unmarshaller.unmarshal(streamReader, Customer.class).getValue();
    final ObjectMapper objectMapper = new ObjectMapper();
    final String jsonEvent = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer);
    System.out.println(jsonEvent);

    //JSON to XML
    Marshaller marshaller = jaxbContext.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
    marshaller.marshal(customer, System.out);
  }
}

When I convert the XML->JSON then I get the following output: (If you observe the fields name and age are not taken as the dedicated fields from Customer class rather its taken as random fields and written within the others)

{
  "name" : "",
  "age" : "",
  "others" : {
    "google:main" : [ {
      "google:sub" : "bye"
    } ],
    "name" : "BATMAN",
    "age" : "2008"
  }
}

I want my output to be something like this: (I want my dedicated fields to be mapped first then if there are any unknown fields then map them later within others MAP). Please note that I do not want to get others tag within my JSON. I want to get the names of the fields only for the dedicated fields.

{
  "name": "BATMAN",
  "age": 2008,
  "google:main": {
    "google:sub": "bye"
  }
}

Following is the XML that I would like to get during the marshaling. Also, please note I am using @XmlPath(".") so that I do not get the others node within my XML during marshaling.

<Customer>
    <name>BATMAN</name>
    <age>2008</age>
    <google:main>>
        <google:sub>bye</google:sub>
    </google:main>
</Customer>

The marshaling is working fine. The problem is happening during .unmarshaling As per my understanding it's happening because of the annotation @XmlPath(".") with XMLAdapter but If I remove this annotation then it won't work as expected. Can someone please help me with this issue?

** Edited **

I thought of a few workarounds but nothing seems to work for me. They are getting messed up due to @XmlPath("."). Still looking for some idea or workarounds. Any help would be really appreciated.

Answer 1

I believe this is somewhat related to the issue: @XmlPath(".") conflicts with @XmlAdapter

As per bug ticket:

org.eclipse.persistence.internal.oxm.record.UnmarshalRecordImpl 
public XPathNode getNonAttributeXPathNode(String namespaceURI, String localName, String qName, Attributes attributes) {
...
Line 1279
       if(null == resultNode && null == nonPredicateNode) {
          // ANY MAPPING
          resultNode = xPathNode.getAnyNode();// by default it return the EventAdapter returing a null at this place fix the problem, but i dont know if its the best solution
       }

Answer 2

As (just) a workaround, dedicated fields could go to a separate class so it's unmarshal on its own. "Customer" would extend from this class.

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "Person", propOrder = { "name", "age"})
public class Person {

    protected String name;
    protected String age;
//getters and stuff
}

Customer

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer") //, propOrder = { "name", "age", "others" }
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer extends Person{
    @XmlPath(".")
    @XmlJavaTypeAdapter(TestAdapter.class)
    private Map<String, Object> others;
    
}

Custom adapter could not parse elements without namespace or exclude a particular namespace added to Person class.

@Override
public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("INSIDE UNMARSHALLING METHOD TEST");
    final Map<String, Object> others = new HashMap<>();

    for (Object obj : value.getElements()) {
        final Element element = (Element) obj;
        final NodeList children = element.getChildNodes();
        System.out.println(element.getNodeName()+ " children.getLength(): " + children.getLength() + ", ns: " + element.getNamespaceURI());

        if (element.getNamespaceURI() != null) {
            // Check if its direct String value field or complex
            if (children.getLength() == 1) {
                others.put(element.getNodeName(), element.getTextContent());
            } else {
                List<Object> child = new ArrayList<>();
                for (int i = 0; i < children.getLength(); i++) {
                    final Node n = children.item(i);
                    if (n.getNodeType() == Node.ELEMENT_NODE) {
                        Wrapper wrapper = new Wrapper();
                        List<Object> childElements = new ArrayList<Object>();
                        childElements.add(n);
                        wrapper.elements = childElements;
                        child.add(unmarshal(wrapper));
                    }
                }
                others.put(element.getNodeName(), child);
            } 
        }
    }

    return others;
}

Finally, parsing would use DOM object so input is read once

public static void main(String[] args) throws Exception {

    // XML to JSON
    jakarta.xml.bind.JAXBContext jaxbContext = JAXBContext.newInstance(Person.class, Customer.class);
    jakarta.xml.bind.Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    File initialFile = new File("src/main/resources/Customer.xml");

    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    dbf.setNamespaceAware(true);
    DocumentBuilder db = dbf.newDocumentBuilder();
    Document doc = db.parse(initialFile);

    final Person person = unmarshaller.unmarshal(doc, Person.class).getValue();
    final Customer customer = unmarshaller.unmarshal(doc, Customer.class).getValue();
    System.out.println("unmarshall:" + customer);

    customer.setName(person.getName());
    customer.setAge(person.getAge());

    final ObjectMapper objectMapper = new ObjectMapper();
    final String jsonEvent = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer);
    System.out.println(jsonEvent);

    // JSON to XML
    Marshaller marshaller = jaxbContext.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
    marshaller.marshal(customer, System.out);
}

Answer 3

ah, finally some relief. This issue ate my head a lot but I was finally able to find a workaround. Tried a lot of things and reached out to many people but nothing seems to work and I thought it's an issue from the JAXB/Moxy library. I was able to find a workaround. Hope it helps someone in the future and do not get frustrated like me :)

I used 2 fields one with @XmlAnyElement(lax=true) List<Object> for storing the elements during the marshaling and another Map<String, Object> with custom serialization for JSON. In addition to this, I got to know that we can use beforeMarshal, afterMarshal, beforeUnmarshal, afterMarshal methods. The name itself suggests what it does.

In my case, I used the beforeMarshal method to add the unknown data from my Map<String, Object> to List<Object> so during the marshaling values from List<Object> will be used. I removed the XMLAdapter.

Also, the afterUnmarshal method to add the read unknown elements from List<Object> to Map<String, Object> so Jackson can utilize it and write to JSON using CustomSearlizer.

Basically, it's a kind of hide-and-show approach. List<Object> will be used during the unmarshalling and marshaling by JAXB/Moxy. Map<String, Object> will be used during the serialization and deserialization by Jackson.

Custome.class with my beforeMarshal and afterUnmarshalling: (It seems bit complex basically it exchanges the data as mentioned above. I will have complex data so I need to recursively loop and arrange. You can make changes according to your need)

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, visible = true, property = "isA")
@JsonInclude(Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "otherElements"})
@XmlAccessorType(XmlAccessType.FIELD)
@Getter
@Setter
@AllArgsConstructor
@ToString
public class Customer {
    @XmlTransient
    private String isA;
    private String name;
    private String age;

    @XmlAnyElement(lax = true)
    @JsonIgnore
    private List<Object> otherElements = new ArrayList<>();


    @JsonIgnore
    @XmlTransient
    private Map<String, Object> userExtensions = new HashMap<>();

    @JsonAnyGetter
    @JsonSerialize(using = CustomExtensionsSerializer.class)
    public Map<String, Object> getUserExtensions() {
        return userExtensions;
    }

    @JsonAnySetter
    public void setUserExtensions(String key, Object value) {
        userExtensions.put(key, value);
    }

    private void beforeMarshal(Marshaller m) throws ParserConfigurationException {
        System.out.println("Before Marshalling User Extension: " + userExtensions);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        otherElements = extensionsModifier.Marshalling(userExtensions);
        System.out.println("Before Marshalling Final Other Elements " + otherElements);
        userExtensions = new HashMap<>();
    }

    private void afterUnmarshal(Unmarshaller m, Object parent) throws ParserConfigurationException {
        System.out.println("After Unmarshalling : " + otherElements);
        ExtensionsModifier extensionsModifier = new ExtensionsModifier();
        userExtensions = extensionsModifier.Unmarshalling(otherElements);
        otherElements = new ArrayList();
    }
}

Then the ExtensionsModifier.class which will be called by beforeMarshal and afterUnmarshalling method:

import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;

import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class ExtensionsModifier {
    private javax.xml.parsers.DocumentBuilderFactory documentFactory;
    private javax.xml.parsers.DocumentBuilder documentBuilder;
    private org.w3c.dom.Document document;

    public ExtensionsModifier() throws ParserConfigurationException {
        documentFactory = DocumentBuilderFactory.newInstance();
        documentBuilder = documentFactory.newDocumentBuilder();
        document = documentBuilder.newDocument();
    }

    public List<Object> Marshalling(Map<String, Object> userExtensions) throws ParserConfigurationException {
        if (userExtensions == null) {
            return null;
        }
        List<Object> tempElement = new ArrayList<>();

        for (Map.Entry<String, Object> property : userExtensions.entrySet()) {
            Element root = document.createElement(property.getKey());
            if (property.getValue() instanceof Map) {
                List<Object> mapElements = Marshalling((Map<String, Object>) property.getValue());
                mapElements.forEach(innerChildren -> {
                    if (innerChildren instanceof Element) {
                        if (((Element) innerChildren).getTextContent() != null) {
                            root.appendChild(document.appendChild((Element) innerChildren));
                        }
                    }
                });
                tempElement.add(root);
            } else if (property.getValue() instanceof String) {
                root.setTextContent(((String) property.getValue()));
                tempElement.add(root);
            } else if (property.getValue() instanceof ArrayList) {
                for (Object dupItems : (ArrayList<Object>) property.getValue()) {
                    if (dupItems instanceof Map) {
                        Element arrayMap = document.createElement(property.getKey());
                        List<Object> arrayMapElements = Marshalling((Map<String, Object>) dupItems);
                        arrayMapElements.forEach(mapChildren -> {
                            if (mapChildren instanceof Element) {
                                if (((Element) mapChildren).getTextContent() != null) {
                                    arrayMap.appendChild(document.appendChild((Element) mapChildren));
                                }
                            }
                        });
                        tempElement.add(arrayMap);
                    } else if (dupItems instanceof String) {
                        Element arrayString = document.createElement(property.getKey());
                        arrayString.setTextContent((String) dupItems);
                        tempElement.add(arrayString);
                    }
                }
            }
        }
        return tempElement;
    }

    public Map<String, Object> Unmarshalling(List<Object> value) {
        if (value == null) {
            return null;
        }
        final Map<String, Object> extensions = new HashMap<>();
        for (Object obj : value) {
            org.w3c.dom.Element element = (org.w3c.dom.Element) obj;
            final NodeList children = element.getChildNodes();

            //System.out.println("Node Name : " + element.getNodeName() + " Value : " + element.getTextContent());
            List<Object> values = (List<Object>) extensions.get(element.getNodeName());

            if (values == null) {
                values = new ArrayList<Object>();
            }

            if (children.getLength() == 1) {
                values.add(element.getTextContent());
                extensions.put(element.getNodeName(), values);
            } else {
                List<Object> child = new ArrayList<>();
                for (int i = 0; i < children.getLength(); i++) {
                    final Node n = children.item(i);
                    if (n.getNodeType() == Node.ELEMENT_NODE) {
                        List<Object> childElements = new ArrayList();
                        childElements.add(n);
                        values.add(Unmarshalling(childElements));
                        child.add(Unmarshalling(childElements));

                    }
                }
                extensions.put(element.getNodeName(), values);
            }
        }
        return extensions;
    }
}

Following is my CustomSearlizer which will be used by Jackson to create JSON:

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Map;

public class CustomExtensionsSerializer extends JsonSerializer<Map<String, Object>> {

    private static final ObjectMapper mapper = new ObjectMapper();

    @Override
    public void serialize(Map<String, Object> value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
        System.out.println("Custom Json Searlizer: " + value);
        recusiveSerializer(value, gen, serializers);
    }

    public void recusiveSerializer(Map<String, Object> value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
        for (Map.Entry<String, Object> extension : value.entrySet()) {
            if (extension.getValue() instanceof Map) {
                //If instance is MAP then call the recursive method
                recusiveSerializer((Map) extension.getValue(), gen, serializers);
            } else if (extension.getValue() instanceof String) {
                //If instance is String directly add it to the JSON
                gen.writeStringField(extension.getKey(), (String) extension.getValue());
            } else if (extension.getValue() instanceof ArrayList) {
                //If instance if ArrayList then loop over it and add it to the JSON after calling recursive method
                //If size more than 1 add outer elements
                if (((ArrayList<Object>) extension.getValue()).size() > 1) {
                    gen.writeFieldName(extension.getKey());
                    gen.writeStartObject();
                    for (Object dupItems : (ArrayList<Object>) extension.getValue()) {
                        if (dupItems instanceof Map) {
                            recusiveSerializer((Map) dupItems, gen, serializers);
                        } else {
                            gen.writeStringField(extension.getKey(), (String) dupItems);
                        }
                    }
                    gen.writeEndObject();
                } else {
                    for (Object dupItems : (ArrayList<Object>) extension.getValue()) {
                        if (dupItems instanceof Map) {
                            gen.writeFieldName(extension.getKey());
                            gen.writeStartObject();
                            recusiveSerializer((Map) dupItems, gen, serializers);
                            gen.writeEndObject();
                        } else {
                            gen.writeStringField(extension.getKey(), (String) dupItems);
                        }
                    }
                }
            }
        }
    }
}

If I provide input as following XML:

<Customer xmlns:google="https://google.com">
    <name>Rise Against</name>
    <age>2000</age>
    <google:main>
        <google:sub>MyValue</google:sub>
        <google:sub>MyValue</google:sub>
    </google:main>
</Customer>

Then I get the following JSON as output:

{
  "isA" : "Customer",
  "name" : "Rise Against",
  "age" : "2000",
  "google:main" : {
    "google:sub" : "MyValue",
    "google:sub" : "MyValue"
  }
}

Viceversa would also work fine. Hope it's clear if not leave a comment will try to respond.