Support root level collection serialization

[sdeleuze]

I have done some tests with 2.1.0-SNAPSHOT since on RESThub project (https://github.com/resthub/resthub-spring-stack) we are waiting XML serialization of unwrapped lists for a while.

We use generic SpringMVC controller with Jackson 2.x for serialization.

@RequestMapping(method = RequestMethod.GET, params="page=no")
@responsebody
public List findAllNotPaginated() {
return (List)repository.findAll();
}

For JSON we get this normal output :

[{"id":17,"name":"toto"},{"id":18,"name":"toto"}]

For XML we get this bogus output (the same than is 2.0 I think) :

<ArrayList xmlns=""><id>41</id><name>toto</name></ArrayList><zdef1166645736:ArrayList xmlns:zdef1166645736=""><zdef1166645736:id>42</zdef1166645736:id><zdef1166645736:name>toto</zdef1166645736:name></zdef1166645736:ArrayList>

Is there something to do in order to activate Jackson 2.1 XML unwrapped list support, or is it a bug ?

[cowtowncoder]

Ah. Good question. So, here are some initial suggestions:

1. Extra namespace appears to be caused by Sun SJSXP Stax provider: using Woodstox will remove this. We don't know if there's a way to resolve this for SJSXP
2. Just to make sure: are you using XmlMapper (regular ObjectMapper will not work for all XML aspects). And for configurability, may also want to register JacksonXmlModule on that mapper (see next one)
3. Jackson 2.1 will still require explicit definition of whether to NOT use wrapping (due to backwards compatibility). There are multiple ways to do this; usually either use JacksonXmlModule.setDefaultUseWrapper(false), or annotation @JacksonXmlElementWrapper(useWrapping=false)
4. When using JAXB annotations, wrapping is disabled by default (i.e. no extra work is needed), if @xmlelement (etc) are found, and no @XmlElementWrapper

Third part is messy, but we weren't sure how much 2.0 backwards compatibility would matter.
But as long as you use the module, configure it, things should work.

Output looks funky, either way. If above does NOT help, please update the bug and I will have a look before 2.1.0 release.

[sdeleuze]

Hi,

Thanks for your feedback. I made more test :

1. When I use Woodstox, I get an exception saying it can't output two roots
2. Yes, I use XmlMapper
3. I have modify my code in order to pass an JacksonXmlModule with setDefaultUseWrapper(false) in XmlMapper constructor
4. Nice, thanks for the tip

Do you have some basic XmlMapper + unwrapped List simple unit tests in order to allow me to compare with my code ? We use a lot Generics, perhaps the issue comme from this.

I will also try to provide you later today some unit tests that reproduce my issue.

[cowtowncoder]

Ok. One more thing, if you can reproduce this with a stand-alone, that would be helpful.
Possibly extracting setup code that container uses.

The only other concern I have is that often root-level Collections and Maps are more problematic in XML than with JSON. But it's not necessary problem here.

[sdeleuze]

I will provide you standalone unit tests.

[sdeleuze]

You will find bellow my unit test, I used JDK 1.7 update 5

Without Woodstox, I got this serialization :

<ArrayList xmlns=""><id>123</id><name>Albert</name><description>desc</description></ArrayList><zdef1084405763:ArrayList xmlns:zdef1084405763=""><zdef1084405763:id>123</zdef1084405763:id><zdef1084405763:name>Albert</zdef1084405763:name><zdef1084405763:description>desc</zdef1084405763:description></zdef1084405763:ArrayList>

With Woodstox, I got a javax.xml.stream.XMLStreamException : trying to output second root, ArrayList

My use case seems really simple, I am missing something or is there an issue in Jackson 2.1.0-SNAPSHOT ?

The test

@Test
    public void testListSerialization() {
        SampleResource r1 = new SampleResource();
        r1.setId(123L);
        r1.setName("Albert");
        r1.setDescription("desc");

        SampleResource r2 = new SampleResource();
        r2.setId(123L);
        r2.setName("Albert");
        r2.setDescription("desc");

        List<SampleResource> l = new ArrayList<SampleResource>();
        l.add(r1);
        l.add(r2);

        String result = XmlHelper.serialize(l);
        Assertions.assertThat(result).contains("<id>123</id>");
        Assertions.assertThat(result).contains("<name>Albert</name>");
        Assertions.assertThat(result).contains("<description>desc</description>");
    }

SampleResource :

public class SampleResource {
    private Long id;
    private String name;
    private String description;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getDescription() {
        return description;
    }

    public void setDescription(String description) {
        this.description = description;
    }
}

XmlHelper :

package org.resthub.web;

import com.fasterxml.jackson.core.type.TypeReference;
import java.io.ByteArrayOutputStream;
import java.io.OutputStream;

import org.resthub.web.exception.SerializationException;

import com.fasterxml.jackson.databind.AnnotationIntrospector;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.introspect.JacksonAnnotationIntrospector;
import com.fasterxml.jackson.dataformat.xml.JacksonXmlModule;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;

/**
 * Helper for XML serialization and deserialization.
 */
public class XmlHelper {

    /**
     * Jackson Object Mapper used to serialization/deserialization
     */
    protected static ObjectMapper objectMapper;

    protected static void initialize() {
        JacksonXmlModule module = new JacksonXmlModule();
        module.setDefaultUseWrapper(false);
        objectMapper = new XmlMapper(module);
        AnnotationIntrospector introspector = new JacksonAnnotationIntrospector();
        objectMapper.setAnnotationIntrospector(introspector);
    }

    /**
     * Return the objectMapper. It can be used to customize serialization/deserialization configuration.
     * @return 
     */
    public ObjectMapper getObjectMapper() {
        if (objectMapper == null)
            initialize();
        return objectMapper;
    }

    /**
     * Serialize and object to an XML String representation
     * @param o The object to serialize
     * @return The XML String representation
     */
    public static String serialize(Object o) {
        if (objectMapper == null)
            initialize();
        OutputStream baOutputStream = new ByteArrayOutputStream();
        try {
            objectMapper.writeValue(baOutputStream, o);
        } catch (Exception e) {
            throw new SerializationException(e);
        }
        return baOutputStream.toString();
    }

    /**
     * Deserialize a XML string
     * @param content The XML String object representation
     * @param type The type of the deserialized object instance
     * @return The deserialized object instance
     */
    public static <T> T deserialize(String content, Class<T> type) {
        if (objectMapper == null)
            initialize();
        try {
            return objectMapper.readValue(content, type);
        } catch (Exception e) {
            throw new SerializationException(e);
        }
    }

    /**
     * Deserialize a XML string
     * @param content The JSON String object representation
     * @param valueTypeRef The typeReference containing the type of the deserialized object instance
     * @return The deserialized object instance
     */
    public static <T> T deserialize(String content, TypeReference valueTypeRef) {
        if (objectMapper == null)
            initialize();
        try {
            return objectMapper.readValue(content, valueTypeRef);
        } catch (Exception e) {
            throw new SerializationException(e);
        }
    }

}

[cowtowncoder]

I will have a look. Most tests so far have focused on Collections (and arrays) as POJO members.
So it is likely that root-level value is an edge-case.

[sdeleuze]

Thanks a lot !

For information, my real life use case is Spring MVC returning a XML collection.
I got the same issue than in this test case.

[cowtowncoder]

Ok good to know.

[cowtowncoder]

Ok. The problem is definitely due to this being a root value, and it is possible that we will have to actually explicitly state that root-level Collections and arrays are not supported. The issue is that modifications that are needed for collection types are handled through POJO property mechanisms, and at root level there is no referring property.

I will leave this issue open just in case we'll figure out a way to make things work, but for now (in 2.0 and 2.1), this use case is not supported.

[sdeleuze]

OK, I hope you will be able to support such functionnality in the future since handling List of elements is a very common use case, at least when you are using Jackson for serialization in REST webservices.

Since it is not supported by JAXB by default, it could make a really good reason for people using Jackson for XML. I know that Spring MVC folks are intersted by that (3.2 release with use Jackson by default but only for json for now)

Is there an existing annotation that we could use to specify the root property, or do you need to create a new one ? If the annotation exists, could it be possible to support this in 2.1.x ? Maybe I can help by contributing a fix with your guideance?

[cowtowncoder]

By root property I simply mean that Collection value needs to be a value of a POJO property. Simple wrapping within (XML) element does not matter. The problem is with the implementation of wrapping (or not) that relies on modifying handling of "bean properties" (BeanPropertyWriter at low level for writing; for reading handling is bit different).
So what is needed for now is simple wrapper object. Spring (et al) could provide such wrappers.

Note that Lists are fully supported as values of POJO properties (and as List elements as well, as long as outermost is a property value).

Ideally root-value lists would be supported, but it would be in 2.2 at earlier just due to work involved.
I mean, unless a simpler work-around is found, which could go in 2.1.x (never say never).

[cowtowncoder]

One last thing: I don't think JAXB supports serialization of Lists directly either. At least I wasn't able to do that for the use case, probably since there is no way to indicate type of elements (i.e. serializes as lists of empty objects).
Although serialization of arrays does work, since arrays retain type unlike Collections (no generic type erasure)..

[sdeleuze]

JAXB with SpringMVC does not support serialization of collections, but JAXB with Jersey does. It don't know how they did that.

Thanks for the tip, I will use plain arrays on my controllers in order to make them work with XML too.

Should I update the title of this issue or create a new issue for root Collection serialization ?

[cowtowncoder]

Updating the title would make sense if you can do it. I guess there is some way to specify full type, since what seemed to fail was simply passing of Collection value type; otherwise Collection itself was serialized correctly.

Interesting bit about Jersey -- would be good to know what they do, even if it does not directly help here.

On Jackson side, I think that with 2.2 we should try to find a better way to deal with Collections in general.
The other current problem is that if the static type is not Collection -- like, when we have plain Object -- same problem exists. One possibility is to add a new hook to allow post-processing non-POJO (de)serializers, similar to how Bean(De)SerializerModifier works. This is necessary for improved Scala support as well.