Not able to deserialize Enum with default typing after upgrading 2.15.4 -> 2.17.1

[lenrok258]

Search before asking

• I searched in the issues and found nothing similar.

Describe the bug

After bumping up the version to 2.17.1 I'm not longer able to deserialize values serialized with the version 2.15.4.
Moreover, values serialized with the new version (2.17.1) give error when deserializing.

Version Information

2.17.1

Reproduction

package com.kze.test;

import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.jsontype.BasicPolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.PolymorphicTypeValidator;
import com.fasterxml.jackson.databind.jsontype.impl.StdTypeResolverBuilder;

import java.io.IOException;
import java.util.EnumSet;

import static com.fasterxml.jackson.annotation.JsonTypeInfo.As.PROPERTY;
import static com.fasterxml.jackson.databind.ObjectMapper.DefaultTyping.NON_FINAL;
import static com.kze.test.TestClazz.TestEnum.TEST_ENUM_VALUE;

public class TestClazz {

    public enum TestEnum {
        TEST_ENUM_VALUE
    }

    public static void main(String[] args) throws IOException {

        ObjectMapper mapper = new ObjectMapper();

        final PolymorphicTypeValidator validator = BasicPolymorphicTypeValidator.builder()
                .allowIfSubType("com.kze.test")
                .allowIfSubType("java")
                .build();

        ObjectMapper.DefaultTypeResolverBuilder resolverBuilder
                = new ObjectMapper.DefaultTypeResolverBuilder(NON_FINAL, validator) {
            @Override
            public boolean useForType(JavaType t) {
                return true;
            }
        };

        StdTypeResolverBuilder stdTypeResolverBuilder = resolverBuilder
                .init(JsonTypeInfo.Id.CLASS, null)
                .inclusion(PROPERTY);

        mapper.setDefaultTyping(stdTypeResolverBuilder);
        mapper.configure(JsonParser.Feature.INCLUDE_SOURCE_IN_LOCATION, true);

        EnumSet<TestEnum> input = EnumSet.of(TEST_ENUM_VALUE);
        String inputJson = mapper.writeValueAsString(input);
        System.out.println(inputJson);

        Object inputDeserialized = mapper.readValue(inputJson, Object.class);
        System.out.println(inputDeserialized);
    }

}

Version 2.15.4 output

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]
[TEST_ENUM_VALUE]

Version 2.17.1 error:

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]
Exception in thread "main" com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_STRING), expected START_ARRAY: need Array value to contain `As.WRAPPER_ARRAY` type information for class com.kze.test.TestClazz$TestEnum
 at [Source: (String)"["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]"; line: 1, column: 56] (through reference chain: java.util.RegularEnumSet[0])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1913)
	at com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1699)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:141)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:96)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromScalar(AsArrayTypeDeserializer.java:66)
	at com.fasterxml.jackson.databind.deser.std.StdScalarDeserializer.deserializeWithType(StdScalarDeserializer.java:66)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer._deserialize(EnumSetDeserializer.java:281)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer.deserialize(EnumSetDeserializer.java:246)
	at com.fasterxml.jackson.databind.deser.std.EnumSetDeserializer.deserialize(EnumSetDeserializer.java:24)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:120)
	at com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromArray(AsArrayTypeDeserializer.java:53)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromAny(AsPropertyTypeDeserializer.java:238)
	at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserializeWithType(UntypedObjectDeserializerNR.java:112)
	at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:74)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4905)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3848)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3816)
	at com.kze.test.TestClazz.main(TestClazz.java:52)

Expected behavior

Should be able to deserialized values

Additional context

No response

[JooHyukKim]

Seems similar to the one I am working on ryt now FasterXML/jackson-modules-java8#335

[lenrok258]

I'm not sure the route cause is the same.

In my case serializing Enum does not preceded it with a class name as a first array element

["TEST_ENUM_VALUE"]

From some reason older version is able to read it. The newer one throws an error, expecting class name at index 0.

As a proof if you proceed a serialized Enum with a class name at the index "0" it works perfectly fine:

["com.kze.test.TestClazz$TestEnum", "TEST_ENUM_VALUE"]

[edit]
So the bottom line is:
newer Jackson serialize Enum without explicitly putting class name as a first value in the array (as in the all other cases) and throws an error it trying to deserialize it

[JooHyukKim]

There is ${rootDir}/release-notes to check what's changed from 2.15.4 -> later version. Could you try out with 2.16.0 as well? It would help track down which version

[lenrok258]

Just as a note, using recommended in docs activateDefaultTyping() instead of deprecated setDefaultTyping() doesn't help.

        ObjectMapper mapper = new ObjectMapper();
        PolymorphicTypeValidator polymorphicTypeValidator =  BasicPolymorphicTypeValidator.builder()
                .allowIfSubType("com.kze.test")
                .allowIfSubType("java")
                .build();
        mapper.activateDefaultTypingAsProperty(polymorphicTypeValidator, EVERYTHING, "type");
        mapper.registerModule(new JavaTimeModule());

[lenrok258]

There is ${rootDir}/release-notes to check what's changed from 2.15.4 -> later version. Could you try out with 2.16.0 as well? It would help track down which version

Works fine with 2.16.0.

I've checked 2.17.3 and it crashes exactly as the 2.17.1.

[cowtowncoder]

While it probably makes no difference, testing should try the latest patch -- 2.18.2 -- to eliminate possibility of the issue being fixed.

I don't think any change in-between versions would change behavior purposefully, but this is pretty complicated case so probably a side-effect of a fix.

Now: difference between:

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]

and

[TEST_ENUM_VALUE]

is that latter has no (polymorphic) type id, just EnumSet value; and former includes type id using so-called "Wrapper array"; that is, JSON Array where [0] is type id and [1] is the actual value).

And deserialization expects one or the other, depending on whether polymorphic handling is indicated. Due to default typing being enabled, presumably second structure should be serialized.

It is also possible that the discrepancy between expecting inclusion (or not) of Type Id, via default Typing, differs between read- and writer-side due to Java Type Erasure -- serializing polymorphic root values, as is done here, is always riskier than serializing values referred indirectly (because in latter case full type information is available, unlike in root value case).

[lenrok258]

While it probably makes no difference, testing should try the latest patch -- 2.18.2 -- to eliminate possibility of the issue being fixed.

I've tested it with the 2.18.2 and it's still not working.

What I found out is the problematic bit is the EnumSet. If I replace it with regular Set it works perfectly fine with all the versions!
In my example attached to this ticket if you change the line

EnumSet<TestEnum> input = EnumSet.of(TEST_ENUM_VALUE);

to

Set<TestEnum> input = Set.of(TEST_ENUM_VALUE);

mapper is able to deserialize json, no exception is thrown and the output is:

["java.util.ImmutableCollections$Set12",[["com.kze.test.TestClazz$TestEnum","TEST_ENUM_VALUE"]]]
[TEST_ENUM_VALUE]

So I guess it is rather rare case as it needs to have EnumSet with enum values serialized and deserialized with default typing enabled.

Nevertheless there is a decadency between serializer (decides not to include type as a index 0) and deserializer which, in such case, requires type at the beginning of the array.

[edit]
Not having type for enum in EnumSet seems sensible as there's already an enum type as a generic type in EnumSet.

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]]

My assumption is that deserializer logic is too strict thus throwing Exception even if the type could be inferred from EnumSet

[cowtowncoder]

@lenrok258 good analysis. Yes, not super common problem but definitely something that'd be good to fix.

EnumSet is quite special since type id serialized is the only type Jackson adds generic type info on, because otherwise it is impossible to resolve actual type (must use Enum class in constructor).

[lenrok258]

@cowtowncoder
Any ideas for a workaround?
As values are in a production db I need to be able to deserialize them using newer Jackson.

Reading

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]

will always end up with an Exception (_com.fasterxml.jackson.databind.exc.MismatchedInputException: Unexpected token (VALUE_STRING), expected START_ARRAY: _).

I've tried to add @JsonTypeInfo

@JsonTypeInfo(use = JsonTypeInfo.Id.NONE)
public EnumSet<TestEnum> testEnumCollection = EnumSet.of(TEST_ENUM_VALUE);

on the EnumSet property but with no luck. Seems like deserializer doesn't care and still expects array wrapper on the enum value itself (no only on an EnumSet).

[JooHyukKim]

For urgent case, u may want tp update the varchar "j.u.EnumSet" to Set then modify code to deserialize into Set not EnumSet.
But... even better create a clone column and leave the existing data as is.

[lenrok258]

For urgent case, u may want tp update the varchar "j.u.EnumSet" to Set then modify code to deserialize into Set not EnumSet. But... even better create a clone column and leave the existing data as is.

Unformtunately as the json in db has the java.util.EnumSet<com.kze.test.TestClazz$TestEnum> as the type I'm not able to deserialize it into Set.

I think my only option is to write a migration code which takes all the values from the db and saves them again with EnumSet->Set migrated.

[lenrok258]

Just to let you know, applying @JsonTypeInfo(use = NONE) on the enum declaration instead of on EnumSet property seems to be working fine.

@JsonTypeInfo(use = NONE)
public enum TestEnum {
        TEST_ENUM_VALUE
    }

Reading

["java.util.EnumSet<com.kze.test.TestClazz$TestEnum>",["TEST_ENUM_VALUE"]

doesn't throw Exception in newer versions of Jackson so it could be treated as a good workaround for now.