SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS should not apply to Maps with uncomparable keys

[nathanukey]

Search before asking

• I searched in the issues and found nothing similar.

Describe the bug

We have a global ObjectMapper configured in springboot used to convert objects to JSON for REST return values as well as sending objects over JMS and other places.

To have somewhat deterministic ordering of the JSON elements, we have turned on

objectMapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);

This has been working fine for years, but now one of the other teams have implemented a map of exchange rates
Map<java.util.Currency, internal.dto.ExchangeRate> exchangeRates

When the system tries to convert this to JSON, it causes an exception:

Caused by: java.lang.ClassCastException: class java.util.Currency cannot be cast to class java.lang.Comparable (java.util.Currency and java.lang.Comparable are in module java.base of loader 'bootstrap')
	at java.base/java.util.TreeMap.compare(TreeMap.java:1569)
	at java.base/java.util.TreeMap.addEntryToEmptyMap(TreeMap.java:776)
	at java.base/java.util.TreeMap.put(TreeMap.java:785)
	at java.base/java.util.TreeMap.put(TreeMap.java:534)
	at java.base/java.util.AbstractMap.putAll(AbstractMap.java:281)
	at java.base/java.util.TreeMap.putAll(TreeMap.java:326)
	at java.base/java.util.TreeMap.<init>(TreeMap.java:187)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer._orderEntries(MapSerializer.java:1182)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:754)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720)
	at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35)
	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:733)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
	... 32 common frames omitted

The solution seems to be to disable map sorting feature, but then we lose the huge value this has given everywhere else with deterministic ordering.

In this situation, would be preferable if the fallback was just not to sort a map that can't be sorted. Perhaps warrants a different feature flag, ORDER_COMPARABLE_MAP_ENTRIES_BY_KEYS?

Some way to specify global sorting comparators would be nice, but since there is always possibility a new Map would be created using keys that weren't sorted, defaulting to converting/sending message (even if not sorted) would still be preference.

That secondary ability to sort seems to already be raised here: #2162
But still feel this is separate issue/bug, ability to keep sorting feature without failing on unsortables.

Version Information

2.14

Reproduction

    private static class Entity {
        private Map<Currency, String> exampleMap = new HashMap<>();

        public Map<Currency, String> getExampleMap() {
            return exampleMap;
        }

        public void setExampleMap(Map<Currency, String> exampleMap) {
            this.exampleMap = exampleMap;
        }
    }

    @Test
    void testDeserializationCurrencyMap3() throws IOException {
        final Entity entity = new Entity();
        final Map<Currency, String> exampleMap = entity.getExampleMap();
        exampleMap.put(Currency.getInstance("GBP"), "GBP_TEXT");
        exampleMap.put(Currency.getInstance("AUD"), "AUD_TEXT");

        final ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true);

        objectMapper.writeValueAsString(entity);
    }

Expected behavior

No response

Additional context

No response

[cowtowncoder]

Yes, this is an unfortunate issue and it does feel like not-sorting non-Comparable Maps might be the way to go. This is not necessarily fool-proof, still, because static information may be incomplete wrt actual type -- specifically, I think sorting needs to be left enabled for Map<Object, X> (or more likely, non-typed Map values) since chance is intent is to enable sorting.

But until there's a fix (which probably needs to go in 2.19 for being bit risky change, potentially), there's one thing you may be able to use disable sorting for such one thing:

@JsonFormat.Feature.WRITE_SORTED_MAP_ENTRIES

see f.ex test "src/test/java/com/fasterxml/jackson/databind/struct/FormatFeatureOrderedMapTest.java"

    static class SortedKeysMap {
        // to enable
        @JsonFormat(with = JsonFormat.Feature.WRITE_SORTED_MAP_ENTRIES)
        public Map<String,Integer> values;

        // to disable
        @JsonFormat(without = JsonFormat.Feature.WRITE_SORTED_MAP_ENTRIES)
        public Map<String,Integer> values = new LinkedHashMap<>();

it might also be possible to apply it via ConfigOverrides (@JsonFormat itself is) but since there is no test it perhaps isn't implemented (not 100% sure where annotation there'd be... probably for type of enclosing class).

Anyway, on per-field basis this override could help.

[JooHyukKim]

Add a reproducible failing test via #4776. Thank you for reproduction @nathanukey!

This issue feels like general case where we make the behavior... (of simply serializing un-ordered)

• configurable SerializationFeature in Jackson 2 , but
• default in Jackson 3 (Optionally configurable)

WDYT, @cowtowncoder ?

[cowtowncoder]

@JooHyukKim Agreed: sounds like a plan -- with a feature, with defaults like you suggested.

[JooHyukKim]

Implemented in 2.19 version via #4778 , we just released 2.18.1, so not so sure about expected date.
Thank you anyways @nathanukey!

Further Actions to Take

1. ✅ Update master accordingly Resolve #4773 Change FAIL_ON_ORDER_MAP_BY_INCOMPARABLE_KEY default and JavaDoc #4781
2. ✅ JSTEP update (new addition)

[JooHyukKim]

Done!
Added default change history in JSTEP 2

[cowtowncoder]

@JooHyukKim Thank you for updating docs too -- I don't always remember to do that.