Problem with deserializing singleton object in polymorphic context with custom serializer

[nsk90]

Describe the bug
I have a Singleton object, for which I am trying to implement a KSerializer.
Serializer works fine only if it is used outside of polymorphic context.
When it is used in polymorphic context deserialization fails with unexpected exception:

 * Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
 * JSON input: {"type":{"type":"Singleton"}}
 * kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
 * JSON input: {"type":{"type":"Singleton"}}

1)Am I doing something wrong, or it is the library issue?
2) looks that I have to use api marked as internal (not working also) - buildSerialDescriptor("Singleton", StructureKind.OBJECT)

To Reproduce
Attach a code snippet or test data if possible.
I have provided 4 tests.
1 and 2 looks approximately like I expect, but they fail, 3 and 4 are workarounds that I have tried.

package com.nsk.test

import io.kotest.core.spec.style.StringSpec
import kotlinx.serialization.InternalSerializationApi
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.builtins.nullable
import kotlinx.serialization.builtins.serializer
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encodeToString
import kotlinx.serialization.encoding.*
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic

/** Requires custom serializer */
private interface PolymorphicType

/** Requires custom serializer */
private object Singleton : PolymorphicType

@Serializable
private data class MyData(val type: PolymorphicType)

/**
 * Does not work.
 * Uses [buildClassSerialDescriptor]
 */
private object NotWorkingSingletonSerializer : KSerializer<Singleton> {
    override val descriptor = buildClassSerialDescriptor("Singleton")

    override fun serialize(encoder: Encoder, value: Singleton) {
        encoder.encodeStructure(descriptor) {}
    }

    override fun deserialize(decoder: Decoder): Singleton {
        return decoder.decodeStructure(descriptor) { Singleton }
    }
}

/**
 * Does not work.
 * This looks like correct variant for me, but is uses internal api with [StructureKind.OBJECT]
 */
private object NotWorkingObjectSingletonSerializer : KSerializer<Singleton> {
    @OptIn(InternalSerializationApi::class)
    // have to use internal api
    override val descriptor = buildSerialDescriptor("Singleton", StructureKind.OBJECT)

    override fun serialize(encoder: Encoder, value: Singleton) {
        encoder.encodeStructure(descriptor) {}
    }

    override fun deserialize(decoder: Decoder): Singleton {
        return decoder.decodeStructure(descriptor) { Singleton }
    }
}

/**
 * Works fine
 * But I have to add fake field
 */
private object WorkingSerializableSerializer : KSerializer<Singleton> {
    override val descriptor = buildClassSerialDescriptor("Singleton") {
        element<Boolean>("field_to_fix_an_issue")
    }

    override fun serialize(encoder: Encoder, value: Singleton) {
        encoder.encodeStructure(descriptor) {
            encodeBooleanElement(descriptor, 0, false)
        }
    }

    override fun deserialize(decoder: Decoder): Singleton {
        return decoder.decodeStructure(descriptor) {
            while (true) {
                when (val index = decodeElementIndex(descriptor)) {
                    0 -> decodeBooleanElement(descriptor, 0) // just read it
                    CompositeDecoder.DECODE_DONE -> break
                    else -> error("Unexpected index: $index")
                }
            }
            Singleton
        }
    }
}

/**
 * Works fine, allows to omit "field_to_fix_an_issue" in json, but it is still in code.
 */
private object WorkingOptionalSerializableSerializer : KSerializer<Singleton> {
    override val descriptor = buildClassSerialDescriptor("Singleton") {
        element("field_to_fix_an_issue", Boolean.serializer().nullable.descriptor, isOptional = true)
    }

    override fun serialize(encoder: Encoder, value: Singleton) {
        encoder.encodeStructure(descriptor) {
            encodeNullableSerializableElement(descriptor, 0, Boolean.serializer().nullable, null)
        }
    }

    override fun deserialize(decoder: Decoder): Singleton {
        return decoder.decodeStructure(descriptor) {
            while (true) {
                when (val index = decodeElementIndex(descriptor)) {
                    0 -> decodeNullableSerializableElement(descriptor, 0, Boolean.serializer().nullable) // just read it
                    CompositeDecoder.DECODE_DONE -> break
                    else -> error("Unexpected index: $index")
                }
            }
            Singleton
        }
    }
}

class ObjectSerializationTest : StringSpec({
    "test1 NotWorkingSingletonSerializer - fail" {
        val jsonFormat = Json {
            serializersModule = SerializersModule {
                polymorphic(PolymorphicType::class) {
                    subclass(Singleton::class, NotWorkingSingletonSerializer)
                }
            }
        }
        val json = jsonFormat.encodeToString(MyData(Singleton))
        println(json) // {"type":{"type":"Singleton"}}
        val data = jsonFormat.decodeFromString<MyData>(json)
        /*
         * deserialization exception:
         * Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
         * JSON input: {"type":{"type":"Singleton"}}
         * kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
         * JSON input: {"type":{"type":"Singleton"}}
         */
    }

    "test2 NotWorkingObjectSingletonSerializer - fail" {
        val jsonFormat = Json {
            serializersModule = SerializersModule {
                polymorphic(PolymorphicType::class) {
                    subclass(Singleton::class, NotWorkingObjectSingletonSerializer)
                }
            }
        }
        val json = jsonFormat.encodeToString(MyData(Singleton))
        println(json) // {"type":{"type":"Singleton"}}
        val data = jsonFormat.decodeFromString<MyData>(json)
        /*
         * deserialization exception:
         * Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
         * JSON input: {"type":{"type":"Singleton"}}
         * kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 9: Expected end of the object '}', but had '"' instead at path: $.type
         * JSON input: {"type":{"type":"Singleton"}}
         */
    }

    "test3 WorkingSerializableSerializer - ok" {
        val jsonFormat = Json {
            serializersModule = SerializersModule {
                polymorphic(PolymorphicType::class) {
                    subclass(Singleton::class, WorkingSerializableSerializer)
                }
            }
        }
        val json = jsonFormat.encodeToString(MyData(Singleton))
        println(json) // {"type":{"type":"Singleton","field_to_fix_an_issue":false}}
        val data = jsonFormat.decodeFromString<MyData>(json)
    }

    "test4 WorkingOptionalSerializableSerializer - ok" {
        val jsonFormat = Json {
            serializersModule = SerializersModule {
                polymorphic(PolymorphicType::class) {
                    subclass(Singleton::class, WorkingOptionalSerializableSerializer)
                }
            }
        }
        val json = jsonFormat.encodeToString(MyData(Singleton))
        println(json) // {"type":{"type":"Singleton","field_to_fix_an_issue":false}} or {"type":{"type":"Singleton"}}
        val data = jsonFormat.decodeFromString<MyData>(json)
    }
})

Expected behavior

All 4 cases pass successfully

Environment

• Kotlin version: [2.0.20]
• Library version: [1.7.3]
• Kotlin platforms: [JVM]
• Gradle version: [7.6.1]
• IDE version -

[sandwwraith]

It seems the problem here is that code which selects deserializer by "type":"Singleton" input (here:

kotlinx.serialization/formats/json/commonMain/src/kotlinx/serialization/json/internal/StreamingJsonDecoder.kt

Line 73 in e35c28d

val type = lexer.peekLeadingMatchingValue(discriminator, configuration.isLenient)

) does not actually consume this string from input. Therefore, it has to be skipped by other means. For generated code, it happens in decodeElementIndex call which would skip type key known to be a discriminator. To workaround this in your code, you can make this call as well:

override fun deserialize(decoder: Decoder): Singleton {
      return decoder.decodeStructure(descriptor) {
          while (decodeElementIndex(descriptor) != CompositeDecoder.DECODE_DONE) {}
          Singleton
      }
  }

I'm not sure if this is an implementation-defined behavior or a bug, but for simplicity of writing custom serializers we probably would address this.

[nsk90]

Thanks, your workaround looks much better than mine, I'll try it.👍

This is worth mentioning in documentation, at least, I suppose.

There is still a question with buildSerialDescriptor, which is marked internal.