IndexOutOfBoundsException when serializing concrete class, unable to debug the error

[octaviospain]

I want to create a class that can be serialized, like this

@Serializable
abstract class JsonFileRepository<E : QueryEntity>(var jsonFilepath: Path): InMemoryRepositoryBase<E>() {

     override fun add(entity: E): Boolean {
        val result = super.add(entity)

        if (result) {
            Json.encodeToString(serializer(queryEntitySerializer()), this).also { json ->
                jsonFilepath.toFile().writeText(json)
            }
        }
        return result
    }

    abstract fun queryEntitySerializer(): KSerializer<E>
}

therefore I have an implementation of it

@Serializable
class PersonJsonRepository : JsonFileRepository<Person> {

    constructor(jsonFilepath: Path) : super(jsonFilepath)

    override fun queryEntitySerializer(): KSerializer<Person> = Person.serializer()
}

where Person is

@Serializable
data class Person(override val id: Int, val name: String? = null, val money: Long? = null) : QueryEntity {
    override val uniqueId: String = "$id-$name-$money"
}

and QueryEntity is an interface irrelevant (I guess) for now.

In order to test that the PersonJsonRepository serializes itself when adding a Person element, I'm using this test written with Kotest

class JsonFileRepositoryTest : StringSpec({

    "Repository saves and loads entities" {
        val jsonFileRepository = tempfile("json-repository-test", ".json").also { it.deleteOnExit() }
        val repository = PersonJsonRepository(jsonFileRepository.toPath())
        val person = Person(1, "John Doe", 123456789L)
        repository.add(person)

        jsonFileRepository.readText() shouldBe """
            {"entitiesById":
                {   "${person.id}":
                    {   "id": ${person.id},
                        "name": "${person.name}",
                        "money": ${person.money}
                    }
                }
            }"""
    }
})

And the exception I'm getting in IntelliJ is this...

java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
	at java.base/java.util.Objects.checkIndex(Objects.java:372)
	at java.base/java.util.ArrayList.get(ArrayList.java:459)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$formEncodeDecodePropertyCall$1.invoke(SerializerIrGenerator.kt:295)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$formEncodeDecodePropertyCall$1.invoke(SerializerIrGenerator.kt:288)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance(GeneratorHelpers.kt:935)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.serializerInstance(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance$instantiate(GeneratorHelpers.kt:941)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance(GeneratorHelpers.kt:1107)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.serializerInstance(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.formEncodeDecodePropertyCall(GeneratorHelpers.kt:1381)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.formEncodeDecodePropertyCall(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.formEncodeDecodePropertyCall(SerializerIrGenerator.kt:288)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$generateLoad$1.invoke(SerializerIrGenerator.kt:383)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$generateLoad$1.invoke(SerializerIrGenerator.kt:324)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.contributeFunction(GeneratorHelpers.kt:75)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.contributeFunction(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.contributeFunction$default(GeneratorHelpers.kt:69)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.generateLoad(SerializerIrGenerator.kt:324)
	at org.jetbrains.kotlinx.serialization.compiler.backend.common.SerializerCodegen.generateLoadIfNeeded(SerializerCodegen.kt:116)
	at org.jetbrains.kotlinx.serialization.compiler.backend.common.SerializerCodegen.generate(SerializerCodegen.kt:38)
...
...

As you see, the exception doesn't tell me anything about where is the bug here. If there is a bug, it does not throw a custom exception in order to fix the mistake. If there is a mistake made using the library and no bug, do you know what am I missing?

Environment

• Kotlin version: 1.7.20
• Library version: 1.4.1
• Kotlin platforms: JVM
• Gradle version: 7.5
• IDE version (if bug is related to the IDE): IntelliJ IDEA 2022.2.4, Build #IU-222.4459.24
• JVM 16.0.1.fx by Liberica

[sandwwraith]

I had to edit your code a bit to get it to run without all definitions, and this test works fine for me on Kotlin 1.8.10 + serialization 1.5.0:

interface QueryEntity {
    val id: Int

    val uniqueId: String
}

abstract class InMemoryRepositoryBase<E> {
    open fun add(entity: E): Boolean { return true }
}

@Serializable
sealed class JsonFileRepository<E>(var jsonFilepath: String): InMemoryRepositoryBase<E>() {

    override fun add(entity: E): Boolean {
        val result = super.add(entity)

        if (result) {
            Json.encodeToString(serializer(queryEntitySerializer()), this).also { json ->
                println(json)
            }
        }
        return result
    }

    abstract fun queryEntitySerializer(): KSerializer<E>
}

@Serializable
class PersonJsonRepository : JsonFileRepository<Person> {

    constructor(jsonFilepath: String) : super(jsonFilepath)

    override fun queryEntitySerializer(): KSerializer<Person> = Person.serializer()
}

@Serializable
data class Person(override val id: Int, val name: String? = null, val money: Long? = null) : QueryEntity {
    override val uniqueId: String = "$id-$name-$money"
}

class JsonFileRepositoryTest {

    @Test
    fun test() {
        val repository = PersonJsonRepository("json file path")
        val person = Person(1, "John Doe", 123456789L)
        repository.add(person)
    }
}

Maybe the problem is somewhere in the definition of InMemoryRepositoryBase, as I had to change JsonFileRepository to sealed class to be able to serialize it polymorphically. Or it is already fixed in 1.8.0.

[octaviospain]

I didn't make the JsonFileRepository<E> sealed on purpose, I want the users of my library to extend from JsonFileRepository<E>, and providing the serializer for the type E, serialize the implementation of the class itself.
Is this not possible without making the class sealed?

[sandwwraith]

It's possible, you just need to use the correct SerializersModule {} in json setup to achieve polymorphism (https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md#registered-subclasses)

[octaviospain]

I'm trying to do that but I get the same exception with the following code. Note that the class hierarchy is more complex than the one in the linked wiki, since PersonJsonRepository is subclass from JsonFileRepository, which is subclass of InMemoryRepositoryBase, which implements Repository interface.

It is not specified in the wiki how to set up the SerializersModule when I have this hierarchy and I didn't manage to figure out how to do it, and as mentioned the exception thrown is not clear at all.

interface Repository<E : QueryEntity> {
    fun add(entity: E): Boolean
    ...
}

@Serializable
abstract class InMemoryRepositoryBase<E : QueryEntity> : Repository<E> {

    override fun add(entity: E): Boolean {
    ...
    }
    ...
}

@Serializable
abstract class JsonFileRepository<E : QueryEntity>(private var jsonFilepath: Path) : InMemoryRepositoryBase<E>() {

    abstract val repositorySerializersModule: SerializersModule

    abstract val queryEntitySerializer: KSerializer<E>

    @Transient
    private val json = Json { serializersModule = repositorySerializersModule }

    override fun add(entity: E): Boolean {
        val result = super.add(entity)
        if (result) {
            json.encodeToString(serializer(queryEntitySerializer), this).also { json ->
                jsonFilepath.toFile().writeText(json)
            }
        }
        return result
    }
}

@Serializable
class PersonJsonRepository : JsonFileRepository<Person> {

    // How to use the SerializersModule here? This does not work
    @Transient override val repositorySerializersModule = SerializersModule {
        polymorphic(InMemoryRepositoryBase::class) {
            subclass(JsonFileRepository::class)
            subclass(PersonJsonRepository::class)
        }
    }

    @Transient override val queryEntitySerializer = Person.serializer()

    constructor(jsonFilepath: Path) : super(jsonFilepath)
}


@Serializable
data class Person(
    override val id: Int,
    val name: String? = null,
    val money: Long? = null,
) : QueryEntity {
    override val uniqueId: String = "$id-$name-$money"
}

[sandwwraith]

There's section on registering multiple superclasses: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md#registering-multiple-superclasses

Because JsonFileRepository and InMemoryRepositoryBase are abstract classes, they should be scope classes, and PersonRepository should be registered as subclass in both of them:

SerializersModule {
        polymorphic(InMemoryRepositoryBase::class) {
            subclass(PersonJsonRepository.serializer())
        }
        polymorphic(JsonFileRepository::class) {
            subclass(PersonJsonRepository.serializer())
        }
    }

[sandwwraith]

Also, you need to move repositorySerializersModule out of PersonJsonRepository OR make private val json lazy because initialization of subclasses is done after superclasses and you'll get a NPE.

[octaviospain]

Thank you for the clarification but using the SerializersModule as you showed and making json lazy didn't work and throws me similar exception also:

java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
	at java.base/java.util.Objects.checkIndex(Objects.java:359)
	at java.base/java.util.ArrayList.get(ArrayList.java:427)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$formEncodeDecodePropertyCall$1.invoke(SerializerIrGenerator.kt:295)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$formEncodeDecodePropertyCall$1.invoke(SerializerIrGenerator.kt:288)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance(GeneratorHelpers.kt:935)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.serializerInstance(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance$instantiate(GeneratorHelpers.kt:941)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.serializerInstance(GeneratorHelpers.kt:1107)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.serializerInstance(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.formEncodeDecodePropertyCall(GeneratorHelpers.kt:1381)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.formEncodeDecodePropertyCall(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.formEncodeDecodePropertyCall(SerializerIrGenerator.kt:288)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$generateLoad$1.invoke(SerializerIrGenerator.kt:383)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$generateLoad$1.invoke(SerializerIrGenerator.kt:324)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.contributeFunction(GeneratorHelpers.kt:75)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.contributeFunction(SerializerIrGenerator.kt:47)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.IrBuilderExtension$DefaultImpls.contributeFunction$default(GeneratorHelpers.kt:69)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator.generateLoad(SerializerIrGenerator.kt:324)
	at org.jetbrains.kotlinx.serialization.compiler.backend.common.SerializerCodegen.generateLoadIfNeeded(SerializerCodegen.kt:116)
	at org.jetbrains.kotlinx.serialization.compiler.backend.common.SerializerCodegen.generate(SerializerCodegen.kt:38)
	at org.jetbrains.kotlinx.serialization.compiler.backend.ir.SerializerIrGenerator$Companion.generate(SerializerIrGenerator.kt:573)
	at org.jetbrains.kotlinx.serialization.compiler.extensions.SerializerClassLowering.lower(SerializationLoweringExtension.kt:60)
	at org.jetbrains.kotlinx.serialization.compiler.extensions.SerializationLoweringExtensionKt$runOnFileInOrder$1.visitClass(SerializationLoweringExtension.kt:36)
	at org.jetbrains.kotlin.ir.visitors.IrElementVisitorVoid$DefaultImpls.visitClass(IrElementVisitorVoid.kt:111)
	at org.jetbrains.kotlinx.serialization.compiler.extensions.SerializationLoweringExtensionKt$runOnFileInOrder$1.visitClass(SerializationLoweringExtension.kt:30)
	at org.jetbrains.kotlinx.serialization.compiler.extensions.SerializationLoweringExtensionKt$runOnFileInOrder$1.visitClass(SerializationLoweringExtension.kt:30)

could you please test it locally? I also tried adding

polymorphic(Any::class) { 
    subclass(PersonJsonRepository.serializer())
}

as in the wiki section you linked, but same result

[sandwwraith]

I've tested it locally, and it worked. Can you check with Kotlin 1.8.10, please?

[octaviospain]

Thank you, it works with Kotlin 1.8.10 and some refactors, because Path is not serializable, so I replaced it by File and used a custom serializer. The problem I'm facing now as you might have guessed by the example, is when using a structure to store entities in the InMemoryRepository class. I added a map entitiesById in the constructor, but I'm missing something, because this code does not compile:

interface QueryEntity {
    val id: Int
}

interface Repository<E : QueryEntity> {
    fun add(entity: E): Boolean
}

@Serializable
abstract class InMemoryRepositoryBase<E : QueryEntity>(@Required private val entitiesById: MutableMap<Int, E> = mutableMapOf()) :
    Repository<E> {

    override fun add(entity: E): Boolean {
        if (!entitiesById.containsKey(entity.id)) {
            entitiesById[entity.id] = entity
            return true
        }
        return false
    }
}

@Serializable
abstract class JsonFileRepository<E : QueryEntity>(@Contextual private var jsonFile: File) : InMemoryRepositoryBase<E>() {

    @OptIn(ExperimentalSerializationApi::class)
    @Serializer(forClass = File::class)
    object FileSerializer : KSerializer<File> {
        override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("File", PrimitiveKind.STRING)

        override fun serialize(encoder: Encoder, value: File) {
            encoder.encodeString(value.absolutePath)
        }

        override fun deserialize(decoder: Decoder): File {
            return File(decoder.decodeString())
        }
    }

    @Transient
    val repositorySerializersModule = SerializersModule {
        polymorphic(InMemoryRepositoryBase::class) {
            subclass(PersonJsonRepository.serializer())
        }
        polymorphic(JsonFileRepository::class) {
            subclass(PersonJsonRepository.serializer())
        }
        contextual(FileSerializer)
    }

    private val json by lazy { Json { serializersModule = repositorySerializersModule } }

    abstract val queryEntitySerializer: KSerializer<E>

    override fun add(entity: E): Boolean {
        val result = super.add(entity)
        if (result) {
            json.encodeToString(serializer(queryEntitySerializer), this).also { json ->
                jsonFile.writeText(json)
            }
        }
        return result
    }
}

@Serializable
class PersonJsonRepository : JsonFileRepository<Person> {

    @Transient
    override val queryEntitySerializer = Person.serializer()

    constructor(jsonFile: File) : super(jsonFile)
}


@Serializable
data class Person(
    override val id: Int,
    val name: String? = null,
    val money: Long? = null,
) : QueryEntity

I cannot make this work, despite reading carefully the properties requirements section.
Also, I want to add a parameter value to JsonFileRepository and PersonJsonRepository that gives a MutableMap<Int, E> and MutableMap<Int, Person> respectively in order to pass it to the constructor of the InMemoryRepository. And I can't also make that work. There is no mention in the wiki on how to workaround constructor parameters that are not attributes. One way I thought about is to declare is as val but @Transient, meaning that in memory I would have a duplicated MutableMap instance just for the sake of making the library happy.

Thank you for your time.

[sandwwraith]

I've found minimized example that triggers OutOfBoundsError:

@Serializable
abstract class InMemoryRepositoryBase<E : QueryEntity>(@Required private val entitiesById: MutableMap<Int, E> = mutableMapOf())

@Serializable
abstract class JsonFileRepository<E : QueryEntity>() : InMemoryRepositoryBase<E>() {

    abstract val queryEntitySerializer: KSerializer<E>
}

@Serializable
class PersonJsonRepository : JsonFileRepository<Person>() {

    @Transient
    override val queryEntitySerializer = Person.serializer()
}

UPD: jsonFile is not the cause of an issue.

I think you can workaround it by making entitiesById abstract.

[sandwwraith]

It's also possible to make entitiesById internal instead of private. This also solves the issue.