Relocated kotlin-reflect error like: Built-in class `shadow.kotlin.Any` is not found

[makeevrserg]

Expected and Results

When relocating libraries, Exposed having an issue with it's reflection it seems and can't find kotlin.Any

Related environment and versions

Gradle 8.7

ShadowJar 9.0.0-beta12

Exposed 0.50.1

Kotlin 2.2.0

Java 21

OS: Linux(Debian/Arch)/MacOs

Reproduction steps

I have a whole project which reproduces this issue https://github.com/makeevrserg/ExposedGradleFail

Anything else?

No response

[Goooler]

Seems due to #843.

We can't apply the workaround like:

  relocate("", requireProjectInfo.group) {
    exclude("META-INF/**")
    exclude("kotlin/**")
    exclude("kotlinx/**")
  }

as there will be a new issue like:

  at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.visitFile(ShadowCopyAction.kt:167)
  at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.processFile(ShadowCopyAction.kt:152)
  ... 159 more
Caused by: java.lang.IllegalArgumentException: UTF8 string too large
  at org.objectweb.asm.ByteVector.encodeUtf8(ByteVector.java:307)
  at org.objectweb.asm.ByteVector.putUTF8(ByteVector.java:274)
  at org.objectweb.asm.SymbolTable.addConstantUtf8(SymbolTable.java:800)
  at org.objectweb.asm.AnnotationWriter.visit(AnnotationWriter.java:198)
  at org.objectweb.asm.AnnotationVisitor.visit(AnnotationVisitor.java:111)
  at org.objectweb.asm.commons.AnnotationRemapper.visit(AnnotationRemapper.java:113)
  at org.objectweb.asm.ClassReader.readElementValue(ClassReader.java:3083)
  at org.objectweb.asm.ClassReader.readElementValues(ClassReader.java:3012)
  at org.objectweb.asm.ClassReader.readElementValue(ClassReader.java:3186)
  at org.objectweb.asm.ClassReader.readElementValues(ClassReader.java:3006)
  at org.objectweb.asm.ClassReader.accept(ClassReader.java:609)
  at org.objectweb.asm.ClassReader.accept(ClassReader.java:425)
  at com.github.jengelman.gradle.plugins.shadow.tasks.ShadowCopyAction$StreamAction.remapClass(ShadowCopyAction.kt:199)
  ... 161 more

[Goooler]

#1539 should work for your case, please test the artifact from that branch.

[makeevrserg]

#1539 should work for your case, please test the artifact from that branch.

Well, it's better now, but now there's other problem.

tl.dr

The branch is gradleup-1539-null-constructor

The code

object UserRatingTable : LongIdTable("users_ratings", "user_rating_id") {

    init {
        val originalColumn = (UserTable.id.columnType as EntityIDColumnType<Long>).idColumn
        val delegate = (originalColumn.columnType as? AutoIncColumnType)?.delegate ?: originalColumn.columnType
        println("delegate constructors: ${delegate::class.primaryConstructor?.parameters}")
    }
}

Has different output when shadow+relocate and shadow

• shadow+relocate: delegate constructors: null
• shadow: delegate constructors: []

More description:

The code reference("user_created_report", UserTable.id) can't execute now
The problem occurs when called Table.primaryConstructor!!.parameters inside Exposed library
The primaryConstructor after shadow relocation is null. But without relocation it's not null
The val <T : Any> KClass<T>.primaryConstructor is part of kotlin.reflect.full library

Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.makeevrserg.exposedgradlefail.MainKt.main$lambda$0(Main.kt:15)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.inTopLevelTransaction$run(ThreadLocalTransactionManager.kt:355)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.inTopLevelTransaction$lambda$11(ThreadLocalTransactionManager.kt:402)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.keepAndRestoreTransactionRefAfterRun(ThreadLocalTransactionManager.kt:410)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.inTopLevelTransaction(ThreadLocalTransactionManager.kt:401)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.transaction$lambda$5(ThreadLocalTransactionManager.kt:310)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.keepAndRestoreTransactionRefAfterRun(ThreadLocalTransactionManager.kt:410)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.transaction(ThreadLocalTransactionManager.kt:259)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.transactions.ThreadLocalTransactionManagerKt.transaction(ThreadLocalTransactionManager.kt:237)
        at com.makeevrserg.exposedgradlefail.MainKt.main(Main.kt:12)
        at com.makeevrserg.exposedgradlefail.MainKt.main(Main.kt)
Caused by: java.lang.NullPointerException
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.clone(Table.kt:1686)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.clone$default(Table.kt:1685)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.cloneAsBaseType(Table.kt:1698)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.entityId(Table.kt:691)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.referenceByIdColumn(Table.kt:1232)
        at com.makeevrserg.exposedgradlefail.org.jetbrains.exposed.sql.Table.referenceByIdColumn$default(Table.kt:1223)
        at com.makeevrserg.exposedgradlefail.UserRatingTable.<clinit>(UserRatingTable.kt:9)
        ... 11 more

[Goooler]

Ooops, needs replaceAll instead of replaceFirst.

[Goooler]

I refactored the logic and tested your reproducer

The key point is

shadowJar.configure {
    // ...

--    relocationPrefix = requireProjectInfo.group
--    enableAutoRelocation = true

++    relocate("kotlin/", "example/kotlin/") {
++        exclude("kotlin/kotlin.kotlin_builtins")
++    }
}

We have to exclude relocating kotlin/kotlin.kotlin_builtins, this is why

Exception in thread "main" java.lang.AssertionError: Built-in class com.makeevrserg.exposedgradlefail.kotlin.Any is not found
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.builtins.KotlinBuiltIns$3.invoke(KotlinBuiltIns.java:93)
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.builtins.KotlinBuiltIns$3.invoke(KotlinBuiltIns.java:88)
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:578)
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunctionToNotNull.invoke(LockBasedStorageManager.java:681)
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.builtins.KotlinBuiltIns.getBuiltInClassByName(KotlinBuiltIns.java:223)
  at com.makeevrserg.exposedgradlefail.kotlin.reflect.jvm.internal.impl.builtins.KotlinBuiltIns.getAny(KotlinBuiltIns.java:228)

[makeevrserg]

Wow, yes, indeed it works

It also works on shadow-9.1.0

So the final fix here is not to use auto-relocation and exclude builtins manually. Thank you a lot, you are my personal hero!

shadowJar.configure {
    // ...

--    relocationPrefix = requireProjectInfo.group
--    enableAutoRelocation = true
++    relocate("org/jetbrains/exposed", "example/exposed") {
++        exclude("kotlin/kotlin.kotlin_builtins")
++    }
++    relocate("kotlin/", "example/kotlin/") {
++        exclude("kotlin/kotlin.kotlin_builtins")
++    }
}

[Goooler]

Yeah. I'll add the non-relocating kotlin.kotlin_builtins logic into #1695 and test more cases.

[shanshin]

Hi,
@makeevrserg, could you please describe your use-case in more detail, why was the relocation of Exposed framework necessary?

[makeevrserg]

Hi, @shanshin
There’s an issue with Minecraft plugins and Kotlin. We don’t have a "shared" Kotlin jar on the server, so every plugin shades Kotlin into its own jar.

The problem is that different plugins may use different Kotlin versions. If two plugins use different versions and don’t relocate their Kotlin dependencies, they can conflict with each other.

For example, this is known to happen with Auxilor’s plugins because he doesn't relocate kotlin inside his jars

Example of few popular plugins, which are relocating kotlin into their jars:

• LevelledMobs
• Nova