-
Notifications
You must be signed in to change notification settings - Fork 232
Add initial support for Kotlin Native toolchain and klib compilation #1351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Add initial support for Kotlin Native toolchain and klib compilation #1351
Conversation
@@ -19,6 +19,8 @@ bazel_dep(name = "rules_shell", version = "0.4.1") | |||
|
|||
bazel_dep(name = "buildifier_prebuilt", version = "8.0.3", dev_dependency = True) | |||
|
|||
bazel_dep(name = "aspect_bazel_lib", version = "2.19.4") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this to leverage copy_to_directory
to expose the Kotlin native compiler distribution as a directory in the toolchain here (as it's massive and has a ton of files)
), | ||
] | ||
|
||
kt_klib_library = rule( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the new kt_klib_library
rule that produces klibs using the kotlin/native toolchain. Potentially we could just have it as action and have an attr on the existing rules to trigger klib compilation on them as well (I'm not sure if that's better - let me know)
// within the current working directory to isolate its cache (which is unique to this worker anyway) | ||
// Ideally we disable caching though and rely only on Bazel | ||
add("-Xauto-cache-dir=${autoCacheDirectory.absolutePathString()}") | ||
add("-Xauto-cache-from=${autoCacheFromDirectory.absolutePathString()}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without this, the compiler tries to write within the cache in the external repository where konan.home
is and bazel errors out. Ideally we disable caching but there doesn't seem to be a flag for this.
val konanHome = System.getenv("KONAN_HOME") | ||
requireNotNull(konanHome) {"KONAN_HOME env var must be set!"} | ||
|
||
System.setProperty("konan.home", konanHome) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is needed for the compiler to find the native code/distribution to produce klib.
@@ -0,0 +1,98 @@ | |||
load("//src/main/starlark/core/repositories/kotlin:templates.bzl", "TEMPLATES") | |||
|
|||
def _kotlin_capabilities_impl(repository_ctx): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was copied into a separate file to allow leveraging it with kotlin_native_compiler_repository
out = "klib/cache/macos_arm64STATIC/klib_cache_marker", | ||
content = [ | ||
"Marker file intended to create the klib system cache placeholder even if we don't use it. The native compiler errors out otherwise", | ||
"See See https://github.com/JetBrains/kotlin/blob/v2.1.21/kotlin-native/backend.native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/KonanConfig.kt#L567" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The compiler on bootstrap tries to create klib/cache/<platform>STATIC
directory in the external repo and this is non-hermetic and causes Bazel to fail. To workaround it (unfortunately there doesn't appear to be a way to disable it with a flag), so we create this marker/dummy file to avoid running into the issue (happy to consider other solutions)
In #1347 (comment), it was mentioned that it's preferred to bootstrap a Kotlin native toolchain first to leverage the new IR compiler to utilize a compilation model to output klibs in a generic way that can be leveraged for JS and potentially other platforms in the future. This PR tries to add that to the toolchain and also adds a new rule
kt_klib_library
rule which is intended to use the Kotlin-native compiler to produce klibs for platform indepedent code that can be shared between various platform specific targets (think ofcommonMain
in a gradle project) and can be in the future consumed by platform specific JS/JVM (and maybe WASM targets as well).Some notes on the implementation details:
kotlin_native_compiler_repository
to bootstrap/download the native distributions for different platforms supported along with its "capabilities" repository similar to how kotlinc is setup. I'll add some more comments inline.konan.home
or the native compiler files as a TreeArtifact/directory to avoid passing a massive number of files during compilation.produce=library
//kotlin/compiler:kotlin-native
which is supposed to alias to the native compiler jar/distribution on the execution platform.kt_klib_library
depending on another target). Add some junit test cases to assert klib has some entries we expect.A simple example shown below:
TODO: