Skip to content

Inlining code in Scala targets #1717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gergelyfabian opened this issue Mar 14, 2025 · 9 comments
Open

Inlining code in Scala targets #1717

gergelyfabian opened this issue Mar 14, 2025 · 9 comments

Comments

@gergelyfabian
Copy link
Contributor

gergelyfabian commented Mar 14, 2025

With Scala 2.13.16 if you try inlining code that has been added to your classpath as an ijar/hjar it throws an exception:

error: java.lang.IndexOutOfBoundsException
	at scala.tools.asm.tree.InsnList.get(InsnList.java:94)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.computeMaxLocalsMaxStack(BackendUtils.scala:728)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.maxLocals(BackendUtils.scala:636)
	at scala.tools.nsc.backend.jvm.opt.Inliner.inlineCallsite(Inliner.scala:856)
	at scala.tools.nsc.backend.jvm.opt.Inliner.$anonfun$runInliner$10(Inliner.scala:352)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:905)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInliner(Inliner.scala:378)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInlinerAndClosureOptimizer(Inliner.scala:281)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:156)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:98)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1564)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1548)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1540)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1674)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:48)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:30)
	at scala.tools.nsc.Driver.process(Driver.scala:68)
	at io.bazel.rulesscala.scalac.ScalacInvoker.invokeCompiler(ScalacInvoker.java:24)
	at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:272)
	at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:88)
	at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
	at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
	at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:48)
java.lang.IndexOutOfBoundsException
	at scala.tools.asm.tree.InsnList.get(InsnList.java:94)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.computeMaxLocalsMaxStack(BackendUtils.scala:728)
	at scala.tools.nsc.backend.jvm.analysis.BackendUtils$.maxLocals(BackendUtils.scala:636)
	at scala.tools.nsc.backend.jvm.opt.Inliner.inlineCallsite(Inliner.scala:856)
	at scala.tools.nsc.backend.jvm.opt.Inliner.$anonfun$runInliner$10(Inliner.scala:352)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
	at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:905)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInliner(Inliner.scala:378)
	at scala.tools.nsc.backend.jvm.opt.Inliner.runInlinerAndClosureOptimizer(Inliner.scala:281)
	at scala.tools.nsc.backend.jvm.PostProcessor.runGlobalOptimizations(PostProcessor.scala:156)
	at scala.tools.nsc.backend.jvm.GeneratedClassHandler$GlobalOptimisingGeneratedClassHandler.complete(GeneratedClassHandler.scala:98)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.$anonfun$run$1(GenBCode.scala:81)
	at scala.tools.nsc.backend.jvm.GenBCode$BCodePhase.run(GenBCode.scala:78)
	at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1564)
	at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1548)
	at scala.tools.nsc.Global$Run.compileSources(Global.scala:1540)
	at scala.tools.nsc.Global$Run.compile(Global.scala:1674)
	at scala.tools.nsc.Driver.doCompile(Driver.scala:48)
	at scala.tools.nsc.MainClass.doCompile(Main.scala:30)
	at scala.tools.nsc.Driver.process(Driver.scala:68)
	at io.bazel.rulesscala.scalac.ScalacInvoker.invokeCompiler(ScalacInvoker.java:24)
	at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:272)
	at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:88)
	at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
	at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
	at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:48)

This seems to be a Scala regression since 2.13.9+ and Scala declares that it will fix it in 2.13.17.

Steps to reproduce:

git clone git@github.com:gergelyfabian/bazel-scala-example.git
cd bazel-scala-example
git checkout scala-compiler-bug
bazel build //example-maven:example-maven

Example code:

package mypackage

import com.google.protobuf.CodedOutputStream

object Maven {
  def repro(): Unit = {
    val result = new Array[Byte](0)
    CodedOutputStream.newInstance(result)
  }
}

Using the following scalacopt: -opt-inline-from:**.


Once Scala fixes this bug, it will stop inlining code from signature jar files, but you'll get a warning:

$> scq T.scala -cp libcore-hjar.jar:liblite_runtime_only-hjar.jar '-opt:inline:**' -Wopt:_
T.scala:6: warning: com/google/protobuf/CodedOutputStream::newInstance([B)Lcom/google/protobuf/CodedOutputStream; could not be inlined:
Method com/google/protobuf/CodedOutputStream::newInstance([B)Lcom/google/protobuf/CodedOutputStream; cannot be inlined because it does not have any instructions, even though it is not abstract. The class may come from a signature jar file (such as a Bazel 'hjar').
    CodedOutputStream.newInstance(result)
                                 ^
1 warning

This behavior is caused by Bazel using hjars/ijars for dependencies.


Related Bazel Slack topics:

@gergelyfabian
Copy link
Contributor Author

Scala issue: scala/bug#13097
Scala PR: scala/scala#11016

@gergelyfabian
Copy link
Contributor Author

gergelyfabian commented Mar 14, 2025

Bazel will use ijars for dependencies in your Scala targets for:

  • java_library targets
  • scala_library targets (when you don't use macros)
  • and pretty much all other stuff

Ijars won't be used for:

  • external dependencies pulled in with rules_jvm_external (due to some Kotlin incompatibilities)
  • scala_macro_library targets
  • jars that you have imported with scala_import
  • if you enable --nouse_ijars --nojava_header_compilation flags

It happens to be the case that rules_scala already has support for turning off ijars, but it's main purpose is to help with Scala macros (as you need to have the implementation also on the classpath when compiling with macros).

It seems that inlining code is very similar to using macros.

If you want to inline code:

  • External dependencies pulled in with rules_jvm_external are fine, for now
  • Use scala_macro_library for your in-repo dependencies, that you'd like to inline
  • Wrap all java_library targets that you may want to inline with a scala_import, those java_library targets can be autogenerated e.g. for protobuf code
  • If nothing else works, you can try disabling ijars/hjars altogether: --nouse_ijars --nojava_header_compilation

Before Scala 2.13.* is fixed, these steps are necessary to make your code compile (when using inlining), after 2.13.17, you need this to ensure Scala will be able to inline that code and won't emit warnings.

@gergelyfabian
Copy link
Contributor Author

Just for clarity, the inline instruction that my project uses is -opt-inline-from:com.mycompany.**,scalaz.**,scala.**, and then some of the com.mycompany classes are compiled by java_library targets (others are either rules_jvm_external-generated dependencies or scala_macro_library targets, those are OK).

@gergelyfabian
Copy link
Contributor Author

The Scala bugfix was merged, should be in Scala 2.13.17. This means that Scala compiler won't throw an exception starting from then, but inlining code from hjar/ijar dependencies will be simply ignored.

@SethTisue
Copy link

Note that you can test it in a nightly if you like (https://docs.scala-lang.org/overviews/core/nightlies.html)

The current nightly is 2.13.17-bin-f5faa03

@gergelyfabian
Copy link
Contributor Author

Note that you can test it in a nightly if you like (https://docs.scala-lang.org/overviews/core/nightlies.html)

The current nightly is 2.13.17-bin-f5faa03

Thanks @SethTisue, greatly appreciated!

@gergelyfabian
Copy link
Contributor Author

Thanks @SethTisue, I can confirm that with 2.13.17-bin-f5faa03 the error is gone. I just need to document how one can use a "nightly" Scala version with rules_scala.

@gergelyfabian
Copy link
Contributor Author

I haven't seen a warning though, as compiling this code.

@gergelyfabian
Copy link
Contributor Author

gergelyfabian commented Mar 18, 2025

How one can use a "nightly" Scala version... Patch file for rules_scala itself.

diff --git a/scala/scala_cross_version.bzl b/scala/scala_cross_version.bzl
index f7d8980..42188f2 100644
--- a/scala/scala_cross_version.bzl
+++ b/scala/scala_cross_version.bzl
@@ -21,6 +21,7 @@ def default_maven_server_urls():
         "https://maven-central.storage-download.googleapis.com/maven2",
         "https://mirror.bazel.build/repo1.maven.org/maven2",
         "https://jcenter.bintray.com",
+        "https://scala-ci.typesafe.com/artifactory/scala-integration",
     ]
 
 def extract_major_version(scala_version):
@@ -28,7 +29,7 @@ def extract_major_version(scala_version):
     return scala_version[:scala_version.find(".", 2)]
 
 def extract_minor_version(scala_version):
-    return scala_version.split(".")[2]
+    return scala_version.split(".")[2].split("-")[0]
 
 def extract_major_version_underscore(scala_version):
     """Return major Scala version with underscore given a full version,
diff --git a/scala/scala_cross_version_select.bzl b/scala/scala_cross_version_select.bzl
index 615b6a7..4695ff7 100644
--- a/scala/scala_cross_version_select.bzl
+++ b/scala/scala_cross_version_select.bzl
@@ -1,5 +1,5 @@
-load(":scala_cross_version.bzl", "version_suffix")
 load("@rules_scala_config//:config.bzl", "SCALA_VERSIONS")
+load(":scala_cross_version.bzl", "version_suffix")
 
 def select_for_scala_version(default = [], **kwargs):
     """
@@ -61,7 +61,7 @@ def _matches_for_version(scala_version, kwargs, default_value):
 def _match_one_arg(scala_version, matcher_scala_version, compare):
     # Some rudimentary version parsing to allow a lexicographical compare later.
     # Works for versions containing numbers only.
-    scala_version = tuple([int(x) for x in scala_version.split(".")])
+    scala_version = tuple([int(x.split("-")[0]) for x in scala_version.split(".")])
     matcher_scala_version = tuple([int(x) for x in matcher_scala_version.split("_")])
 
     # Compare only a part of version – to allow wildcarding.
diff --git a/third_party/repositories/scala_2_13.bzl b/third_party/repositories/scala_2_13.bzl
index c0e7b0a..4952cdf 100644
--- a/third_party/repositories/scala_2_13.bzl
+++ b/third_party/repositories/scala_2_13.bzl
@@ -3,7 +3,7 @@
 Mostly generated and updated by scripts/create_repository.py.
 """
 
-scala_version = "2.13.17"
+scala_version = "2.13.17-bin-f5faa03"
 
 artifacts = {
     "com_github_jnr_jffi_native": {
@@ -184,8 +184,8 @@ artifacts = {
         ],
     },
     "io_bazel_rules_scala_scala_compiler": {
-        "artifact": "org.scala-lang:scala-compiler:2.13.16",
-        "sha256": "f59982714591e321ba9c087af2c8666e2f5fb92b11a0cef72c2c5e9b342152d3",
+        "artifact": "org.scala-lang:scala-compiler:2.13.17-bin-f5faa03",
+        "sha256": "eae22a432cac9c329f2dd0212cd68f92c8579d87679c8ca1708aee3c57e2d572",
         "deps": [
             "@io_bazel_rules_scala_scala_library",
             "@io_bazel_rules_scala_scala_reflect",
@@ -194,8 +194,8 @@ artifacts = {
         ],
     },
     "io_bazel_rules_scala_scala_library": {
-        "artifact": "org.scala-lang:scala-library:2.13.16",
-        "sha256": "1ebb2b6f9e4eb4022497c19b1e1e825019c08514f962aaac197145f88ed730f1",
+        "artifact": "org.scala-lang:scala-library:2.13.17-bin-f5faa03",
+        "sha256": "d01f578049ccad2a39d12c412632ed6ae41987b6d043865aa57b4dc4fd2b8b4c",
     },
     "io_bazel_rules_scala_scala_parallel_collections": {
         "artifact": "org.scala-lang.modules:scala-parallel-collections_2.13:1.2.0",
@@ -212,8 +212,8 @@ artifacts = {
         ],
     },
     "io_bazel_rules_scala_scala_reflect": {
-        "artifact": "org.scala-lang:scala-reflect:2.13.16",
-        "sha256": "fb49ccd9cac7464486ab993cda20a3c1569d8ef26f052e897577ad2a4970fb1d",
+        "artifact": "org.scala-lang:scala-reflect:2.13.17-bin-f5faa03",
+        "sha256": "9152c89795d6341040abacae69418d49237ba5d984998feb76e371703ba9997f",
         "deps": [
             "@io_bazel_rules_scala_scala_library",
         ],
@@ -439,8 +439,8 @@ artifacts = {
         ],
     },
     "org_scala_lang_scalap": {
-        "artifact": "org.scala-lang:scalap:2.13.16",
-        "sha256": "7963c72c4c74d52278e42b0108ae8ae866d4d1c4579e20209a2f9617e6aacfca",
+        "artifact": "org.scala-lang:scalap:2.13.17-bin-f5faa03",
+        "sha256": "cc322cbd07722d203fd39f8055d1b2ac82420079768457f3f7d98387f7492fcb",
         "deps": [
             "@io_bazel_rules_scala_scala_compiler",
         ],

Apart from that, obviously, you also need to change your project to use this nightly version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants