Inlining code causes an java.lang.IndexOutOfBoundsException for Scala 2.13 #13097
Comments
|
Managed to reproduce with sbt too. Here is my project: build.sbt: Example.scala: The two files in Also uploaded these files to Google Drive. Repro (using sbt only): Produces: |
|
If I change my sbt repro to use Scala 2.12.20, it does not fail compiling: |
|
I wonder if it's related to #12612 — wdyt @lrytz ? @gergelyfabian do those JARs exist on the internet, can I grab them with |
|
I am not sure whether you can grab them from somewhere, but it's easy to
generate them. You just need `bazelisk` installed (
https://github.com/bazelbuild/bazelisk), and then run the `bazel build`
command as I wrote in the repro steps. That will generate the jars, and I
added the path to their location.
Btw. I did this on Linux.
…
Message ID: ***@***.***>
|
To be honest, this is a level of extra effort that compiler maintainers aren't typically willing to go through. You might get lucky with someone else, but I'm not personally willing. The ideal actionable compiler bug report doesn't involve any external dependencies whatsoever. Having an external dependency of any kind greatly decreases the likelihood of an investigation occurring. But if a report must involve an external dependency, that dependency ought to be readily available. Aren't the JARs on Maven Central somewhere...? |
|
In the Bazel world everything is generated on the user's side (that's the reason why protobuf is also rebuilt - to make things reproducible). But, I'll soon send a link to the jar files uploaded to an url (from what I built locally). |
And can we be confident that the resulting JARs aren't somehow corrupt...? We seek to reduce or eliminate external dependencies in bug reports not only to reduce the effort level for maintainers, but also to narrow down the possible root causes of whatever is going wrong. |
Is that process intended to produce something that's equivalent to something on Maven Central? |
Yes, and if you check the protobuf tar.gz, it includes a Bazel BUILD file that is used to generate the jar file, and is also used to push jars in further targets (by protobuf's CI I suppose) to Maven Central. However, there are different versions of those jars, and I'm not sure it will be exactly the same jar that would end up as a dependency when I use it in my Bazel project. |
I can never be sure, as this is a bug that happens in the interaction of Bazel with Scala (and there is rules_scala in the middle too). Obviously there can be an issue on Bazel or rules_scala side, but that would mean that either Bazel builds a wrong jar (for protobuf) or that rules_scala provides a wrong jar to the scalac compiler. I'd doubt that though as for Scala 2.12 there is no issue, only with Scala 2.13 the problem appears. And the same thing is with my sbt repro. |
|
Uploaded the two jars:
|
|
With curl: |
|
Here is the updated repro: |
|
There was another issue with ASM where @lrytz asked them to take a fix; I don't have a link. That's to say, it's not inconceivable that a strange build product interacts badly with ASM facilities. |
|
I can confirm this is reproducible for me on JDK 21 with: Example.scala //> using scala 2.13.16
//> using jar libcore-hjar.jar liblite_runtime_only-hjar.jar
//> using options -opt-inline-from:**
import com.google.protobuf.CodedOutputStream
object Example {
def repro(): Unit = {
val result = new Array[Byte](0)
CodedOutputStream.newInstance(result)
}
}Justfile |
|
The modern option is I have tried to get this change in the project build: Maybe that just needed a re-starr at some point? |
|
I thought I might see if this had regressed in some 2.13.x version, but it was broken as early as 2.13.11 and I can't test 2.13.10 or earlier because the JARs are JDK 21+ only. |
I could bake the jars for some earlier JDK version for you. What would you exactly need? |
|
Ideally 8, but I'm not sure at the moment if it's a good use of your time. note that |
|
@gergelyfabian I haven't given up on wanting a reproduction that doesn't involve any Bazel-built JARs. you wrote:
and what version is that? is the problem reproducible if we swap in the Maven Central JAR for that version? |
|
Files for Java 8 (hopefully): Generated from a |
|
There is no reason I included both. I'm quite new to debugging Scala compiler issues, so I was just trying to create a better repro. The protobuf version is https://github.com/protocolbuffers/protobuf/archive/refs/tags/v21.7.tar.gz, but I tried with e.g. v3.21.10, and it also failed. If I swap in the Maven Central JAR for e.g. 3.21.10 or 3.25.5, I could not reproduce it (I tried at least once for the exact same protobuf version, Maven Central JAR vs. Bazel JAR). The issue seems to be only reproducible if I use the protobuf JAR built in Bazel, and I try my example Scala code with Scala 2.13.16 (not Scala 2.12.20). Please note, that these repro jars seem to be "hjars", according to AI that would be:
|
|
Thanks, the Java 8 JAR enabled me to determine that the problem is reproducible for me in Scala 2.13.9 but not in 2.13.8.
Okay, that marks the endpoint past which I'm not willing to investigate. (It's possible someone else will be more willing...)
Seems like inlining from such a JAR cannot possibly be expected to work...? In which case the bug, if any, might simply be that we don't give a nicer error message? |
(Though that still leaves one wondering why it didn't crash and burn in earlier Scala versions...) |
|
Another thing that makes me wonder is that originally we were inlining I simply inlined everything for the simplest repro. |
That does sound like a worthwhile experiment. |
|
Thanks all for the details. This is definitely due to the hjars, the inliner implementation expects non-abstract methods to have instructions. We can add a better error message and skip inlining in this case, that should be easy to add. |
|
Did some further investigation. I use a It would be great, if the Scala compiler would just ignore classes in hjars (even if the pattern matches them), but still inline those classes that match the pattern and are in normal jars. In the meantime I'm still trying to find out what's up with these Java dependencies and hjars. |
That's what should happen after scala/scala#11016 |
https://bazelbuild.slack.com/archives/CA31HN1T3/p1741940032376929 |
|
Related to bazelbuild/bazel#3528 as well. |
|
Opened an issue in rules_scala (bazel-contrib/rules_scala#1717), where we can discuss the Bazel side of things (including possible workarounds), especially why and how hjars are used as dependencies, and how that relates to inlining. Thanks all for your help! |
Uh oh!
There was an error while loading. Please reload this page.
After upgrading my code from Scala 2.12 to 2.13 I found a bug.
If I inline some code that uses a certain protobuf version I get the following exception from the compiler:
Scala version: 2.13.16 (could not reproduce with Scala 2.12.20).
Steps to reproduce:
Example code:
I use the following scalacopt:
-opt-inline-from:**.Example Bazel config:
This is a Bazel project, not an sbt one, as I wasn't able to reproduce this issue with sbt.
Related to that, in this example code, if I replace the dependency I use from
@com_google_protobuf//:protobuf_java(de-facto built-in Bazel one) to@maven//:com_google_protobuf_protobuf_java(one generated by rules_jvm_external), then I cannot reproduce the bug any more, so it seems to be related to the protobuf version Bazel uses.That would also explain why I cannot reproduce with sbt, as in sbt I suppose I get the same protobuf-java as with rules_jvm_external.
In any case, I wouldn't expect the Scala compiler to throw such an exception.
The text was updated successfully, but these errors were encountered: