gazelle plugin

[jvandew]

There was a thread on Bazel slack some weeks back asking about options for BUILD file generation with Scala. There are at least 3 options that came up there:

• stackb's gazelle plugin
• netflix's bzl-gen-build tool
• foursquare's gazelle plugin (I am a maintainer of this one)

I wanted to at least start a discussion about whether there is any appetite for upstreaming a solution for this... I am obviously biased towards the Foursquare plugin -- it is fast, battle tested in 2 of our repos on ~2M lines of Scala code, Gazelle is the general Bazel tool of choice here, and it has someone willing to put the effort into seeing it adopted (me) -- but Scala is one of the last major Bazel rulesets without an "official" solution for BUILD generation, so it would be awesome to see anything gain traction. To my knowledge there is nothing with meaningful community adoption currently.

There is maybe also an open question as to whether this discussion makes more sense here or in rules_jvm, as a Scala Gazelle plugin could in theory share logic with the existing Java plugin for, e.g., handling Maven jars. But I figured here is as good a place as any to start 😄

[WojciechMazur]

I think that having a single, offical gazelle plugin would be a good direction. I'd be happy to help with any initiatives related to this.
I was not aware of the other Scala gazelle extensions, thank you! I'll test out the other 2 soon.

Lately I was adapting the stackb plugin, but it seems to be hacky and non idiomatic regarding the Gazelle, also it lacked generation of new rules and seems to be mostly developed regarding the deps resolution, including some dedicated logic to resolve conflicts. It also seemed to enforce the other stackb related dependencies, eg. rules_proto instead of using the default `scala_proto_library.

Regarding placing of the gazelle I think that this repo might work the best, the common logic from rules_jvm might be extracted to library and reused if needed.
Having official rules and gazelle extension in 1 repo can also help us to enforce a good coverage of support for all Scala rule kinds.

Btw. how's the tree-sitter working for you're implementation? Does it handle well different versions of Scala, eg. Scala 2 or Scala 3? stackb solution was using embeded node.js server running (outdated) scalameta parser via Scala.js, altough it was relatively slow to start ~1s. Rewriting this http server/client parser to JVM reduced startup times to ~250ms - a one-time cost per gazelle invocation.

[jvandew]

Thank you for taking a look! Let me know if there is anything I can help with here.

Btw. how's the tree-sitter working for you're implementation? Does it handle well different versions of Scala, eg. Scala 2 or Scala 3?

We've found tree-sitter to work well... It is very much not perfect and often returns returns parsing errors, particularly with DSL style code such as scala-xml (we see tree-sitter parsing errors on roughly 10% of our source files), but it generally returns usable data even in error states. The exception is errors at the root of the syntax tree, and we fall back to a brute force regex scan in that case.

It is much faster than the compiler plugin parsing approach we used previously, though parsing is still the overall bottleneck. In our larger monorepo, parsing ~1.8M lines of Scala code across 12.7k source files takes around 50 seconds on my M2 macbook without the parsing cache enabled.

The Scala tree-sitter grammar does support Scala 3, but we are still on Scala 2 ourselves and haven't tested our plugin's parser integration on Scala 3 code unfortunately. I would guess there are probably some syntax constructs we don't yet know how to handle.

[eed3si9n]

Since I've joined tree-sitter-scala, I and others have added most of Scala 3 syntax support including SIP-level features like SIP-44 (Fewer braces) back in 2023 tree-sitter/tree-sitter-scala#128, SIP-58 Named tuples in 2024 tree-sitter/tree-sitter-scala#446, and SIP-71 "into" last week - tree-sitter/tree-sitter-scala#475. We also have open bugs as well, but let us know if you find corner cases or common parsing errors in Scala 2.x or 3.x.