Fix Bazel 8.2.1 WORKSPACE builds, bump dev deps #1748
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes a few problems when building under `WORKSPACE` with Bazel 8.2.1 (7.6.1 doesn't require these changes). Adds to `.bazelrc` the `--incompatible_autoload_externally=` flag as a common option for all builds, and a (disabled) line of options for `WORKSPACE` builds. Bumps these development dependency versions: - `com_google_buildifier_buildtools`: 5.1.0 => 8.2.1 - `rules_shell`: 0.4.1 => 0.5.0 --- Though `WORKSPACE` is on the way out, we should ensure that `rules_scala` remains as compatible as it can be until it's totally gone. All of these errors happened when running `./test_all` with Bazel 8.2.1 and `WORKSPACE` enabled while working on bazel-contrib#1747. The first error was the following "cycle". (I later realized it's somehow due to bazelbuild/rules_java#294 from `rules_java` 8.12.0. See the note at the very end below.) ```sh $ bazel run //tools:lint_check ERROR: Cycle caused by autoloads, failed to load .bzl file '@@bazel_features_version//:version.bzl'. Add 'bazel_features_version' to --repositories_without_autoloads or disable autoloads by setting '--incompatible_autoload_externally=' More information on bazelbuild/bazel#23043. ``` `--incompatible_autoload_externally=` fixed this problem, but also precipitated all the other errors below. - bazelbuild/bazel#23043 - https://bazel.build/reference/command-line-reference#common_options-flag--incompatible_autoload_externally Updating `com_github_bazelbuild_buildtools` to v8.2.1 fixes the next error, whereby Bazel no longer autoloaded `sh_test`. v5.1.0 was missing the required `load("@rules_shell//shell:sh_test.bzl", "sh_test")` statement, added in v8.0.3 by bazelbuild/buildtools#1332: ```sh $ bazel run //tools:lint_check ERROR: .../external/com_github_bazelbuild_buildtools/buildifier/BUILD.bazel:60:1: name 'sh_test' is not defined (did you mean 'cc_test'?) ERROR: .../external/com_github_bazelbuild_buildtools/buildifier/BUILD.bazel: no such target '@@com_github_bazelbuild_buildtools//buildifier:runner.bash.template': target 'runner.bash.template' not declared in package 'buildifier' defined by .../external/com_github_bazelbuild_buildtools/buildifier/BUILD.bazel; however, a source file of this name exists. (Perhaps add 'exports_files(["runner.bash.template"])' to buildifier/BUILD?) ERROR: /Users/mbland/src/bazel-contrib/rules_scala/tools/BUILD:19:11: every rule of type _buildifier implicitly depends upon the target '@@com_github_bazelbuild_buildtools//buildifier:runner.bash.template', but this target could not be found because of: no such target '@@com_github_bazelbuild_buildtools//buildifier:runner.bash.template': target 'runner.bash.template' not declared in package 'buildifier' defined by .../external/com_github_bazelbuild_buildtools/buildifier/BUILD.bazel; however, a source file of this name exists. (Perhaps add 'exports_files(["runner.bash.template"])' to buildifier/BUILD?) ERROR: Analysis of target '//tools:lint_check' failed; build aborted: Analysis failed ``` Upgrading to v8.2.1 and updating `//tools:lint_check` to become a `buildifier_test` also finally got rid of this next warning. This required exporting the `MODULE.bazel` file from the root package, disabling one lint warning, and accepting a couple of new lint fixes: ```txt DEBUG: .../external/+dev_deps+com_github_bazelbuild_buildtools/buildifier/internal/factory.bzl:17:10: DEPRECATION NOTICE: value 'check' for attribute 'mode' will be removed in the future. Migrate '@@//tools:lint_check' to buildifier_test. ``` Adding `rules_jvm_external` 6.7 to `//scala:latest_deps.bzl` fixes this next error, described in detail in: - protocolbuffers/protobuf#19129 (comment) ```sh $ bazel build --test_output=errors src/... test/... ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:169:21: @@com_google_protobuf//java/core:lite_mvn-lib: no such attribute 'javacopts' in 'java_library' rule ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:169:21: @@com_google_protobuf//java/core:lite_mvn-lib: no such attribute 'resources' in 'java_library' rule (did you mean 'features'?) ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:169:21: @@com_google_protobuf//java/core:lite_mvn-lib: no such attribute 'runtime_deps' in 'java_library' rule ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:287:21: @@com_google_protobuf//java/core:core_mvn-lib: no such attribute 'javacopts' in 'java_library' rule ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:287:21: @@com_google_protobuf//java/core:core_mvn-lib: no such attribute 'resources' in 'java_library' rule (did you mean 'features'?) ERROR: .../external/com_google_protobuf/java/core/BUILD.bazel:287:21: @@com_google_protobuf//java/core:core_mvn-lib: no such attribute 'runtime_deps' in 'java_library' rule ERROR: .../external/com_google_protobuf/BUILD.bazel:475:6: Target '@@com_google_protobuf//java/core:core' contains an error and its package is in error and referenced by '@@com_google_protobuf//:protobuf_java' ERROR: Analysis of target '//test/proto/custom_generator:failing_scala_proto_deps_toolchain_def' failed; build aborted: Analysis failed ``` All of the previous errors only affected `rules_scala`'s own development builds and test runs, when it is the main repository/root module. This final error is something users could possibly run into when using `--incompatible_autoload_externally=` with `rules_scala`. `jvm_import` statements in Maven dependency repos generated by `jvm_import_external` from `//scala:scala_maven_import_external.bzl` began to fail. Defining `_JAVA_IMPORT_RULE_LOAD` and using it as the default value for the `rule_load` attribute of the `_jvm_import_external` rule fixed this: ```sh $ bazel test --test_output=errors third_party/... ERROR: .../external/org_apache_commons_commons_lang_3_5_without_file/BUILD:7:12: @@org_apache_commons_commons_lang_3_5_without_file//:org_apache_commons_commons_lang_3_5_without_file: no such attribute 'jars' in 'java_import' rule ERROR: .../external/org_apache_commons_commons_lang_3_5_without_file/BUILD:7:12: @@org_apache_commons_commons_lang_3_5_without_file//:org_apache_commons_commons_lang_3_5_without_file: no such attribute 'neverlink' in 'java_import' rule ERROR: .../external/org_apache_commons_commons_lang_3_5_without_file/BUILD:14:12: @@org_apache_commons_commons_lang_3_5_without_file//:linkable_org_apache_commons_commons_lang_3_5_without_file: no such attribute 'jars' in 'java_import' rule ERROR: .../third_party/dependency_analyzer/src/test/BUILD:4:6: Target '@@org_apache_commons_commons_lang_3_5_without_file//:linkable_org_apache_commons_commons_lang_3_5_without_file' contains an error and its package is in error and referenced by '//third_party/dependency_analyzer/src/test:strict_deps_test' ERROR: Analysis of target '//third_party/dependency_analyzer/src/test:strict_deps_test' failed; build aborted: Analysis failed ``` --- And just now, I'm noticing that I'd already added the following to `.bazelrc`, where the `rules_java` release references bazelbuild/bazel#26119: ```sh # Uncomment for WORKSPACE builds for Bazel [8.0.0, 8.3.0) per: # https://github.com/bazelbuild/rules_java/releases/tag/8.12.0 #common --repositories_without_autoloads=bazel_features_version,bazel_features_globals ``` Oh well. But now we're future proof.
|
FYI, I filed protocolbuffers/protobuf#22225 for the |
simuons
approved these changes
Jul 23, 2025
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Jul 24, 2025
Bumps most dependency versions (except `rules_python` under `WORKSPACE`) to the latest available before releasing `rules_scala` v7.1.0. Also bumps the `rules_scala` version number in `MODULE.bazel` to `7.1.0`. - Go: 1.24.4 => 1.24.5 - Scalafmt: 3.9.7 => 3.9.8 - `bazel_skylib`: 1.7.1 => 1.8.1 - `gazelle`: 0.43.0 => 0.44.0 - `golang.org/x/tools`: v0.34.0 => v0.35.0 - `org.scala-sbt:util-interface`: 1.11.1 => 1.11.3 - `proto-google-common-protos`: 2.58.0 => 2.59.2 - `rules_cc`: 0.1.1 => 0.1.4 - `rules_go`: 0.55.0 => 0.55.1 - `rules_java`: 8.12.0 => 8.14.0 - `rules_jvm_external`: 6.7 => 6.8 - `rules_python`: 1.4.1 => 1.5.1 (Bzlmod only) Also: - Moves the `http_archive` instantiations of `rules_python` and `rules_shell` from individual `WORKSPACE` files to `//scala:latest_deps.bzl`. Removes `load` statements for `http_archive` throughout all `WORKSPACE` files. - Removes the `--incompatible_autoload_externally` flag from `.bazelrc`. - Fixes `_get_compiler_srcjar` from `//scala/private:macros/scala_repositories.bzl` to return empty `compiler_srcjar_objects` instead of returning a valid object. Caught by `test_compiler_srcjar_error 2.12.11` from `dt_patches/dt_patch_test.sh` when testing under `WORKSPACE`. Added `test_fail_if_compiler_srcjar_object_is_empty` to `test/shell/test_compiler_sources_integrity.sh` to catch the specific problem sooner. --- This is to ensure that we're compatible with the latest dependencies available before releasing a new minor version. I moved the `rules_python` instantiation to `//scala:latest_deps.bzl` because it was such a pain to update everywhere. `latest_deps.bzl` is essentially a development file, or a "use at your own risk" file, and the primary consumers are internal test modules. This will make future `rules_python` updates less noisy. The only `rules_shell` instantiations were in the top level `WORKSPACE` and `test_version/WORKSPACE.template`. I moved `rules_shell` into `latest_deps.bzl` to eliminate these direct instantations as well, and to remove the `http_archive` imports from those files. I'd added `--incompatible_autoload_externally` in bazel-contrib#1748 when fixing problems with `WORKSPACE` builds under Bazel 8.2.1. However, it's not really necessary, so I removed it. Updating `rules_python` from 1.4.1 to 1.5.1 produced the following error in Bazel 7.6.1 WORKSPACE builds, even with `--incompatible_autoload_externally=+@rules_python`: ```txt $ bazel build //src/... //test/... WARNING: Couldn't auto load rules or symbols, because no dependency on module/repository 'rules_python' found. This will result in a failure if there's a reference to those rules or symbols. ERROR: test/BUILD:824:10: in py_binary rule //test:py_resource_binary: Traceback (most recent call last): File ".../external/rules_python/python/private/py_binary_rule.bzl", line 24, column 30, in _py_binary_impl return py_executable_impl( File ".../external/rules_python/python/private/py_executable.bzl", line 255, column 35, in py_executable_impl py_executable_base_impl( File ".../external/rules_python/python/private/py_executable.bzl", line 1140, column 46, in py_executable_base_impl exec_result = semantics.create_executable( File ".../external/rules_python/python/private/py_executable.bzl", line 365, column 33, in _create_executable _create_stage1_bootstrap( File ".../external/rules_python/python/private/py_executable.bzl", line 809, column 27, in _create_stage1_bootstrap if runtime and runtime.supports_build_time_venv: Error: 'PyRuntimeInfo' value has no field or method 'supports_build_time_venv' Available attributes: bootstrap_template, coverage_files, coverage_tool, files, interpreter, interpreter_path, python_version, stub_shebang ERROR: test/BUILD:824:10: Analysis of target '//test:py_resource_binary' failed ERROR: Analysis of target '//test:py_resource_binary' failed; build aborted ``` This is due to bazel-contrib/rules_python#2760. In contrast, Bazel 8.3.1 builds work fine, as do Bazel 7.6.1 builds under Bzlmod. Trying `--incompatible_autoload_externally=PyRuntimeInfo` to work around the problem under Bazel 7.6.1 produced: ```txt $ bazel build //src/... //test/... FATAL: bazel crashed due to an internal error. Printing stack trace: java.lang.IllegalStateException: Symbol in 'PyRuntimeInfo' can't be removed, because it's still used by: py_binary, py_test, py_library at com.google.devtools.build.lib.packages.AutoloadSymbols.<init>(AutoloadSymbols.java:192) [...snip...] ``` And with `--incompatible_autoload_externally=@rules_python` produced: ```txt WARNING: Couldn't auto load rules or symbols, because no dependency on module/repository 'rules_python' found. This will result in a failure if there's a reference to those rules or symbols. ERROR: .../external/bazel_tools/tools/python/toolchain.bzl:25:44: name 'PyRuntimeInfo' is not defined ERROR: .../external/bazel_tools/tools/python/toolchain.bzl:33:44: name 'PyRuntimeInfo' is not defined ERROR: .../external/bazel_tools/tools/python/toolchain.bzl:64:26: name 'PyRuntimeInfo' is not defined ERROR: .../external/bazel_tools/tools/python/toolchain.bzl:72:26: name 'PyRuntimeInfo' is not defined WARNING: Target pattern parsing failed. ERROR: Skipping '//test/...': error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:21:6: at .../external/rules_python/python/py_runtime_pair.bzl:17:6: compilation of module 'tools/python/toolchain.bzl' failed ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:21:6: at .../external/rules_python/python/py_runtime_pair.bzl:17:6: compilation of module 'tools/python/toolchain.bzl' failed ``` `tools/python/toolchain.bzl` from Bazel 7.6.1 breaks because it references the builtin `PyRuntimeInfo` instead of `load`ing it from `@rules_python`. Bazel 8 replaces that file's entire implementation with a function that `fail`s with a deprecation error and doesn't use `PyRuntimeInfo`: - https://github.com/bazelbuild/bazel/blob/7.6.1/tools/python/toolchain.bzl - https://github.com/bazelbuild/bazel/blob/8.0.0/tools/python/toolchain.bzl Using `+@rules_python` or `+PyRuntimeInfo` produced the original error. Nothing I could do could avoid using the builtin `PyRuntimeInfo` under Bazel 7.6.1. Since it's only a development dependency for `WORKSPACE` builds, I decided to keep `rules_python` at 1.4.1 in `latest_deps.bzl` for now. I've filed bazel-contrib/rules_python#3119 to get it on the record.
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Jul 26, 2025
Uses the `bazel_worker_{api,java}` modules and a legacy `WORKSPACE` shim
to replace `@bazel_tools//src/main/protobuf:worker_protocol_java_proto`.
This, and bumping `rules_java` to 8.14.0, fixes breakages in the latest
`last_green` Bazel at commit 00b48b27aa216befc419e540919a1f1df4cda337.
Also:
- Removes the `--incompatible_autoload_externally` flag from `.bazelrc`.
- Fixes `_get_compiler_srcjar` to return empty `compiler_srcjar_objects`
instead of returning a valid object. `test_compiler_srcjar_error
2.12.11` from `dt_patches/dt_patch_test.sh` caught this under
`WORKSPACE`. Added `test_fail_if_compiler_srcjar_object_is_empty` to
`test/shell/test_compiler_sources_integrity.sh` to catch the specific
problem sooner.
- Fixed regex in `target_file_location` from `test_helper.sh` to work
with Bazel 6.5.0, and fixed unbound variable in `fail` message.
- Added logic to `teardown_suite` in `test_bzlmod_macros.sh` to avoid
unbound variable breakage when running under Bazel 6.5.0.
- Added logic to skip `test_check_module_bazel_template` under Bazel 6
in `test_version.sh`.
- Removed restriction to use `protobuf` v29 or lower with Bazel 6 from
`README.md`.
- Added `scala_compiler_source.*` to the regular expressions for "canonical
reproducible" messages in `dt_patches/dt_patch_test.sh`. This resolved
a test breakage when `rules_python` began to emit a "canonical
reproducible" message. (Also changed `--expunge` to
`--expunge_async`.)
---
bazelbuild/bazel#26477, which also landed in the 9.0.0-pre.20250714.1
rolling release, broke the `last_green` build:
```
$ USE_BAZEL_VERSION=9.0.0-pre.20250714.1 bazel test //src/... //test/...
ERROR: error loading package '@@bazel_tools//src/main/protobuf':
Unable to find package for
@@[unknown repo 'grpc-java' requested from @@bazel_tools]//:java_grpc_library.bzl:
The repository '@@[unknown repo 'grpc-java' requested from @@bazel_tools]'
could not be resolved:
No repository visible as '@grpc-java' from repository '@@bazel_tools'.
```
The failing target, and several others, depended upon
`@bazel_tools//src/main/protobuf:worker_protocol_java_proto`. Nothing
else depends on any other target from that package. So upon @Wyverald's
advice, I attempted to use the `bazel_worker_java` module to replace
this specific dependency:
- bazelbuild/bazel#26579 (comment)
Using just `bazel_worker_java` 0.0.5 at first produced this breakage:
```txt
$ USE_BAZEL_VERSION=last_green bazel build //src/... //test/...
ERROR: .../external/bazel_worker_api+/BUILD.bazel:3:1:
name 'proto_library' is not defined
```
This is because that version of `bazel_worker_java` still depends upon
`bazel_worker_api` 0.0.1, which didn't load `proto_library` from
`@com_google_protobuf`. Adding `bazel_worker_api` 0.0.5 directly to
`MODULE.bazel` fixed this.
The next failure was due to `rules_scala` overriding `rules_java` to be
8.12.0, whereas `last_green` now requires 8.14.0:
```txt
$ USE_BAZEL_VERSION=last_green bazel build //src/... //test/...
ERROR: src/java/io/bazel/rulesscala/jar/BUILD:3:13:
in java_library rule //src/java/io/bazel/rulesscala/jar:jar:
Traceback (most recent call last):
File ".../external/rules_java+/java/bazel/rules/bazel_java_library.bzl",
line 26, column 35, in _proxy
return bazel_java_library_rule(
File ".../external/rules_java+/java/common/rules/impl/bazel_java_library_impl.bzl",
line 68, column 43, in bazel_java_library_rule
target, base_info = basic_java_library(
File ".../external/rules_java+/java/common/rules/impl/basic_java_library_impl.bzl",
line 127, column 49, in basic_java_library
java_info, compilation_info = compile_action(
File ".../external/rules_java+/java/common/rules/impl/compile_action.bzl",
line 139, column 46, in compile_action
java_info = _compile_private_for_builtins(
File ".../external/rules_java+/java/private/java_common_internal.bzl",
line 217, column 68, in compile
get_internal_java_common().create_header_compilation_action(
Error in create_header_compilation_action:
create_header_compilation_action() missing 2 required positional
arguments: header_compilation_jar, header_compilation_direct_deps
ERROR: src/java/io/bazel/rulesscala/jar/BUILD:3:13:
Analysis of target '//src/java/io/bazel/rulesscala/jar:jar' failed
```
Bumping `rules_java` to 8.14.0 in the `single_version_override` fixed
that.
Finally, since there's no legacy `WORKSPACE` API in
bazelbuild/bazel-worker-api, I had to roll my own new
`workspace_compat()` macro. This macro creates
`@bazel_worker_{api,java}` repos with aliases to targets in
`@bazel-worker-api`.
Regarding `--incompatible_autoload_externally`, I'd added it in bazel-contrib#1748
when fixing problems with `WORKSPACE` builds under Bazel 8.2.1. However,
it's not really necessary, so I removed it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes a few problems when building under
WORKSPACEwith Bazel 8.2.1 (7.6.1 doesn't require these changes). Adds to.bazelrcthe--incompatible_autoload_externally=flag as a common option for all builds, and a (disabled) line of options forWORKSPACEbuilds.Bumps these development dependency versions:
com_google_buildifier_buildtools: 5.1.0 => 8.2.1rules_shell: 0.4.1 => 0.5.0Motivation
Though
WORKSPACEis on the way out, we should ensure thatrules_scalaremains as compatible as it can be until it's totally gone. All of these errors happened when running./test_allwith Bazel 8.2.1 andWORKSPACEenabled while working on #1747.The first error was the following "cycle". (I later realized it's somehow due to bazelbuild/rules_java#294 from
rules_java8.12.0. See the note at the very end below.)--incompatible_autoload_externally=fixed this problem, but also precipitated all the other errors below.Updating
com_github_bazelbuild_buildtoolsto v8.2.1 fixes the next error, whereby Bazel no longer autoloadedsh_test. v5.1.0 was missing the requiredload("@rules_shell//shell:sh_test.bzl", "sh_test")statement, added in v8.0.3 by bazelbuild/buildtools#1332:Upgrading to v8.2.1 and updating
//tools:lint_checkto become abuildifier_testalso finally got rid of this next warning. This required exporting theMODULE.bazelfile from the root package, disabling one lint warning, and accepting a couple of new lint fixes:Adding
rules_jvm_external6.7 to//scala:latest_deps.bzlfixes this next error, described in detail in:All of the previous errors only affected
rules_scala's own development builds and test runs, when it is the main repository/root module. This final error is something users could possibly run into when using--incompatible_autoload_externally=withrules_scala.jvm_importstatements in Maven dependency repos generated byjvm_import_externalfrom//scala:scala_maven_import_external.bzlbegan to fail. Defining_JAVA_IMPORT_RULE_LOADand using it as the default value for therule_loadattribute of the_jvm_import_externalrule fixed this:And just now, I'm noticing that I'd already added the following to
.bazelrc, where therules_javarelease references bazelbuild/bazel#26119:Oh well. But now we're future proof.