Bazel 7 compatibility updates #1619

mbland · 2024-10-04T22:43:26Z

Description

Resolves all but one Bazel 7 compatibility issue, while maintaining Bazel 6 and WORKSPACE compatibility.

The changes here are orthogonal to one another and can be broken out into separate pull requests if desired. See the list of commits and their corresponding messages for more details.

Updates the build to use Bazel 6.5.0, the last in the Bazel 6 line. The one Bazel 7 problem not addressed is the protobuf problem described below, which is resolved by #1620. With the change from that pull request, the changes from this pull request also successfully build under the latest mainline release, Bazel 7.3.2.

Motivation

I'm attempting to resolve #1482 to add Bzlmod support, based on the bzlmod branch in my fork. I'd like to ensure rules_scala builds cleanly under both Bazel 6 and Bazel 7 before beginning the Bzlmod refactoring.

With these changes, ./test_{lint,all,examples,coverage}.sh all succeed under Bazel 6.5.0. Then, when changing nothing else but updating to Bazel 7.0.0, protobuf generation fails as seen below. Applying the change from #1620 resolves the problem, while maintaining Bazel 6 compatibility.

$ bazel build //test/proto3:all

ERROR: .../rules_scala/test/proto3/BUILD:14:14:
  ProtoScalaPBRule test/proto3/generated-proto-lib_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto3:generated-proto-lib)
  bazel-out/darwin_arm64-opt-exec-ST-13d3ddad9198/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

Could not find file in descriptor database:
  bazel-out/darwin_arm64-fastbuild/bin/test/proto3/generated.proto:
  No such file or directory

java.lang.RuntimeException: Exit with code 1
        at scala.sys.package$.error(package.scala:30)
        at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
        at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
        at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
        at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
        at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Use --verbose_failures to see the command lines of failed build steps.

ERROR: .../rules_scala/test/proto3/BUILD:14:14
  scala @//test/proto3:generated-proto-lib
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target //test/proto3:generated-proto-lib)
  bazel-out/darwin_arm64-opt-exec-ST-13d3ddad9198/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

cc: @BillyAutrey @jayconrod @benjaminp @TheGrizzlyDev

This fixes the following error: ```txt Error in create_header_compilation_action: Rule 'thrift_library' in 'rules_scala/test/src/main/scala/scalarules/test/twitter_scrooge/thrift/ thrift_with_compiler_args/BUILD:3:15' must declare '@@bazel_tools//tools/jdk:toolchain_type' toolchain in order to use java_common. See bazelbuild/bazel#18970. ``` Interestingly, adding the toolchain type to `thrift_library` isn't enough; I had to add it to the twitter_scrooge aspects as well. Until I did, it produced the same error message pointing at `thrift_library`, after first reporting: ```txt ERROR: rules_scala/test/src/main/scala/scalarules/test/twitter_scrooge/ thrift/thrift_with_compiler_args/BUILD:3:15: in //twitter_scrooge:twitter_scrooge.bzl%scrooge_java_aspect aspect on thrift_library rule //test/src/main/scala/scalarules/test/twitter_scrooge/thrift/ thrift_with_compiler_args:thrift5: Traceback (most recent call last): File "rules_scala/twitter_scrooge/twitter_scrooge.bzl", line 420, column 49, in _scrooge_java_aspect_impl return _common_scrooge_aspect_implementation(target, ctx, "java", _compile_generated_java) [ ...snip... ] ```

Bazel 7 enforces that this be a proper target label, not a relative path. ```txt ERROR: WORKSPACE:91:37: fetching new_local_repository rule //external:proto_cross_repo_boundary: .../external/test/proto_cross_repo_boundary/repo/BUILD.repo is not a regular file; if you're using a relative or absolute path for `build_file` in your `new_local_repository` rule, please switch to using a label instead INFO: Repository remote_jdk8_macos instantiated at: WORKSPACE:175:18: in <toplevel> .../external/rules_java/java/repositories.bzl:120:10: in remote_jdk8_repos .../external/bazel_tools/tools/build_defs/repo/utils.bzl:240:18: in maybe .../external/rules_java/toolchains/remote_java_repository.bzl:48:17: in remote_java_repository Repository rule http_archive defined at: .../external/bazel_tools/tools/build_defs/repo/http.bzl:384:31: in <toplevel> ERROR: no such package '@@proto_cross_repo_boundary//': .../external/test/proto_cross_repo_boundary/repo/BUILD.repo is not a regular file; if you're using a relative or absolute path for `build_file` in your `new_local_repository` rule, please switch to using a label instead ERROR: test/proto_cross_repo_boundary/BUILD:3:22: //test/proto_cross_repo_boundary:sample_scala_proto depends on @@proto_cross_repo_boundary//:sample_proto in repository @@proto_cross_repo_boundary which failed to fetch. no such package '@@proto_cross_repo_boundary//': .../external/test/proto_cross_repo_boundary/repo/BUILD.repo is not a regular file; if you're using a relative or absolute path for `build_file` in your `new_local_repository` rule, please switch to using a label instead Use --verbose_failures to see the command lines of failed build steps. ERROR: Analysis of target '//test/proto_cross_repo_boundary:sample_scala_proto' failed; build aborted: Analysis failed ```

Fixes the following breakages under Bazel 7: ```txt $ bazel build //src/... ERROR: error loading package '@@bazel_tools//src/main/protobuf': cannot load '@@rules_python//python:proto.bzl': no such file ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: error loading package '@@bazel_tools//src/main/protobuf': cannot load '@@rules_python//python:proto.bzl': no such file and referenced by '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' ```

Makes sure Bazel doesn't autogenerate MODULE.bazel and MODULE.bazel.lock files for now.

This is the last release in the Bazel 6 line. With the changes up to this point, the Bazel 7 build only fails on protobuf generation, e.g. on `bazel build //test/proto3:all`, as seen below. Bumping to Bazel 6.5.0 on top of all the previous changes isolates the Bazel 7.0.0 changes causing the failure. ```txt $ bazel build //test/proto3:all ERROR: .../rules_scala/test/proto3/BUILD:14:14: ProtoScalaPBRule test/proto3/generated-proto-lib_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target //test/proto3:generated-proto-lib) bazel-out/darwin_arm64-opt-exec-ST-13d3ddad9198/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) Could not find file in descriptor database: bazel-out/darwin_arm64-fastbuild/bin/test/proto3/generated.proto: No such file or directory java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:30) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) Use --verbose_failures to see the command lines of failed build steps. ERROR: .../rules_scala/test/proto3/BUILD:14:14 scala @//test/proto3:generated-proto-lib failed: (Exit 1): scalapb_worker failed: error executing ProtoScalaPBRule command (from target //test/proto3:generated-proto-lib) bazel-out/darwin_arm64-opt-exec-ST-13d3ddad9198/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ```

One of these days I'll remember to run `bazel run //tools:lint_fix` before opening a pull request.

Learning more about BuildKite every day.

mbland · 2024-10-05T01:41:08Z

BTW, I've found the smoking gun for the protobuf problem from Bazel 7.0.0, though I'm not yet sure exactly how it went off and what to do about it yet:

Fix paths for sibling repository setup and generated .proto files - appears in Bazel 7.0.0-pre.20230517.4

I'm posting my notes here, but feel free to ignore them for the purpose of this review.

This Bazel 7.0.0 change is playing havoc with the _import_paths() function from scala_proto/private/scala_proto_aspect.bzl. Inserting a print(source_root) statement as the second line revealed the following values:

Bazel 6.5.0: bazel-out/darwin_arm64-fastbuild/bin/test/proto3/_virtual_imports/generated-proto-lib
Bazel 7.0.0: .

As a result, _import_paths() strips the source_root properly under 6.5.0 (see the diff below), but doesn't stand a chance under 7.0.0.

The resulting paths are passed as command line arguments to the scalapb_worker launched by _generate_sources in the same file.

I got the hint to start looking here after running bazel build --verbose_failures //test/proto3:all and noticed the final argument, @bazel-out/darwin_arm64-fastbuild/bin/test/proto3/generated-proto-lib_jvm_extra_protobuf_generator_scalapb.srcjar-0.params. I saved off the versions of this .params file for 6.5.0 and 7.0.0, and got this diff:

--- params-6.5.0.txt	2024-10-04 20:28:49
+++ params-7.0.0.txt	2024-10-04 20:29:12
@@ -8,4 +8,4 @@
 grpc,single_line_to_proto_string
 --descriptor_set_in
 bazel-out/darwin_arm64-fastbuild/bin/test/proto3/generated-proto-lib-descriptor-set.proto.bin
-test/proto3/generated.proto
+bazel-out/darwin_arm64-fastbuild/bin/test/proto3/generated.proto

FWIW, the corresponding generated-proto-lib-descriptor-set.proto.bin (the descriptor database referenced by the protobuf command line interface error message) compared the same between both Bazel versions.

Now that I know I'm looking in the right place, I'll see if I can make sense of it and come up with a proper fix. If someone wants to beat me to it, though, that's fine by me.

Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation documented at: - bazelbuild#1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into bazelbuild#1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

mbland · 2024-10-05T04:58:35Z

I figured out the //test/proto3 breakage and created #1620 to fix it. It doesn't fix the hanging scalapb_workers from #1618, but it's still an important fix. Either that PR or this one can go in first, but both must go in before upgrading to Bazel 7.

Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation documented at: - bazelbuild#1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into bazelbuild#1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

`bash ./test_all.sh` passes after minor updates to the following test cases to handle slightly different behavior under Bazel 7: - `test_custom_reporter_class_has_been_set` makes the test output dir writable, since it no longer is by default. - `test_scala_import_expect_failure_on_missing_direct_deps_warn_mode` removes preexisting output, since Bazel 7 won't emit warnings if it already exists. --- FYI: Under Bazel 7.0 and Bazel 7.1, buildifier warnings for external targets in the following test cases changed to begin with `@@repo_name` instead of `@repo_name`: - test/shell/test_scala_library.sh: `test_scala_library_expect_failure_on_missing_direct_external_deps_jar` `test_scala_library_expect_failure_on_missing_direct_external_deps_file_group` - test/shell/test_strict_dependency.sh: `test_stamped_target_label_loading` `test_strict_deps_filter_included_target` - test/shell/test_test_unused_dependency.sh: `test_unused_deps_filter_included_target` However, as of Bazel 7.2, those warnings changed back to `@repo_name`, so those changes aren't reflected in this commit.

`bazel run //tools:lint_{check,fix}` required updating rules_go to 0.39.1 to work under Bazel 7. The previous rules_go version 0.38.1 caused the following error: ```txt $ ./test_lint.sh ERROR: .../external/io_bazel_rules_go/BUILD.bazel:71:16: in (an implicit dependency) attribute of go_context_data rule @@io_bazel_rules_go//:go_context_data: in $whitelist_function_transition attribute of go_context_data rule @@io_bazel_rules_go//:go_context_data: package group '@@bazel_tools//tools/whitelists/function_transition_whitelist:function_transition_whitelist' is misplaced here (they are only allowed in the visibility attribute) ERROR: .../external/io_bazel_rules_go/BUILD.bazel:71:16: Analysis of target '@@io_bazel_rules_go//:go_context_data' failed ERROR: Analysis of target '//tools:lint_check' failed; build aborted: Analysis failed ```

Updated rules_java to 7.9.0 to fix errors of the following type under Bazel 7.2.1: ```txt ERROR: test/BUILD:640:10: While resolving toolchains for target //test:jar_lister (096dcc8): invalid registered toolchain '@remote_jdk8_linux_aarch64_toolchain_config_repo//:bootstrap_runtime_toolchain': no such target '@@remote_jdk8_linux_aarch64_toolchain_config_repo//:bootstrap_runtime_toolchain': target 'bootstrap_runtime_toolchain' not declared in package '' defined by .../external/remote_jdk8_linux_aarch64_toolchain_config_repo/BUILD.bazel ERROR: Analysis of target '//test:jar_lister' failed; build aborted ``` Also removed the seemingly unused `rules_java_extra` stanza from `WORKSPACE`. --- Though `rules_java` version 7.12.1 is available, and largely works with this repo when using Bazel 7.3.2, it requires a few temporary workarounds. (As I write this, 8.0.0 came out just a few hours ago and I haven't tried it.) Rather than commit the workarounds, upgrading only to 7.9.0 now seems less crufty. Though this commit only updates Bazel to 7.2.1, I suspect it's probably the same basic problem at play. What follows is a very detailed explanation of what happens with 7.12.1 with Bazel 7.3.2, just to have it on the record. --- The workaround is to change a few toolchain and macro file targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. This isn't a terribly bad or invasive workaround, but `@bazel_tools//tools/jdk:` is clearly the canonical path. Best to keep it that way, lest we build up technical debt. Without the workaround, these targets would fail: - //test/src/main/resources/java_sources:CompiledWithJava11 - //test/src/main/resources/java_sources:CompiledWithJava8 - //test/toolchains:java21_toolchain - //test:JunitRuntimePlatform - //test:JunitRuntimePlatform_test_runner - //test:scala_binary_jdk_11 with this error: ```txt ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. ``` This appears to be a consequence of both upgrading the Bazel version to 7.3.2 and updating `rules_java` to 7.12.1. The `rules_java_builtin` repo is part of the `WORKSPACE` prefix that adds implicit dependencies: - https://bazel.build/external/migration#builtin-default-deps This repo was added to 7.0.0-pre.20231011.2 in the following change, mapped to `@rules_java` within the scope of the `@bazel_tools` repo: - bazelbuild/bazel: Add rules_java_builtin to the users of Java modules bazelbuild/bazel@ff1abb2 This change tried to ensure `rules_java` remained compatible with earlier Bazel versions. However, it changed all instances of `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` to `//toolchains:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java: Make rules_java backwards compatible with Bazel 6.3.0 bazelbuild/rules_java@30ecf3f Bazel has bumped `rules_java` in its `workspace_deps.bzl` from 7.9.0 to 7.11.0, but it's only available as of 8.0.0-pre.20240911.1. - bazelbuild/bazel: Update rules_java 7.11.1 / java_tools 13.8 bazelbuild/bazel#23571 bazelbuild/bazel@f92124a --- What I believe is happening is, under Bazel 7.3.2 and `rules_java` 7.12.1: - Bazel creates `rules_java` 7.9.0 as `@rules_java_builtin` in the `WORKSPACE` prefix. - `@bazel_tools` has `@rules_java` mapped to `@rules_java_builtin` when initialized during the `WORKSPACE` prefix, during which `@bazel_tools//tools/jdk` registers `alias()` targets to `@rules_java` toolchain targets. These aliased toolchains specify `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` in their `toolchains` attribute. - `WORKSPACE` loads `@rules_java` 7.12.1 and registers all its toolchains with type `@rules_java//toolchains:bootstrap_runtime_toolchain_type`. - Some `@rules_java` rules explicitly specifying toolchains from `@bazel_tools//tools/jdk` can't find them, because the `@bazel_tools//tools/jdk` toolchain aliases expect toolchains of type `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`. This has broken other projects in the same way: - bazelbuild/bazel: [Bazel CI] Downstream project broken by rules_java upgrade #23619 bazelbuild/bazel#23619 These problems don't appear under Bzlmod, whereby `@rules_java_builtin` was never required. This is because `WORKSPACE` executes its statements sequentially, while Bzlmod builds the module dependency graph _before_ instantiating repositories (within module extensions). It seems a fix is on the way that removes `@rules_java_builtin` from the `WORKSPACE` prefix, and adds `@rules_java` to the suffix. At this moment, though, it's not even in a prerelease: - bazelbuild/bazel: Remove rules_java_builtin in WORKSPACE prefix bazelbuild/bazel@7506690 --- Note that the error message matches that from the following resolved issue, but that issue was for non-Bzlmod child repos when `WORKSPACE` was disabled. - bazelbuild/bazel: Undefined @@rules_java_builtin repository with --noenable_workspace option bazelbuild/bazel#22754

Related to bazelbuild#1482, bazelbuild#1618, and bazelbuild#1619. Results from the investigation documented at: - bazelbuild#1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into bazelbuild#1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

Related to #1482, #1618, and #1619. Results from the investigation documented at: - #1619 (comment) Updates `_import_paths()` in `scala_proto_aspect.bzl` to handle differences `ProtoInfo.proto_source_root` and `ProtoInfo.direct_sources` values between Bazel 6 and Bazel 7. Without this change, `_import_paths()` emits incorrect values under Bazel 7, causing targets containing generated `.proto` inputs to fail, e.g. `//test/proto3:test_generated_proto`. See also: - Fix paths for sibling repository setup and generated .proto files bazelbuild/bazel@6c6c196 - The docstring for `ProtoInfo.proto_source_root` in the Bazel sources: https://github.com/bazelbuild/bazel/blob/7.3.2/src/main/starlark/builtins_bzl/common/proto/proto_info.bzl#L155-L172 - Remove incompatible_generated_protos_in_virtual_imports bazelbuild/bazel@3efaa32 - Comment from: Generated Protos are no longer considered as virtual_imports in Bazel 7 bazelbuild/bazel#21075 (comment) --- I cherrypicked this commit into #1618. While it fixed the `//test/proto3` build failure, it does _not_ fix the hanging scalapb_workers from the ProtoScalaPBRule aspect. I'll have to investiate further whether than hang is related to Bazel, rules_proto, com_google_protobuf, or some mixture thereof. Still, progress!

simuons

Thank you @mbland

Incorporates: - bazelbuild#1619 - bazelbuild#1620

Incorporates: - bazelbuild#1619 - bazelbuild#1620 Removes --noenable_bzlmod from .bazelrc

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. This commit also removes `test/toolchains/jdk.bzl` or `//test/toolchains:java21_toolchain` since we no longer need them. (I actually think we didn't need them with 7.9.0, either, but this seems like a good point at which to remove them.) These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. This commit also removes `test/toolchains/jdk.bzl` or `//test/toolchains:java21_toolchain` since we no longer need them. (I actually think we didn't need them with 7.9.0, either, but this seems like a good point at which to remove them.) These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

Bazel 6.5.0 builds. Bazel 7.4.1 fails with the same error mentioned in bazelbuild#1619. See the notes under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" from commit cd22d88: ```txt $ USE_BAZEL_VERSION=7.4.1 bazel build \ //{src,jmh,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` Bazel 8.0.0rc6 and 8.0.0rc7 fail as described in the previous commit. --- Without the `WORKSPACE` updates from earlier, Bazel 6.5.0 fails with a similar error to 7.4.1 with the `WORKSPACE` update: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../rules_java/toolchains/BUILD:285:14: While resolving toolchains for target @rules_java//toolchains:platformclasspath: No matching toolchains found for types @rules_java//toolchains:bootstrap_runtime_toolchain_type. ERROR: Analysis of target '//test:CheckBytecodeMajorVersion' failed; build aborted: ``` Bazel 7.4.1 fails with a different error: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... ERROR: .../external/remote_java_tools_darwin_arm64/BUILD: no such target '@@remote_java_tools_darwin_arm64//:prebuilt_one_version': target 'prebuilt_one_version' not declared in package '' defined by .../external/remote_java_tools_darwin_arm64/BUILD ERROR: .../external/rules_java/toolchains/BUILD:163:14: no such target '@@remote_java_tools_darwin_arm64//:prebuilt_one_version': target 'prebuilt_one_version' not declared in package '' defined by .../external/remote_java_tools_darwin_arm64/BUILD and referenced by '@@rules_java//toolchains:prebuilt_one_version_darwin_arm64' ERROR: Analysis of target '//test/src/main/resources/java_sources:CompiledWithJava8' failed; build aborted: Analysis failed ```

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@fmeum

Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

@hvadehra

Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```

mbland added 5 commits October 4, 2024 16:18

Set --noenable_bzlmod to prevent MODULE.bazel

a4284a2

Makes sure Bazel doesn't autogenerate MODULE.bazel and MODULE.bazel.lock files for now.

mbland requested review from liucijus and simuons as code owners October 4, 2024 22:43

mbland added 2 commits October 4, 2024 18:47

Fix WORKSPACE lint errors

091f487

One of these days I'll remember to run `bazel run //tools:lint_fix` before opening a pull request.

Set --enable_workspace on last_green CI build

14d3fa6

Learning more about BuildKite every day.

mbland mentioned this pull request Oct 5, 2024

Fix Bazel 7 related protobuf build failures #1620

Merged

mbland added 3 commits October 6, 2024 16:05

mbland changed the title ~~Prepare for Bazel 7 while isolating the protobuf related failure~~ Bazel 7 compatibility updates Oct 6, 2024

mbland mentioned this pull request Oct 6, 2024

Bump Bazel and dependencies to prepare for Bzlmod #1618

Closed

simuons approved these changes Oct 8, 2024

View reviewed changes

liucijus approved these changes Oct 8, 2024

View reviewed changes

liucijus merged commit cd22d88 into bazelbuild:master Oct 8, 2024
2 checks passed

mbland deleted the bzlmod-prepare-for-bazel-7 branch October 8, 2024 13:15

mbland added a commit to mbland/rules_scala that referenced this pull request Oct 8, 2024

Merge branch 'master' into bzlmod

0909f42

Incorporates: - bazelbuild#1619 - bazelbuild#1620

mbland added a commit to mbland/rules_scala that referenced this pull request Oct 8, 2024

Merge branch 'master' into bzlmod

ffdf8ce

Incorporates: - bazelbuild#1619 - bazelbuild#1620 Removes --noenable_bzlmod from .bazelrc

mbland added a commit to mbland/rules_scala that referenced this pull request Oct 8, 2024

Merge branch 'master' into bzlmod

d332ddd

Incorporates: - bazelbuild#1619 - bazelbuild#1620 Removes --noenable_bzlmod from .bazelrc

mbland mentioned this pull request Oct 11, 2024

Support Bzlmod and add rules_scala to bazel-central-registry #1482

Open

mbland mentioned this pull request Oct 23, 2024

Bazel green build CI job now fails persistently due to rules_java versioning #1625

Open

mbland mentioned this pull request Nov 10, 2024

Protobuf compatibility findings and suggestions #1647

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bazel 7 compatibility updates #1619

Bazel 7 compatibility updates #1619

mbland commented Oct 4, 2024 •

edited

Loading

mbland commented Oct 5, 2024

mbland commented Oct 5, 2024

simuons left a comment

Bazel 7 compatibility updates #1619

Bazel 7 compatibility updates #1619

Conversation

mbland commented Oct 4, 2024 • edited Loading

Description

Motivation

mbland commented Oct 5, 2024

mbland commented Oct 5, 2024

simuons left a comment

Choose a reason for hiding this comment

mbland commented Oct 4, 2024 •

edited

Loading