Make it possible to combine/add plugins

[rfevang]

Description

I'm trying to run a custom protoc plugin that modifies the output of a different (Java) plugin, using insertion points.

I first tried looking for an argument to java_proto_library or java_proto_compile that let me add my plugin. When I didn't see any I then tried making my own compile rule, specifying both plugins. This attempt was however met by errors saying that the insertion points were not found.

I looked at the source and spotted this:

# Each plugin is isolated to its own execution of protoc, as plugins may have differing
# exclusions that cannot be expressed in a single protoc execution for all plugins.

for plugin in plugins:

As far as I can tell this effectively makes it impossible to use insertion point based protoc plugins with rules-proto-grpc.

Is it possible to change this? It seems like the low level function should support multiple plugins in one call, and then leave it up to higher level functions to split plugins into multiple calls to the low level compile, if that is desired.

Ideally I'd like to see an argument to the high level functions where one could pass insertion point based plugins.

[aaliddell]

Hi, do you have an example of such a plugin that depends on the outputs of prior plugins? I wasn't even aware these exist and would need one to test with.

Passing plugins at the top level used to be impossible to support due to the architecture of these rules, but actually since 4.0.0 dropped transitivity it may be feasible to implement a generic_compile rule that takes the plugin list (this already mostly exists in proto_compile_impl). There may be some constraint I am forgetting though...

As for running in a shared execution, that is supported by protoc just fine but the comment you copied about exclusions remains a concern. If the plugins share the same exclusions (which most do) then perhaps the execution could be merged 🤔

[mbgit2]

Hi I would like to chime in that we also would really like to have support for this feature. I've made an example repository for you to see the issue we're running into. We're trying to use an existing protoc plugin that injects into existing java files.

Running bazel build //proto/hello/v1:hello_java_proto_compile will show the issue.

Previously we used stackb/rules_proto but would like to migrate to rules_proto_grpc as there is more support.

[aaliddell]

I'm taking a look at how this might be supported and it's going to require a bit of a rethink of the core execution of protoc used in these rules.

At present, protoc is called once per plugin, meaning each runs in isolation. I don't want to switch to a single combined protoc execution in general, as it's useful to be able to isolate plugins from each other, especially when there are mixed output types (files vs dir).

So, to have opt-in combined execution it'd either need to be a flag on the rule or alternatively plugins could specify other plugins they are dependent on and we then run each dependency tree in its own protoc execution. I prefer the latter