[BUG] Generated Python code leads to runtime errors

[Omar-V2]

Description

The following import line is included in the generated python code:

from buf.validate import validate_pb2 as buf_dot_validate_dot_validate__pb2

However, this code leads to runtime errors since there is no module named buf

ModuleNotFoundError: No module named 'buf'

I have the python package protovalidate installed as dependency in my project.

Steps to Reproduce

1. Write a .proto file that utilises protovalidate

syntax = "proto3";

package examplev1;

import "buf/validate/validate.proto";

service ExampleService {
  ....
}

message ExampleRequest {
  string device_id = 1 [(buf.validate.field).string.pattern = "^([0-9A-Fa-f]{2}[:]){5}([0-9A-Fa-f]{2})$"];
}

2. Generate code with buf generate

The buf.gen.yaml file I'm using

version: v1
managed:
  enabled: true
  go_package_prefix:
    default: example/proto/gen/go
    except:
      - buf.build/bufbuild/protovalidate
plugins:
  - plugin: buf.build/protocolbuffers/go
    out: proto/gen/go
    opt: paths=source_relative
  - plugin: buf.build/bufbuild/connect-go
    out: proto/gen/go
    opt: paths=source_relative
  - plugin: buf.build/grpc/go
    out: proto/gen/go
    opt: paths=source_relative
  - plugin: buf.build/grpc/python
    out: proto/gen/python
  - plugin: buf.build/protocolbuffers/python
    out: proto/gen/python
    opt: pyi_out

the buf.yaml I'm using

version: v1
name: buf.build/mystuff/core
deps:
  - buf.build/bufbuild/protovalidate
breaking:
  use:
    - FILE
lint:
  ignore:
    - validate
  use:
    - DEFAULT
    - COMMENTS
    - UNARY_RPC
    - PACKAGE_NO_IMPORT_CYCLE

I've made sure that I've ran buf mod update and that my buf.lock contains protovalidate as a dep.

buf generate command I'm executing: buf generate --template buf.gen.yaml

3. Inspect generated python code: the following line is contained in the *_pb2.py and *_pb2.pyi files

from buf.validate import validate_pb2 as buf_dot_validate_dot_validate__pb2

There is no module named buf hence a runtime error is encountered

ModuleNotFoundError: No module named 'buf'

Expected Behavior

I expected the validate imports in the generated python code to not lead to a runtime error

Actual Behavior

The generated python code attempts to utilises a module that does not exists/is not present in the project dependencies (despite installed the protovalidate python package)

Screenshots/Logs

N/A

Environment

• Operating System: macOS
• Version: macOS Ventura 13.4.1 (c)
• Compiler/Toolchain:
  Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  Target: x86_64-apple-darwin22.5.0
  Thread model: posix
  InstalledDir: /Library/Developer/CommandLineTools/usr/bin
• Protobuf Compiler & Version: buf v1.25.0, protoc 3.19.4
• Protoc-gen-validate Version: N/A
• Protovalidate Version: 0.1.0

Possible Solution

Additional Context

[Alfus]

Buf does not currently support remote generation for python, so the protovalidate protos have to be generated, by either:

• running buf generate buf.build/bufbuild/protovalidate:$(PROTOVALIDATE_VERSION) directly. For example see: https://github.com/bufbuild/protovalidate-python/blob/main/Makefile#L39
• running 'buf generate --include_imports`. See https://buf.build/docs/reference/cli/buf/generate/#include-imports. This solution may be problematic if you are using other languages in the same repo.
• Using protoc somehow

We are working on improving the documentation #54

Additionally, we are also working on improvements to buf CLI make this a more seamless experience, sorry for the inconvenience.

[Omar-V2]

@Alfus thanks for your response, I can confirm that --include-imports did indeed fix the issue I was facing above. But yes some inclusion of these steps in the docs would be great! Feel free to close this.

[dobet]

What should I do if I am using python -m grpcio_tools.protoc instead of buf to compile the proto file?