Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace gogo protobuf with google's protobuf v2 compiler #317

Merged
merged 30 commits into from
Nov 21, 2023

Conversation

tdeebswihart
Copy link
Contributor

@tdeebswihart tdeebswihart commented Oct 3, 2023

What changed?

gogo/protobuf has been replaced with Google's official go compiler.

I also changed our code generation to use buf as it was slightly easier to manage plugin versions, though I may revert that.

Why?

gogo/protobuf has been deprecated for some time and the community is moving on, building new tools (like vtproto) atop google's v2 compiler.

Breaking changes

  • *time.Time in proto structs will now be timestamppb.Timestamp
  • *time.Duration will now be durationpb.Duration
  • V2-generated structs embed locks, so you cannot dereference them willy-nilly. go vet will scream at you about this
  • Proto enums will, when formatted to JSON, now be in SCREAMING_SNAKE_CASE rather than PascalCase. We decided (in discussion with the SDK team) that now was as good a time as any to rip the bandage off

Release Plan

Our goal is to release our API, Go API, Go SDK, and Server all at once, then update our UI (and ui-server), CLI, and HTTP API in a second pass. Until the second pass happens no changes to our JSON formatting will be apparent to users

While the performance won't be as nice and it won't generate helper
methods for use it'll be easier to maintain and is not deprecated.
This will let us use the makefile in other repos instead of copying all
our GRPC stuff everywhere...
We really don't care if the Google protos get an Equal method...
Doing this manually is tedious and buf will manage dependencies (like
the GRPC protoc plugin) for us. We already use it for linting so /shrug
Whoops.

Eventually we can swap to vtprotobuf and simply call the vtprotobuf
helpers from these generic methods without changing our code elsewhere
for a nice performance gain
I'm probably going to ditch buf for codegen eventually anyways...
This also fixes a bug where fmt.Fprintf wouldn't write the entire
response (as I never checked how much was written).

Also its binary and writing binary to a file using strings is silly
Makefile Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
buf.yaml Show resolved Hide resolved
protoc-gen-go-helpers/main.go Outdated Show resolved Hide resolved
buf.gen.yaml Show resolved Hide resolved
@tdeebswihart tdeebswihart marked this pull request as ready for review October 27, 2023 22:09
@tdeebswihart tdeebswihart requested review from a team as code owners October 27, 2023 22:09
@tdeebswihart tdeebswihart requested a review from cretz October 27, 2023 22:47
Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (may want to wait on merging until dependents are ready)

buf.yaml Outdated
breaking:
use:
- WIRE_JSON
ignore:
- dependencies
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming we can remove this post-merge

Copy link
Member

@alexshtin alexshtin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sounds like you had to

required vendoring a few of Google's official protos

but this is not a case anymore. Please update PR description before merging.

Comment on lines -6 to -8
github.com/grpc-ecosystem/grpc-gateway v1.16.0
github.com/pseudomuto/protoc-gen-doc v1.5.1
github.com/temporalio/gogo-protobuf v1.22.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was here to pin dependencies to specific versions. Not every project follows semver and from time to time someone may make a breaking change. I am actually ok to take this risk and use latest always.

api-linter.yaml Show resolved Hide resolved
tdeebswihart and others added 5 commits November 19, 2023 14:53
* Create workflow collection proto

* address comments

* add export in the name

* Apply suggestions from code review

Co-authored-by: Roey Berman <[email protected]>

* Apply suggestions from code review

* Update temporal/api/export/v1/message.proto

* address comments

* rename workflowExecutionHistory to workflowExecution

---------

Co-authored-by: Roey Berman <[email protected]>
The stamp file let's us only do so when the file actually changes :)
@tdeebswihart tdeebswihart merged commit a19522a into temporalio:master Nov 21, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants