Skip to content

Update Beam Protobuf Schema (Java) #35150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Conversation

baeminbo
Copy link
Contributor

@baeminbo baeminbo commented Jun 4, 2025

Fix #35081.

Refer to https://s.apache.org/beam-protobuf.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@baeminbo baeminbo force-pushed the fix-35081 branch 4 times, most recently from 9342333 to 23875da Compare June 5, 2025 06:58
@baeminbo baeminbo changed the title [WIP] Update Beam Protobuf Schema (Java) Update Beam Protobuf Schema (Java) Jun 5, 2025
Copy link
Contributor

github-actions bot commented Jun 5, 2025

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@baeminbo baeminbo force-pushed the fix-35081 branch 4 times, most recently from 6ae0d10 to 5e53125 Compare June 6, 2025 02:16
@baeminbo
Copy link
Contributor Author

I haven't finish a full review but I have two big picture comments:

  1. The amount of @NonNull annotations is suspicious, since this is the default. You should very rarely need to use it. There are also many places where you apply @Nullable to a type variable, which is usually rare, but I think might be fine in some of these cases.
  2. It feels like the many subclasses really might be simple as just functions / a switch statement.

Thank you so much for the review! I have refactored the ProtoBeamConverter class.

@baeminbo
Copy link
Contributor Author

I've cherry-picked the commit at #35217. Thanks!

@baeminbo
Copy link
Contributor Author

Run Java PreCommit

@baeminbo
Copy link
Contributor Author

@kennknowles could you please proceed with the review when you have time? Thank you.

@kennknowles
Copy link
Member

I haven't gotten to make another pass yet. I did just look at the tests and there is a prism failure which is pretty rare. I restarted the test job anyhow but you might take a look at the gradle scan at https://develocity.apache.org/s/o35wszaauzfvy/tests/task/:runners:prism:java:test/details/org.apache.beam.runners.prism.PrismRunnerTest/windowing?top-execution=1 just to make sure it really was a flake

@baeminbo
Copy link
Contributor Author

Run Java PreCommit

@baeminbo
Copy link
Contributor Author

I haven't gotten to make another pass yet. I did just look at the tests and there is a prism failure which is pretty rare. I restarted the test job anyhow but you might take a look at the gradle scan at https://develocity.apache.org/s/o35wszaauzfvy/tests/task/:runners:prism:java:test/details/org.apache.beam.runners.prism.PrismRunnerTest/windowing?top-execution=1 just to make sure it really was a flake

The test passed locally, so I triggered a rerun and it's now green.

Copy link
Contributor

Reminder, please take a look at this pr: @m-trieu @damccorm

@damccorm
Copy link
Contributor

R: @kennknowles

Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@liferoad liferoad requested a review from kennknowles July 11, 2025 14:30
@liferoad
Copy link
Contributor

@ahmedabu98

@kennknowles
Copy link
Member

I think that the many cases of @NonNull and @Nullable applied to type variables indicates a problem. Each one really needs a comment justifying it, because it is unusual and actually a somewhat inflexible situation. Probably you can remove most of them. And some places where you have a type variable and you need to constrain what types it can be instantiated with, you would use extends @NonNull Object (the default if you do not annotate is extends @Nullable Object which allows the type variable to be instantiated with any type)

@baeminbo
Copy link
Contributor Author

I've removed @NonNull and @Nullable from generic types in the ProtoBeamConverter class.

I realized I had significantly misunderstood their proper usage. Nullability annotations on generics are only for specific use cases with strong reasoning (apparently a container-like class?).

@baeminbo
Copy link
Contributor Author

Run Java PreCommit

1 similar comment
@baeminbo
Copy link
Contributor Author

Run Java PreCommit

@baeminbo
Copy link
Contributor Author

Run Java_Amazon-Web-Services2_IO_Direct PreCommit

@baeminbo
Copy link
Contributor Author

@kennknowles could you review this pr again? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request]: Beam Protobuf Schema (Java)
4 participants