Skip to content

chore: add profiling labels for pprof analysis #19232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 7, 2025
Merged

Conversation

Emyrk
Copy link
Member

@Emyrk Emyrk commented Aug 7, 2025

PProf labels segment the code into groups for determing the source of cpu/memory profiles. Since the web server and background jobs share a lot of the same code (eg wsbuilder), it helps to know if the load is user induced, or background job based.

Example output:

# Using a 30s cpu sample
$ go tool pprof "http://localhost:6060/debug/pprof/profile?seconds=30"
(pprof) tags
 service: Total 390.0ms
          310.0ms (79.49%): http-server
           80.0ms (20.51%): terraform-provisioner

PProf labels segment the code into groups for determing the source
of cpu/memory profiles. Since the web server and background jobs
share a lot of the same code (eg wsbuilder), it helps to know
if the load is user induced, or background job based.
@Emyrk
Copy link
Member Author

Emyrk commented Aug 7, 2025

I'm sure the pproflabel pkg api can be improved. Just want to test this out, and see how helpful it is. This is exploratory, and seeing what kind of code partitioning can happen in pyroscope to be helpful.

@Emyrk Emyrk marked this pull request as ready for review August 7, 2025 15:47
Copy link
Contributor

@dannykopping dannykopping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only nits, this looks rad

@@ -903,7 +903,8 @@ func (api *API) updateEntitlements(ctx context.Context) error {
}

api.AGPL.PrebuildsReconciler.Store(&reconciler)
go reconciler.Run(context.Background())
// TODO: Should this context be the app context?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the "app" referring to here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api.ctx.

I just noticed the prebuild routine does not get the app context shutdown.

}

const (
ServiceTerraformProvisioner = "terraform-provisioner"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up: would be cool to label pubsub as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which pubsub? The database pubsub listeners?

func (p *PGPubsub) subscribeQueue(event string, newQ *msgQueue) (cancel func(), err error) {

@@ -333,6 +333,7 @@ func New(ctx context.Context, opts *Options) (*Server, error) {
r.Use(
// TODO: @emyrk Should we standardize these in some other package?
httpmw.Recover(s.Logger),
httpmw.WithProfilingLabels,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth discriminating on ws vs https traffic?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that is a really good idea. They should have that upgrade header iirc. Will add

Copy link
Member Author

@Emyrk Emyrk Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dannykopping

request_type: Total 730.0ms
    550.0ms (75.34%): http
    180.0ms (24.66%): websocket

service: Total 810.0ms
    730.0ms (90.12%): http-api
     80.0ms ( 9.88%): terraform-provisioner

@Emyrk Emyrk merged commit 8ba8b4f into main Aug 7, 2025
31 checks passed
@Emyrk Emyrk deleted the stevenmasley/profile_labels branch August 7, 2025 16:21
@github-actions github-actions bot locked and limited conversation to collaborators Aug 7, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants