Skip to content

fix: add constraint and runtime check for provisioner logs size limit #18893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Jul 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
9f8cd50
add logs overflowed field to provisioner jobs and gen models
bcpeinhardt Jul 15, 2025
68d8e5e
update queries and make gen
bcpeinhardt Jul 15, 2025
476c6ff
add logs length field as well with constraint
bcpeinhardt Jul 15, 2025
c8de633
update queries
bcpeinhardt Jul 15, 2025
908c888
handle log size overflow on insertion
bcpeinhardt Jul 15, 2025
c709a9e
update sdk
bcpeinhardt Jul 15, 2025
f32ac68
implement dbauthz stubs
bcpeinhardt Jul 15, 2025
29ad77c
couple tests
bcpeinhardt Jul 15, 2025
f6134aa
initial frontend updates
bcpeinhardt Jul 15, 2025
cc469f8
runtime check possible overflow
bcpeinhardt Jul 21, 2025
29297d6
update log message to be less confusing
bcpeinhardt Jul 21, 2025
cd891db
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 21, 2025
1398491
lint fixes
bcpeinhardt Jul 21, 2025
395c9e1
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 21, 2025
630e2ac
rename migrations after conflicting merge
bcpeinhardt Jul 21, 2025
e44cd47
run make gen after migration
bcpeinhardt Jul 21, 2025
fa118db
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 21, 2025
105de84
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 29, 2025
2923e9d
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 30, 2025
61a58f5
adjust migration numbers
bcpeinhardt Jul 30, 2025
e7f2ec6
update dbauthz and querymetrics for removed query
bcpeinhardt Jul 30, 2025
4a2a7f8
add rbac tests for provisioner job logs length and overflowed queries
bcpeinhardt Jul 30, 2025
8115e4a
reshuffling
bcpeinhardt Jul 30, 2025
99d5090
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 30, 2025
c09f0f6
update migration numbers
bcpeinhardt Jul 30, 2025
1ab1347
remove period from slog log
bcpeinhardt Jul 30, 2025
6964cee
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardt Jul 30, 2025
a03a956
migration rename
bcpeinhardt Jul 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cli/testdata/coder_list_--output_json.golden
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@
"template_name": "",
"template_display_name": "",
"template_icon": ""
}
},
"logs_overflowed": false
},
"reason": "initiator",
"resources": [],
Expand Down
2 changes: 1 addition & 1 deletion cli/testdata/coder_provisioner_jobs_list_--help.golden
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ OPTIONS:
-O, --org string, $CODER_ORGANIZATION
Select which organization (uuid or name) to use.

-c, --column [id|created at|started at|completed at|canceled at|error|error code|status|worker id|worker name|file id|tags|queue position|queue size|organization id|template version id|workspace build id|type|available workers|template version name|template id|template name|template display name|template icon|workspace id|workspace name|organization|queue] (default: created at,id,type,template display name,status,queue,tags)
-c, --column [id|created at|started at|completed at|canceled at|error|error code|status|worker id|worker name|file id|tags|queue position|queue size|organization id|template version id|workspace build id|type|available workers|template version name|template id|template name|template display name|template icon|workspace id|workspace name|logs overflowed|organization|queue] (default: created at,id,type,template display name,status,queue,tags)
Columns to display in table output.

-l, --limit int, $CODER_PROVISIONER_JOB_LIST_LIMIT (default: 50)
Expand Down
2 changes: 2 additions & 0 deletions cli/testdata/coder_provisioner_jobs_list_--output_json.golden
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"template_display_name": "",
"template_icon": ""
},
"logs_overflowed": false,
"organization_name": "Coder"
},
{
Expand Down Expand Up @@ -57,6 +58,7 @@
"workspace_id": "===========[workspace ID]===========",
"workspace_name": "test-workspace"
},
"logs_overflowed": false,
"organization_name": "Coder"
}
]
3 changes: 3 additions & 0 deletions coderd/apidoc/docs.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions coderd/apidoc/swagger.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 16 additions & 0 deletions coderd/database/dbauthz/dbauthz.go
Original file line number Diff line number Diff line change
Expand Up @@ -4489,6 +4489,22 @@ func (q *querier) UpdateProvisionerJobByID(ctx context.Context, arg database.Upd
return q.db.UpdateProvisionerJobByID(ctx, arg)
}

func (q *querier) UpdateProvisionerJobLogsLength(ctx context.Context, arg database.UpdateProvisionerJobLogsLengthParams) error {
// Not sure what the rbac should be here, going with this for now
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only bit I'm unsure of, maybe include the same comment as the other provisioner related functions have?

// TODO: Remove this once we have a proper rbac check for provisioner jobs.
// Details in https://github.com/coder/coder/issues/16160

or ask @mafredri since he opened that issue?

Copy link
Contributor Author

@bcpeinhardt bcpeinhardt Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our manual testing was actually from the admin account, we should look at this again before merging I think.
Edit: I tested with a member account and everything works fine. I could be way off here but I think the action.Update on the ResourceProvisionerJob is coming from the template admin?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several other queries which perform the same rbac check we do to update fields on the ProvisionerJobs table, so I feel comfortable merging this and revisiting the provisioner jobs rbac in the future.

if err := q.authorizeContext(ctx, policy.ActionUpdate, rbac.ResourceProvisionerJobs); err != nil {
return err
}
return q.db.UpdateProvisionerJobLogsLength(ctx, arg)
}

func (q *querier) UpdateProvisionerJobLogsOverflowed(ctx context.Context, arg database.UpdateProvisionerJobLogsOverflowedParams) error {
// Not sure what the rbac should be here, going with this for now
if err := q.authorizeContext(ctx, policy.ActionUpdate, rbac.ResourceProvisionerJobs); err != nil {
return err
}
return q.db.UpdateProvisionerJobLogsOverflowed(ctx, arg)
}

func (q *querier) UpdateProvisionerJobWithCancelByID(ctx context.Context, arg database.UpdateProvisionerJobWithCancelByIDParams) error {
// TODO: Remove this once we have a proper rbac check for provisioner jobs.
// Details in https://github.com/coder/coder/issues/16160
Expand Down
14 changes: 14 additions & 0 deletions coderd/database/dbauthz/dbauthz_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4341,6 +4341,20 @@ func (s *MethodTestSuite) TestSystemFunctions() {
UpdatedAt: time.Now(),
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("UpdateProvisionerJobLogsLength", s.Subtest(func(db database.Store, check *expects) {
j := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{})
check.Args(database.UpdateProvisionerJobLogsLengthParams{
ID: j.ID,
LogsLength: 100,
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("UpdateProvisionerJobLogsOverflowed", s.Subtest(func(db database.Store, check *expects) {
j := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{})
check.Args(database.UpdateProvisionerJobLogsOverflowedParams{
ID: j.ID,
LogsOverflowed: true,
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("InsertProvisionerJob", s.Subtest(func(db database.Store, check *expects) {
dbtestutil.DisableForeignKeysAndTriggers(s.T(), db)
check.Args(database.InsertProvisionerJobParams{
Expand Down
1 change: 1 addition & 0 deletions coderd/database/dbfake/dbfake.go
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ func (b WorkspaceBuildBuilder) Do() WorkspaceResponse {
Input: payload,
Tags: map[string]string{},
TraceMetadata: pqtype.NullRawMessage{},
LogsOverflowed: false,
})
require.NoError(b.t, err, "insert job")

Expand Down
1 change: 1 addition & 0 deletions coderd/database/dbgen/dbgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -775,6 +775,7 @@ func ProvisionerJob(t testing.TB, db database.Store, ps pubsub.Pubsub, orig data
Input: takeFirstSlice(orig.Input, []byte("{}")),
Tags: tags,
TraceMetadata: pqtype.NullRawMessage{},
LogsOverflowed: false,
})
require.NoError(t, err, "insert job")
if ps != nil {
Expand Down
14 changes: 14 additions & 0 deletions coderd/database/dbmetrics/querymetrics.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions coderd/database/dbmock/dbmock.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion coderd/database/dump.sql

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions coderd/database/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,11 @@ func IsWorkspaceAgentLogsLimitError(err error) bool {

return false
}

func IsProvisionerJobLogsLimitError(err error) bool {
var pqErr *pq.Error
if errors.As(err, &pqErr) {
return pqErr.Constraint == "max_provisioner_logs_length" && pqErr.Table == "provisioner_jobs"
}
return false
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
ALTER TABLE provisioner_jobs DROP COLUMN logs_length;
ALTER TABLE provisioner_jobs DROP COLUMN logs_overflowed;
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
-- Add logs length tracking and overflow flag, similar to workspace agents
ALTER TABLE provisioner_jobs ADD COLUMN logs_length integer NOT NULL DEFAULT 0 CONSTRAINT max_provisioner_logs_length CHECK (logs_length <= 1048576);
ALTER TABLE provisioner_jobs ADD COLUMN logs_overflowed boolean NOT NULL DEFAULT false;

COMMENT ON COLUMN provisioner_jobs.logs_length IS 'Total length of provisioner logs';
COMMENT ON COLUMN provisioner_jobs.logs_overflowed IS 'Whether the provisioner logs overflowed in length';
4 changes: 4 additions & 0 deletions coderd/database/models.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions coderd/database/querier.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading