Skip to content

✨ prototyping dynamic graphql handler for catalogd #2109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

joelanford
Copy link
Member

@joelanford joelanford commented Jul 17, 2025

This PR adds a graphql handler to the catalogd server (behind a feature gate). In order to add the graphql handler, I decided to refactor the storage and http handlers to decouple the storage and serving aspects into separate packages.

The idea with the GraphQL storage is that when we store an FBC, we build up a graphql schema dynamically based on the content of the FBC. The GraphQL handler the uses the stored (in memory) schemas for each catalog and serves a GraphQL API for them. The result being that catalogd is still a "dumb pipe" for FBC data. The GraphQL API is directly derived from the FBC API (or at least the parts of the FBC API that are in use somewhere in the served FBC).

To play with it, I've enabled the GraphQL explorer handler, so you can:

make run-experimental
kubectl port-forward -n olmv1-system service/catalogd-service 8443:443

And then open https://localhost:8443/catalogs/operatorhubio/api/v1/graphql

And here's are some interesting queries to run:

Bundle search metadata

{
  olmBundle {
    package
    name
    properties(type: "olm.csv.metadata") {
      value(jq:".annotations | [to_entries[] | select(.key | test(\"features|valid-subscription|description|capabilities|categories\"))] | from_entries")
    }
  }
}

What OLMv1 operator-controller actually needs for resolution:

{
  olmChannel {
    package
    name
    entries {
      name
      replaces
      skips
      skipRange
    }
  }
  olmBundle {
    package
    name
    image
  }
}

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

Signed-off-by: Joe Lanford <joe.lanford@gmail.com>
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 17, 2025
Copy link

netlify bot commented Jul 17, 2025

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit 7d63d5d
🔍 Latest deploy log https://app.netlify.com/projects/olmv1/deploys/687eb0b1433c4c0008324818
😎 Deploy Preview https://deploy-preview-2109--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link

openshift-ci bot commented Jul 17, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tmshort for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@joelanford joelanford force-pushed the catalogd-graphql branch 3 times, most recently from f0f601d to c4cdc70 Compare July 19, 2025 23:50
Copy link

codecov bot commented Jul 20, 2025

Codecov Report

Attention: Patch coverage is 50.94637% with 622 lines in your changes missing coverage. Please review.

Project coverage is 70.00%. Comparing base (0491775) to head (7d63d5d).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...r/internal/handlerutil/http_preconditions_check.go 0.00% 133 Missing ⚠️
...ternal/catalogd/storage/internal/graphql/schema.go 61.68% 105 Missing and 18 partials ⚠️
internal/catalogd/handler/api/v1/metas.go 3.22% 59 Missing and 1 partial ⚠️
...ernal/catalogd/storage/internal/graphql/context.go 0.00% 55 Missing ⚠️
internal/catalogd/storage/internal/graphql/json.go 16.07% 46 Missing and 1 partial ⚠️
internal/catalogd/storage/indices.go 49.36% 33 Missing and 7 partials ⚠️
internal/catalogd/handler/api/v1/graphql.go 3.22% 29 Missing and 1 partial ⚠️
internal/catalogd/storage/files.go 61.84% 21 Missing and 8 partials ⚠️
internal/catalogd/storage/storage.go 80.29% 19 Missing and 8 partials ⚠️
internal/catalogd/storage/index/index.go 57.89% 21 Missing and 3 partials ⚠️
... and 8 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2109      +/-   ##
==========================================
- Coverage   73.48%   70.00%   -3.49%     
==========================================
  Files          78       94      +16     
  Lines        7240     8090     +850     
==========================================
+ Hits         5320     5663     +343     
- Misses       1568     2045     +477     
- Partials      352      382      +30     
Flag Coverage Δ
e2e 40.20% <19.40%> (-3.70%) ⬇️
experimental-e2e 56.11% <49.84%> (+6.10%) ⬆️
unit 49.98% <3.70%> (-8.77%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joelanford joelanford requested a review from Copilot July 20, 2025 00:43
Copilot

This comment was marked as outdated.

Signed-off-by: Joe Lanford <joe.lanford@gmail.com>
@joelanford joelanford requested a review from Copilot July 20, 2025 16:51
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a prototype GraphQL handler for catalogd, enabling dynamic GraphQL API generation based on FBC content. The implementation refactors the existing storage and HTTP handlers to decouple storage and serving concerns into separate packages.

  • Adds GraphQL handler with dynamic schema generation from FBC content
  • Refactors storage layer into modular components (files, indices, GraphQL schemas)
  • Replaces monolithic HTTP handler with composable middleware-based handlers

Reviewed Changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
manifests/experimental*.yaml Enable APIV1GraphQLHandler feature gate
internal/catalogd/storage/storage.go Refactored storage interface with modular instances
internal/catalogd/storage/graphql.go New GraphQL schema storage component
internal/catalogd/handler/api/v1/*.go New modular API v1 handlers (all, metas, graphql)
internal/catalogd/handler/middleware/*.go HTTP middleware components (logging, metrics, gzip)
cmd/catalogd/main.go Updated main to use new modular architecture

joelanford and others added 2 commits July 21, 2025 17:24
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant