Skip to content

Conversation

@dannykopping
Copy link
Collaborator

We need to augment Anthropic's streaming responses to include things like our interception ID, cumulative token usage, etc.

When we stream the intercepted events to the client, we currently marshal them using the SDK's marshaler, which does not omit zero-values.

We ran into an issue with this today in Mux where the SDK they're using does not handle zero-values elegantly:

2:40.449PM dist/node/services/workspaceTitleGenerator.js:38 Failed to generate workspace name with AI APICallError [AI_APICallError]: Invalid JSON response
    at /tmp/mux/npm/node_modules/@ai-sdk/provider-utils/dist/index.js:1098:11
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async postToApi (/tmp/mux/npm/node_modules/@ai-sdk/provider-utils/dist/index.js:890:14)
    ... 3 lines matching cause stack trace ...
    at async _retryWithExponentialBackoff (/tmp/mux/npm/node_modules/ai/dist/index.js:1795:12)
    at async fn (/tmp/mux/npm/node_modules/ai/dist/index.js:7086:32)
    at async /tmp/mux/npm/node_modules/ai/dist/index.js:1644:22
    at async generateObject (/tmp/mux/npm/node_modules/ai/dist/index.js:7045:12) {
  cause: _TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: {"id":"5f189cd1-0236-4c5a-a76a-b33af33110b8","content":[{"citations":null,"text":"Based on the issue about the `vscode-web` module not applying settings.json and taking a long time to install, here's a git-safe branch name:","type":"text","signature":"","thinking":"","data":"","id":"","input":null,"name":"","content":{"OfWebSearchResultBlockArray":null,"error_code":"","type":"web_search_tool_result_error"},"tool_use_id":""},{"citations":null,"text":"","type":"tool_use","signature":"","thinking":"","data":"","id":"toolu_0111SMMH1XaxMs2WpJouLu7L","input":{"name":"fix-vscode-web-settings"},"name":"json","content":{"OfWebSearchResultBlockArray":null,"error_code":"","type":"web_search_tool_result_error"},"tool_use_id":""}],"model":"claude-sonnet-4-5-20250929","role":"assistant","stop_reason":"tool_use","stop_sequence":"","type":"message","usage":{"cache_creation":{"ephemeral_1h_input_tokens":0,"ephemeral_5m_input_tokens":19098},"cache_creation_input_tokens":19098,"cache_read_input_tokens":19098,"input_tokens":1804,"output_tokens":205,"server_tool_use":{"web_search_requests":0},"service_tier":""}}.
  Error message: [
    {
      "expected": "array",
      "code": "invalid_type",
      "path": [
        "content",
        0,
        "citations"
      ],
      "message": "Invalid input: expected array, received null"
    }
  ]

This PR removes the need for this marshaling, and brings the Anthropic implementation in line with the OpenAI implementation, which had to change this for a similar reason.

Signed-off-by: Danny Kopping <danny@coder.com>
Copy link

@ssncferreira ssncferreira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@dannykopping dannykopping merged commit d62a713 into main Dec 3, 2025
1 check passed
dannykopping added a commit to coder/coder that referenced this pull request Dec 3, 2025
Changes from coder/aibridge#70

Signed-off-by: Danny Kopping <danny@coder.com>
dannykopping added a commit to coder/coder that referenced this pull request Dec 3, 2025
Changes from coder/aibridge#70

Signed-off-by: Danny Kopping <danny@coder.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants