-
Notifications
You must be signed in to change notification settings - Fork 380
Description
Summary
When a tool returns both content and structuredContent with a widget (openai/outputTemplate), the content field is not forwarded to the model. Only structuredContent is passed (as a raw JSON code block), causing the model to miss any info included in content.
Expected Behavior
Per the MCP server documentation:
content– optional narration (Markdown or plaintext) for the model's response.
The widget reads those payloads through
window.openai.toolOutputandwindow.openai.toolResponseMetadata, while the model only seesstructuredContent/content.
The model should receive both structuredContent and content, allowing developers to provide a short narration text (e.g., "Found 14 products") that guides the model's response.
Actual Behavior
When a widget renders:
structuredContentis passed to the model as acontent_type: "code"block withlanguage: "json"contentis not present in the model's context_metais correctly forwarded totool_response_metadata- A hidden system message is added: "The tool included embedded UI which has been displayed to the user"
The model then sees the full raw JSON from structuredContent and attempts to summarize it, echoing data (titles, prices, URLs, descriptions) that is already displayed in the widget UI.
Reproduction
MCP Server returns:
return {
content: [{ type: "text", text: "Found 14 products for "jackets"." }],
structuredContent: { products: [...], searches: [...], brand: {...} },
_meta: { apiBaseUrl: "...", subject: null },
};Verified via MCP Inspector:
- "Unstructured Content" shows:
"Found 14 products for "jackets"." - This confirms the server IS sending
content
Captured ChatGPT SSE stream (/backend-api/f/conversation):
- Tool response message has
content.content_type: "code"withcontent.textcontaining onlystructuredContentfields - No trace of the
contentarray text _metaappears inmetadata.chatgpt_sdk.tool_response_metadata(proving MCP response is received)
Impact
- Model loses the
contentfield, which can have important information
Environment
@modelcontextprotocol/sdk: 1.12.1- Transport:
StreamableHTTPServerTransport - Tool has
openai/outputTemplateset (widget renders)
Questions
- Is this intentional behavior when a widget is rendered?
- If so, the documentation should clarify that
contentis ignored whenopenai/outputTemplateis set - If not, can this be fixed so
contentreaches the model alongsidestructuredContent?