RFC: [JS] Bidirectional Actions, Flows, and Models #4210
Conversation
pavelgj
changed the title
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive RFC document outlining bidirectional streaming capabilities for Genkit Actions, Flows, and Models. The document clearly articulates the motivation, design principles, and usage examples for defineBidiAction, defineBidiFlow, and defineBidiModel, along with the streamBidi execution API. The integration with Reflection API V2 is also well-described. The proposal is well-structured and easy to understand.
| #### The Role of `init` | ||
| For real-time sessions, the connection to the model API often requires configuration (temperature, system prompt, tools) to be established *before* the first user message is received. The `init` payload fulfills this requirement, separating session configuration from the conversation stream. | ||
|
|
||
| - **`init`**: `GenerateRequest` (contains config, tools, system prompt). |
There was a problem hiding this comment.
For clarity, consider rephrasing this to indicate that the system prompt is part of the messages array within the GenerateRequest, rather than a direct field. While GenerateRequest can contain a system prompt, it's typically nested within its messages field.
|
Hello @pavelgj, I have reviewed the RFC document |
| Actions and Flows expose a `streamBidi` method that returns a `BidiStreamingResponse`. | ||
|
|
||
| ```typescript | ||
| interface BidiStreamingResponse<O, S, I> { |
There was a problem hiding this comment.
| interface BidiStreamingResponse<O, S, I> { | |
| interface BidiStreamingResponse<I, O, S> { |
| model: 'my-model', | ||
| config: init?.config, | ||
| systemPrompt: init?.messages?.find(m => m.role === 'system'), | ||
| tools: init?.tools, |
There was a problem hiding this comment.
Tool requests are also streamed out, unless I'm missing something, we have no way of handling them automatically, the client will have to handle these?
xavidop
left a comment
xavidop
left a comment
There was a problem hiding this comment.
This RFC is heading in a good direction, but I wonder if we can simplify the API further. Introducing defineBidiFlow adds another concept that users need to learn and distinguish from defineFlow, which may increase cognitive load without providing a clearly different mental model.
Would it be possible to support this functionality by extending defineFlow with a configuration option instead? For example, using defineFlow with a specific config to indicate bidirectional or streaming behavior would keep the API surface smaller and make it easier for users to understand what’s going on without having to learn a parallel set of abstractions.
From a usability perspective, having a single defineFlow entry point feels cleaner and more consistent.
3 participants
Introduces bidirectional streaming capabilities to Genkit Actions, Flows, and Models. This allows these primitives to accept a continuous input stream and an initialization payload, in addition to producing an output stream.