diff --git a/docs/about/changelog.md b/docs/about/changelog.md index a26fe7f..9c26c96 100644 --- a/docs/about/changelog.md +++ b/docs/about/changelog.md @@ -13,6 +13,10 @@ Major features and changes are noted here. To review all updates, see the Related: [Upgrade CodeGate](../how-to/install.md#upgrade-codegate) +- **Request type muxing** - 26 Feb, 2025\ + Workspace model muxing now supports filtering based on chat and FIM request + types. Check the [model muxing docs](../features/muxing.mdx) to learn more. + - **New integration: Open Interpreter** - 20 Feb, 2025\ CodeGate v0.1.24 adds support for the [avante.nvim](https://github.com/yetone/avante.nvim) plugin for Neovim with diff --git a/docs/features/muxing.mdx b/docs/features/muxing.mdx index 09a4903..583c3f2 100644 --- a/docs/features/muxing.mdx +++ b/docs/features/muxing.mdx @@ -15,10 +15,10 @@ especially useful when you're working on multiple projects or tasks that require different AI models. In each of your CodeGate workspaces, you can select the AI provider and model -combinations to use, even dynamically switching the active model based on file -types found in your prompt. Then, configure your AI coding tool to use the -CodeGate muxing endpoint `http://localhost:8989/v1/mux` as an OpenAI-compatible -API provider. +combinations to use, even dynamically switching the active model based on the +request type and file types found in your prompt. Then, configure your AI coding +tool to use the CodeGate muxing endpoint `http://localhost:8989/v1/mux` as an +OpenAI-compatible API provider. To change the model(s) currently in use, simply switch your active CodeGate workspace. @@ -38,9 +38,11 @@ flowchart LR CodeGate --> WS1 CodeGate --> WS2 CodeGate --> WS3 - WS1 --> |api| LLM1 - WS2 --> |api| LLM2 - WS3 --> |api| LLM3 + WS1 --> |FIM requests| LLM1 + WS1 --> |Chat| LLM2 + WS2 --> |.md files| LLM2 + WS2 --> |.js files| LLM3 + WS3 --> |All prompts| LLM3 ``` ## Use cases @@ -109,21 +111,33 @@ workspace selection menu or the In the **Model Muxing** section, select the default ("catch-all") model to use with the workspace. -To assign a different model based on filename, click **Add Filter**. In the -**Filter by** column, enter a file name or extension string to match. This is a -simple substring match, wildcards are not supported. For example, to match -Python files, enter `.py`. Then select the model to use with that file type. +To assign a different model based on request type or filename, click **Add +Filter**. + +In the **Request Type** column, select the type of prompt to match: + +- `FIM & Chat` matches all prompt types +- `FIM` matches fill-in-the-middle (completion) requests +- `Chat` matches chat prompts + +In the **Filter by** column, enter a file name or extension string to match. +This is an exact or `endswith` match, wildcards are not supported. For example, +to match Python files, enter `.py`. + +Finally, select the model to use for prompts that match the rule. Filter rules are evaluated top-down. CodeGate selects the active model for a request using the first matching rule. If the prompt contains multiple files in context, the first rule that matches _any_ of the files is used. If no filter is matched, the catch-all rule applies. +#### Example configuration +