gpt-oss 20b support by chochowski · Pull Request #889 · NVIDIA/Model-Optimizer

chochowski · 2026-02-13T11:18:24Z

What does this PR do?

Adds gpt-oss-20b support for puzzle any-model pruning.

Type of change:
new feature

Overview:
adds descriptor, converter and yaml configuration files for expert removal. Introduces slight changes on conversion to account for mxfp4 quantized checkpoint of gpt-oss

Usage

# Add a code snippet demonstrating how to use this

Testing

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed.
Is this change backward compatible?: Yes/No
Did you write any new necessary tests?: Yes/No
Did you add or update any necessary documentation?: Yes/No
Did you update Changelog?: Yes/No

Additional Information

Signed-off-by: mchochowski <mchochowski@nvidia.com>

copy-pr-bot · 2026-02-13T11:18:28Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai · 2026-02-13T11:18:33Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)

main
release/.*
feature/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

modelopt/torch/puzzletron/anymodel/models/gpt_oss_20b/gpt_oss_pruned_to_mxfp4.py

examples/puzzletron/README.md

danielkorzekwa · 2026-02-16T08:04:32Z

examples/puzzletron/configs/gptoss-20b_remove_experts_memory/gptoss-20b.yaml

+  - _self_
+
+puzzle_dir: ???
+descriptor: llama


seems wrong

danielkorzekwa · 2026-02-16T08:05:43Z

...es/puzzletron/configs/gptoss-20b_remove_experts_memory/gptoss-20b_remove_experts_memory.yaml

+# FFN intermediate sizes to search over (heterogeneous architecture)
+# teacher_intermediate_size is 8192, so we use proportionally smaller values
+pruning:
+  intermediate_size_list: [2048, 4096, 6144]


is it needed if we prune for num_of_experts?

danielkorzekwa · 2026-02-16T08:09:17Z

examples/puzzletron/README.md


 Modify `llama-3_1-8B_pruneffn_memory.yaml` file for advanced compression scenarios.
+
+## GptOss - 20b


Let's not put it at the same level as ## Advanced Usage, I would put it into a separate MD file (in the model descriptor dir) and link nicely in the main tutorial. Consult also with @LianaMikael how best to do it. We want the tutorial to have a great user experience.

Let's also check for English style/grammar. E.g. should be no ',' after that. , likely comma after 'n the prunning steps '

modelopt/torch/puzzletron/anymodel/models/gpt_oss_20b/gpt_oss_pruned_to_mxfp4.py

Signed-off-by: mchochowski <mchochowski@nvidia.com>

…uator (NVIDIA#894) This PR adds Nemo Evaluator support to the AnyModel branch. It includes documentation and a deployment script that allow for evaluation of AnyModel Puzzletron checkpoints with Nemo Evaluator. We assume development on a GPU node, following the current tutorial style, so we don't rely on Slurm-based deployment/evaluation, but instead use direct evaluation via `eval-factory run_eval`. --------- Signed-off-by: jrausch <jrausch@nvidia.com>

## What does this PR do? **Overview:** - Update the AnyModel Puzzletron tutorial to use lm-eval. We add a script that monkey patches lm-eval to use the patched AnyModel model loading - No need for running ray deployments or replacing the NeMo Export-Deploy deployment script with a patched version - Moved instructions for using NeMo Evaluator to an alternative readme file --------- Signed-off-by: jrausch <jrausch@nvidia.com>

## What does this PR do? **Overview:** Updated license of examples/puzzletron/evaluation/lm_eval_anymodel.py to match that of reference examples/llm_eval/lm_eval_hf.py. Signed-off-by: jrausch <jrausch@nvidia.com>

Signed-off-by: mchochowski <mchochowski@nvidia.com>

…ml config Signed-off-by: mchochowski <mchochowski@nvidia.com>

kevalmorabia97 · 2026-02-22T15:44:46Z

examples/puzzletron/evaluation/lm_eval_anymodel.py

+# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.


This goes after the EleutherAI license

chochowski added 2 commits February 13, 2026 02:54

gpt-oss 20b support

04e6e41

Signed-off-by: mchochowski <mchochowski@nvidia.com>

added paragraph in readme

5ad56c5

Signed-off-by: mchochowski <mchochowski@nvidia.com>

chochowski requested review from a team as code owners February 13, 2026 11:18

chochowski requested review from jingyu-ml and removed request for a team February 13, 2026 11:18

kevalmorabia97 requested review from danielkorzekwa and kevalmorabia97 and removed request for jingyu-ml February 13, 2026 19:33

kevalmorabia97 reviewed Feb 13, 2026

View reviewed changes

modelopt/torch/puzzletron/anymodel/models/gpt_oss_20b/gpt_oss_pruned_to_mxfp4.py Show resolved Hide resolved

kevalmorabia97 reviewed Feb 13, 2026

View reviewed changes

examples/puzzletron/README.md Outdated Show resolved Hide resolved

danielkorzekwa reviewed Feb 16, 2026

View reviewed changes

chochowski and others added 6 commits February 19, 2026 04:20

fixes to readme, and config yaml. added copyrights header

93cee76

Signed-off-by: mchochowski <mchochowski@nvidia.com>

Update license of lm_eval_anymodel.py (NVIDIA#899)

8d344ec

## What does this PR do? **Overview:** Updated license of examples/puzzletron/evaluation/lm_eval_anymodel.py to match that of reference examples/llm_eval/lm_eval_hf.py. Signed-off-by: jrausch <jrausch@nvidia.com>

fix gptosss hook

296722a

Signed-off-by: mchochowski <mchochowski@nvidia.com>

fix hooks, descriptor and converter to support gptoss. fix typo in ya…

12d46e7

…ml config Signed-off-by: mchochowski <mchochowski@nvidia.com>

chochowski requested review from a team as code owners February 21, 2026 16:32

chochowski requested review from kevalmorabia97 and removed request for a team February 21, 2026 16:32

kevalmorabia97 reviewed Feb 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

gpt-oss 20b support#889

gpt-oss 20b support#889
chochowski wants to merge 8 commits intoNVIDIA:dkorzekwa/any_modelfrom
chochowski:mchochowski/any_model_gptoss

chochowski commented Feb 13, 2026 •

edited

Loading

Uh oh!

copy-pr-bot bot commented Feb 13, 2026

Uh oh!

coderabbitai bot commented Feb 13, 2026 •

edited

Loading

Review skipped

Uh oh!

Uh oh!

Uh oh!

danielkorzekwa Feb 16, 2026

Uh oh!

danielkorzekwa Feb 16, 2026

Uh oh!

danielkorzekwa Feb 16, 2026

Uh oh!

Uh oh!

kevalmorabia97 Feb 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		Modify `llama-3_1-8B_pruneffn_memory.yaml` file for advanced compression scenarios.

		## GptOss - 20b

Comments

Conversation

chochowski commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Usage

Testing

Before your PR is "Ready for review"

Additional Information

Uh oh!

copy-pr-bot bot commented Feb 13, 2026

Uh oh!

coderabbitai bot commented Feb 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Uh oh!

Uh oh!

danielkorzekwa Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

danielkorzekwa Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

danielkorzekwa Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kevalmorabia97 Feb 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

chochowski commented Feb 13, 2026 •

edited

Loading

coderabbitai bot commented Feb 13, 2026 •

edited

Loading