Skip to content

Use new Safetensors packaging #666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ilopezluna merged 9 commits into from
Feb 16, 2026
Merged

Use new Safetensors packaging #666

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ac57eb7
feat(unpack): enhance safetensors model handling with V0.2 packaging …
ilopezluna Feb 12, 2026
f06e39b
feat(unpack): update Safetensors model packaging to support layer-per…
ilopezluna Feb 12, 2026
ff896b5
feat(unpack): update Safetensors model packaging to support individua…
ilopezluna Feb 12, 2026
1f61305
feat(unpack): refine model packaging documentation and streamline com…
ilopezluna Feb 12, 2026
7210222
feat(unpack): add logging for format handler and extraction errors
ilopezluna Feb 13, 2026
7dfa01b
Merge branch 'main' into use-new-packaging
ilopezluna Feb 13, 2026
e2549e3
fix(parse): propagate filesystem errors in model directory walk
ilopezluna Feb 13, 2026
d638a65
fix(parse): update safetensors file walk to use sentinel error for ea…
ilopezluna Feb 13, 2026
3e8c67d
Merge branch 'main' into use-new-packaging
ilopezluna Feb 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 42 additions & 73 deletions cmd/cli/commands/package.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import (
"github.com/docker/model-runner/pkg/distribution/distribution"
"github.com/docker/model-runner/pkg/distribution/oci"
"github.com/docker/model-runner/pkg/distribution/oci/reference"
"github.com/docker/model-runner/pkg/distribution/packaging"
"github.com/docker/model-runner/pkg/distribution/registry"
"github.com/docker/model-runner/pkg/distribution/tarball"
"github.com/docker/model-runner/pkg/distribution/types"
Expand All @@ -40,13 +39,45 @@ func newPackagedCmd() *cobra.Command {

c := &cobra.Command{
Use: "package (--gguf <path> | --safetensors-dir <path> | --dduf <path> | --from <model>) [--license <path>...] [--mmproj <path>] [--context-size <tokens>] [--push] MODEL",
Short: "Package a GGUF file, Safetensors directory, DDUF file, or existing model into a Docker model OCI artifact.",
Long: "Package a GGUF file, Safetensors directory, DDUF file, or existing model into a Docker model OCI artifact, with optional licenses and multimodal projector. The package is sent to the model-runner, unless --push is specified.\n" +
"When packaging a sharded GGUF model, --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).\n" +
"When packaging a Safetensors model, --safetensors-dir should point to a directory containing .safetensors files and config files (*.json, merges.txt). All files will be auto-discovered and config files will be packaged into a tar archive.\n" +
"When packaging a DDUF file (Diffusers Unified Format), --dduf should point to a .dduf archive file.\n" +
"When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.\n" +
"For multimodal models, use --mmproj to include a multimodal projector file.",
Short: "Package a model into a Docker Model OCI artifact",
Long: `Package a model into a Docker Model OCI artifact.

The model source must be one of:
--gguf A GGUF file (single file or first shard of a sharded model)
--safetensors-dir A directory containing .safetensors and configuration files
--dduf A .dduf (Diffusers Unified Format) archive
--from An existing packaged model reference

By default, the packaged artifact is loaded into the local Model Runner content store.
Use --push to publish the model to a registry instead.

MODEL specifies the target model reference (for example: myorg/llama3:8b).
When using --push, MODEL must be a registry-qualified reference.

Packaging behavior:

GGUF
--gguf must point to a .gguf file.
For sharded models, point to the first shard. All shards must:
• reside in the same directory
• follow an indexed naming convention (e.g. model-00001-of-00015.gguf)
All shards are automatically discovered and packaged together.

Safetensors
--safetensors-dir must point to a directory containing .safetensors files
and required configuration files (e.g. model config, tokenizer files).
All files under the directory (including nested subdirectories) are
automatically discovered. Each file is packaged as a separate OCI layer.

DDUF
--dduf must point to a .dduf archive file.

Repackaging
--from repackages an existing model. You may override selected properties
such as --context-size to create a variant of the original model.

Multimodal models
Use --mmproj to include a multimodal projector file.`,
Args: func(cmd *cobra.Command, args []string) error {
if err := requireExactArgs(1, "package", "MODEL")(cmd, args); err != nil {
return err
Expand Down Expand Up @@ -155,17 +186,6 @@ func newPackagedCmd() *cobra.Command {
}
}

// Validate dir-tar paths are relative (not absolute)
for _, dirPath := range opts.dirTarPaths {
if filepath.IsAbs(dirPath) {
return fmt.Errorf(
"dir-tar path must be relative, got absolute path: %s\n\n"+
"See 'docker model package --help' for more information",
dirPath,
)
}
}

return nil
},
RunE: func(cmd *cobra.Command, args []string) error {
Expand All @@ -185,7 +205,6 @@ func newPackagedCmd() *cobra.Command {
c.Flags().StringVar(&opts.fromModel, "from", "", "reference to an existing model to repackage")
c.Flags().StringVar(&opts.chatTemplatePath, "chat-template", "", "absolute path to chat template file (must be Jinja format)")
c.Flags().StringArrayVarP(&opts.licensePaths, "license", "l", nil, "absolute path to a license file")
c.Flags().StringArrayVar(&opts.dirTarPaths, "dir-tar", nil, "relative path to directory to package as tar (can be specified multiple times)")
c.Flags().StringVar(&opts.mmprojPath, "mmproj", "", "absolute path to multimodal projector file")
c.Flags().BoolVar(&opts.push, "push", false, "push to registry (if not set, the model is loaded into the Model Runner content store)")
c.Flags().Uint64Var(&opts.contextSize, "context-size", 0, "context size in tokens")
Expand All @@ -200,7 +219,6 @@ type packageOptions struct {
ddufPath string
fromModel string
licensePaths []string
dirTarPaths []string
mmprojPath string
push bool
tag string
Expand Down Expand Up @@ -274,33 +292,11 @@ func initializeBuilder(ctx context.Context, cmd *cobra.Command, client *desktop.
}
result.builder = pkg
} else if opts.safetensorsDir != "" {
// Safetensors model from directory
// Safetensors model from directory — uses V0.2 layer-per-file packaging
cmd.PrintErrf("Scanning directory %q for safetensors model...\n", opts.safetensorsDir)
safetensorsPaths, tempConfigArchive, err := packaging.PackageFromDirectory(opts.safetensorsDir)
pkg, err := builder.FromDirectory(opts.safetensorsDir)
if err != nil {
return nil, fmt.Errorf("scan safetensors directory: %w", err)
}

// Set up cleanup for temp config archive
if tempConfigArchive != "" {
result.cleanupFunc = func() {
os.Remove(tempConfigArchive)
}
}

cmd.PrintErrf("Found %d safetensors file(s)\n", len(safetensorsPaths))
pkg, err := builder.FromPaths(safetensorsPaths)
if err != nil {
return nil, fmt.Errorf("create safetensors model: %w", err)
}

// Add config archive if it was created
if tempConfigArchive != "" {
cmd.PrintErrf("Adding config archive from directory\n")
pkg, err = pkg.WithConfigArchive(tempConfigArchive)
if err != nil {
return nil, fmt.Errorf("add config archive: %w", err)
}
return nil, fmt.Errorf("create safetensors model from directory: %w", err)
}
result.builder = pkg
} else {
Expand Down Expand Up @@ -354,7 +350,6 @@ func packageModel(ctx context.Context, cmd *cobra.Command, client *desktop.Clien
len(opts.licensePaths) == 0 &&
opts.chatTemplatePath == "" &&
opts.mmprojPath == "" &&
len(opts.dirTarPaths) == 0 &&
cmd.Flags().Changed("context-size")

if canUseDaemonRepackage {
Expand Down Expand Up @@ -456,32 +451,6 @@ func packageModel(ctx context.Context, cmd *cobra.Command, client *desktop.Clien

cmd.PrintErrln("Model variant created successfully")
return nil // Return early to avoid the Build operation in lightweight case
} else {
// Process directory tar archives
if len(opts.dirTarPaths) > 0 {
// Determine base directory for resolving relative paths
var baseDir string
if opts.safetensorsDir != "" {
baseDir = opts.safetensorsDir
} else {
// For GGUF, use the directory containing the GGUF file
baseDir = filepath.Dir(opts.ggufPath)
}

processor := packaging.NewDirTarProcessor(opts.dirTarPaths, baseDir)
tarPaths, cleanup, err := processor.Process()
if err != nil {
return err
}
defer cleanup()

for _, tarPath := range tarPaths {
pkg, err = pkg.WithDirTar(tarPath)
if err != nil {
return fmt.Errorf("add directory tar: %w", err)
}
}
}
}
if opts.push {
cmd.PrintErrln("Pushing model to registry...")
Expand Down
58 changes: 39 additions & 19 deletions cmd/cli/docs/reference/docker_model_package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,44 @@
command: docker model package
short: |
Package a GGUF file, Safetensors directory, DDUF file, or existing model into a Docker model OCI artifact.
short: Package a model into a Docker Model OCI artifact
long: |-
Package a GGUF file, Safetensors directory, DDUF file, or existing model into a Docker model OCI artifact, with optional licenses and multimodal projector. The package is sent to the model-runner, unless --push is specified.
When packaging a sharded GGUF model, --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).
When packaging a Safetensors model, --safetensors-dir should point to a directory containing .safetensors files and config files (*.json, merges.txt). All files will be auto-discovered and config files will be packaged into a tar archive.
When packaging a DDUF file (Diffusers Unified Format), --dduf should point to a .dduf archive file.
When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.
For multimodal models, use --mmproj to include a multimodal projector file.
Package a model into a Docker Model OCI artifact.

The model source must be one of:
--gguf A GGUF file (single file or first shard of a sharded model)
--safetensors-dir A directory containing .safetensors and configuration files
--dduf A .dduf (Diffusers Unified Format) archive
--from An existing packaged model reference

By default, the packaged artifact is loaded into the local Model Runner content store.
Use --push to publish the model to a registry instead.

MODEL specifies the target model reference (for example: myorg/llama3:8b).
When using --push, MODEL must be a registry-qualified reference.

Packaging behavior:

GGUF
--gguf must point to a .gguf file.
For sharded models, point to the first shard. All shards must:
• reside in the same directory
• follow an indexed naming convention (e.g. model-00001-of-00015.gguf)
All shards are automatically discovered and packaged together.

Safetensors
--safetensors-dir must point to a directory containing .safetensors files
and required configuration files (e.g. model config, tokenizer files).
All files under the directory (including nested subdirectories) are
automatically discovered. Each file is packaged as a separate OCI layer.

DDUF
--dduf must point to a .dduf archive file.

Repackaging
--from repackages an existing model. You may override selected properties
such as --context-size to create a variant of the original model.

Multimodal models
Use --mmproj to include a multimodal projector file.
usage: docker model package (--gguf <path> | --safetensors-dir <path> | --dduf <path> | --from <model>) [--license <path>...] [--mmproj <path>] [--context-size <tokens>] [--push] MODEL
pname: docker model
plink: docker_model.yaml
Expand Down Expand Up @@ -40,17 +71,6 @@ options:
experimentalcli: false
kubernetes: false
swarm: false
- option: dir-tar
value_type: stringArray
default_value: '[]'
description: |
relative path to directory to package as tar (can be specified multiple times)
deprecated: false
hidden: false
experimental: false
experimentalcli: false
kubernetes: false
swarm: false
- option: from
value_type: string
description: reference to an existing model to repackage
Expand Down
58 changes: 29 additions & 29 deletions cmd/cli/docs/reference/model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,35 +5,35 @@ Docker Model Runner

### Subcommands

| Name | Description |
|:------------------------------------------------|:-----------------------------------------------------------------------------------------------------------|
| [`bench`](model_bench.md) | Benchmark a model's performance at different concurrency levels |
| [`df`](model_df.md) | Show Docker Model Runner disk usage |
| [`inspect`](model_inspect.md) | Display detailed information on one model |
| [`install-runner`](model_install-runner.md) | Install Docker Model Runner (Docker Engine only) |
| [`launch`](model_launch.md) | Launch an app configured to use Docker Model Runner |
| [`list`](model_list.md) | List the models pulled to your local environment |
| [`logs`](model_logs.md) | Fetch the Docker Model Runner logs |
| [`package`](model_package.md) | Package a GGUF file, Safetensors directory, DDUF file, or existing model into a Docker model OCI artifact. |
| [`ps`](model_ps.md) | List running models |
| [`pull`](model_pull.md) | Pull a model from Docker Hub or HuggingFace to your local environment |
| [`purge`](model_purge.md) | Remove all models |
| [`push`](model_push.md) | Push a model to Docker Hub or Hugging Face |
| [`reinstall-runner`](model_reinstall-runner.md) | Reinstall Docker Model Runner (Docker Engine only) |
| [`requests`](model_requests.md) | Fetch requests+responses from Docker Model Runner |
| [`restart-runner`](model_restart-runner.md) | Restart Docker Model Runner (Docker Engine only) |
| [`rm`](model_rm.md) | Remove local models downloaded from Docker Hub |
| [`run`](model_run.md) | Run a model and interact with it using a submitted prompt or chat mode |
| [`search`](model_search.md) | Search for models on Docker Hub and HuggingFace |
| [`show`](model_show.md) | Show information for a model |
| [`skills`](model_skills.md) | Install Docker Model Runner skills for AI coding assistants |
| [`start-runner`](model_start-runner.md) | Start Docker Model Runner (Docker Engine only) |
| [`status`](model_status.md) | Check if the Docker Model Runner is running |
| [`stop-runner`](model_stop-runner.md) | Stop Docker Model Runner (Docker Engine only) |
| [`tag`](model_tag.md) | Tag a model |
| [`uninstall-runner`](model_uninstall-runner.md) | Uninstall Docker Model Runner (Docker Engine only) |
| [`unload`](model_unload.md) | Unload running models |
| [`version`](model_version.md) | Show the Docker Model Runner version |
| Name | Description |
|:------------------------------------------------|:-----------------------------------------------------------------------|
| [`bench`](model_bench.md) | Benchmark a model's performance at different concurrency levels |
| [`df`](model_df.md) | Show Docker Model Runner disk usage |
| [`inspect`](model_inspect.md) | Display detailed information on one model |
| [`install-runner`](model_install-runner.md) | Install Docker Model Runner (Docker Engine only) |
| [`launch`](model_launch.md) | Launch an app configured to use Docker Model Runner |
| [`list`](model_list.md) | List the models pulled to your local environment |
| [`logs`](model_logs.md) | Fetch the Docker Model Runner logs |
| [`package`](model_package.md) | Package a model into a Docker Model OCI artifact |
| [`ps`](model_ps.md) | List running models |
| [`pull`](model_pull.md) | Pull a model from Docker Hub or HuggingFace to your local environment |
| [`purge`](model_purge.md) | Remove all models |
| [`push`](model_push.md) | Push a model to Docker Hub or Hugging Face |
| [`reinstall-runner`](model_reinstall-runner.md) | Reinstall Docker Model Runner (Docker Engine only) |
| [`requests`](model_requests.md) | Fetch requests+responses from Docker Model Runner |
| [`restart-runner`](model_restart-runner.md) | Restart Docker Model Runner (Docker Engine only) |
| [`rm`](model_rm.md) | Remove local models downloaded from Docker Hub |
| [`run`](model_run.md) | Run a model and interact with it using a submitted prompt or chat mode |
| [`search`](model_search.md) | Search for models on Docker Hub and HuggingFace |
| [`show`](model_show.md) | Show information for a model |
| [`skills`](model_skills.md) | Install Docker Model Runner skills for AI coding assistants |
| [`start-runner`](model_start-runner.md) | Start Docker Model Runner (Docker Engine only) |
| [`status`](model_status.md) | Check if the Docker Model Runner is running |
| [`stop-runner`](model_stop-runner.md) | Stop Docker Model Runner (Docker Engine only) |
| [`tag`](model_tag.md) | Tag a model |
| [`uninstall-runner`](model_uninstall-runner.md) | Uninstall Docker Model Runner (Docker Engine only) |
| [`unload`](model_unload.md) | Unload running models |
| [`version`](model_version.md) | Show the Docker Model Runner version |



Expand Down
Loading
Loading