Skip to content

Conversation

@yiyixuxu
Copy link
Collaborator

@yiyixuxu yiyixuxu commented Jan 25, 2026

This PR adds support for explicitly defining and retrieving workflows (currently only added to SequentialPipelineBlocks).

Previously, to make sure users understand what workflows a pipeline supports, we:

WIth this PR, workflows are now:

  • explicitly defined via _workflow_map on the auto blocks class, see qwen example
class QwenImageAutoBlocks(SequentialPipelineBlocks):
    ...
    _workflow_map = {
        "text2image": {"prompt": True},
        "image2image": {"prompt": True, "image": True},
        "inpainting": {"prompt": True, "mask_image": True, "image": True},
        "controlnet_text2image": {"prompt": True, "control_image": True},
        "controlnet_image2image": {"prompt": True, "image": True, "control_image": True},
        "controlnet_inpainting": {"prompt": True, "mask_image": True, "image": True, "control_image": True},
    }
   ...
  • Auto-documented in the class docstring (in qwen example with the _workflow_map we defined above, the auto docstring look like this)
# auto_docstring
class QwenImageAutoBlocks(SequentialPipelineBlocks):
    """
    Auto Modular pipeline for text-to-image, image-to-image, inpainting, and controlnet tasks using QwenImage.

      Supported workflows:
        - `text2image`: requires `prompt`
        - `image2image`: requires `prompt`, `image`
        - `inpainting`: requires `prompt`, `mask_image`, `image`
        - `controlnet_text2image`: requires `prompt`, `control_image`
        - `controlnet_image2image`: requires `prompt`, `image`, `control_image`
        - `controlnet_inpainting`: requires `prompt`, `mask_image`, `image`, `control_image`

      Components:
          ...

      Inputs:
          ...

      Outputs:
          ...
    """
  • Programmatically accessible via workflow_names and get_workflow()
    for example, to get the "controlnet_text2image" workflow from qwen
from diffusers import ModularPipeline
pipe = ModularPipeline.from_pretrained("Qwen/Qwen-Image")
text2image_blocks = pipe.blocks.get_workflow("controlnet_image2image"")
print(text2image_blocks.sub_blocks)
InsertableDict([
  0: ('text_encoder', <obj 'diffusers.modular_pipelines.qwenimage.encoders.QwenImageTextEncoderStep'>),
  1: ('controlnet_vae_encoder', <obj 'diffusers.modular_pipelines.qwenimage.encoders.QwenImageControlNetVaeEncoderStep'>),
  2: ('denoise.input', <obj 'diffusers.modular_pipelines.qwenimage.inputs.QwenImageTextInputsStep'>),
  3: ('denoise.controlnet_input', <obj 'diffusers.modular_pipelines.qwenimage.inputs.QwenImageControlNetInputsStep'>),
  4: ('denoise.prepare_latents', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImagePrepareLatentsStep'>),
  5: ('denoise.set_timesteps', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageSetTimestepsStep'>),
  6: ('denoise.prepare_rope_inputs', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageRoPEInputsStep'>),
  7: ('denoise.controlnet_before_denoise', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageControlNetBeforeDenoiserStep'>),
  8: ('denoise.controlnet_denoise', <obj 'diffusers.modular_pipelines.qwenimage.denoise.QwenImageControlNetDenoiseStep'>),
  9: ('denoise.after_denoise', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageAfterDenoiseStep'>),
  10: ('decode.decode', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageDecoderStep'>),
  11: ('decode.postprocess', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageProcessImagesOutputStep'>)
])

I will add a test suit for this too!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yiyixuxu yiyixuxu changed the title [Modular] support explicit workflow [Modular] add explicit workflow support Jan 25, 2026
Base automatically changed from modular-doc-improv to main January 27, 2026 03:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants