VeRL-Omni

Getting Started

  • Installation
  • Installation (NPU)
  • Supported Models
  • Quickstart: FlowGRPO training on OCR dataset
  • Multi-Node Training
  • Training Metrics

Configuration

  • Config Explanation

Advanced Features

  • Async Reward for Diffusion Training
  • Rollout Correction for Diffusion Training (Experimental)
  • Separate-Async RL Training for Qwen3-Omni
  • Diffusion Rollout Batching
  • Using an External HTTP Scorer Service
  • Diffusion V1 training
  • Monitor Training with RL-Insight

Algorithms

  • Flow-GRPO
  • Flow-DPPO
  • Diffusion-DPO
  • DiffusionNFT
  • GRPO-Guard
  • Mix-GRPO
  • Diffusion On-Policy Distillation
  • Qwen3-Omni On-Policy Distillation
  • Deterministic Post-Training
  • Performance Reference

Examples

  • FlowGRPO Trainer
  • FlowDPPO Trainer
  • DPO Training
  • Qwen3-Omni Thinker DAPO Trainer
  • DanceGRPO Trainer
  • FLUX.1-dev DanceGRPO
  • DiffusionNFT Trainer
  • GRPO-Guard Trainer
  • Qwen3-Omni Thinker GSPO Trainer
  • MixGRPO Trainer
  • Diffusion On-Policy Distillation Trainer
  • SD3.5 FlowGRPO training with a latent reward model
  • BAGEL-7B-MoT FlowGRPO training
  • Qwen3-TTS GRPO with an audio reward
  • Qwen-Image FlowGRPO
  • Qwen-Image-Edit-2511 FlowGRPO training
  • LTX-2.3 text-to-audio-video FlowGRPO
  • MiniMax H3 T2VA, FL2VA, and Ref2VA DiffusionNFT
  • Train Boogu-Image with FlowGRPO
  • MiniMax H3 T2VA, FL2VA, and Ref2VA FlowGRPO

Performance Tuning Guide

  • Diffusion FLOPs / MFU
  • Profiling FlowGRPO / diffusion training in VeRL-Omni

Hardware Support

  • Quickstart: FlowGRPO training on Qwen-Image OCR dataset with Ascend NPU

API Reference

  • Trainer Interface
  • Workers Interface
  • Rollout & Agent Loop
  • Reward Interface
  • Pipelines Interface
  • Utilities

Developer Guide

  • Editing Agent Instructions
  • CI/CD Layers
  • Testing Guide
  • Integrating the Diffusion Prompt-Embedding Cache
  • How to Add a New Omni Model
  • How to Integrate a New Diffusion Model for FlowGRPO Training
  • How to Integrate an Image-to-Image Diffusion Model
  • How to Integrate a Non-Diffusers Model for FlowGRPO Training
  • How to Add Continuous Batching (Step-Execution) Support for a Diffusion Model
  • How to Integrate a New Policy-Gradient Algorithm for Diffusion Model
  • How to Integrate a New Direct-Preference Algorithm for Diffusion Model
  • GPU Smoke Tests
  • Common Pitfalls

Community

  • Governance
VeRL-Omni
  • Overview: module code

All modules for which code is available

  • verl_omni.agent_loop.diffusion_agent_loop
  • verl_omni.agent_loop.single_turn_agent_loop
  • verl_omni.pipelines.model_base
  • verl_omni.pipelines.qwen_image_flow_grpo.diffusers_training_adapter
  • verl_omni.pipelines.qwen_image_flow_grpo.vllm_omni_rollout_adapter
  • verl_omni.pipelines.rollout_media
  • verl_omni.pipelines.schedulers.flow_match_sde
  • verl_omni.utils.dataset.rl_dataset
  • verl_omni.utils.fs
  • verl_omni.utils.fsdp_utils
  • verl_omni.utils.reward_score
    • verl_omni.utils.reward_score.audio_http_scorer_client
    • verl_omni.utils.reward_score.genrm_ocr
    • verl_omni.utils.reward_score.http_scorer_client
    • verl_omni.utils.reward_score.jpeg_compressibility
    • verl_omni.utils.reward_score.reward_utils
    • verl_omni.utils.reward_score.unified_reward
  • verl_omni.utils.tracking
  • verl_omni.utils.vllm_omni.utils
  • verl_omni.workers.config.diffusion.actor
  • verl_omni.workers.config.diffusion.model
  • verl_omni.workers.config.diffusion.rollout
  • verl_omni.workers.utils.padding

© Copyright 2026 Bytedance Ltd. and/or its affiliates.

Built with Sphinx using a theme provided by Read the Docs.