Welcome to VeRL-Omni’s documentation!

Last updated: 05/29/2026

VeRL-Omni is a general RL training framework focused on multimodal generative models, built on top of verl. It originated from the multi-modal generation RL effort in verl, and now has a dedicated home so it can evolve in a more focused way.

Scope

VeRL-Omni targets RL post-training for three families of generative models:

1. Diffusion generative models for image, video, and audio — e.g., Qwen-Image, Wan2.2.

2. Unified multimodal understanding + generation models — e.g., BAGEL, HunyuanImage-3.0.

3. Omni-modality models that jointly handle text, image, audio, and video — e.g., Qwen3-Omni.

Key capabilities

• Specialized rollout via vLLM-Omni for high-throughput diffusion and multimodal generation.

• Flexible reward pipelines spanning rule-based rewards, model-based rewards, and multimodal reward computation.

• Modular training backends that plug into existing parallelism (FSDP, USP) and other optimizations rather than rebuilding the stack from scratch.

• End-to-end examples and benchmarks validating co-located sync and fully-async RL on the model families above.

• High training throughput — on our reference Qwen-Image FlowGRPO setup, VeRL-Omni achieves up to ~25% higher end-to-end throughput than the diffusers-based flow_grpo reference implementation, driven by vLLM-Omni rollout, FSDP/USP training, and asynchronous reward computation on a dedicated GPU pool.

Getting Started

• Installation
  • Requirements
  • Install
  • Optional Dependencies
  • Optional engine backends
  • Post-Installation Verification
• Quickstart: FlowGRPO training on Qwen-Image OCR dataset
  • Introduction
  • Prerequisite
  • Dataset Introduction
  • Step 1: Prepare the dataset
  • Step 2: Obtain models for RL training
  • Step 3: Perform FlowGRPO training
  • FAQ: tuning OOM-related parameters
  • Wandb logging
  • Further reading
• Training Metrics

Advanced Features

• Rollout Correction for Diffusion Training (Experimental)
• Using an External HTTP Scorer Service

Algorithms

• Flow-GRPO
• DiffusionNFT
• GRPO-Guard
• Mix-GRPO
• Performance Reference

Performance Tuning Guide

• Diffusion FLOPs / MFU
• Profiling FlowGRPO / diffusion training in VeRL-Omni

Hardware Support

• Quickstart: FlowGRPO training on Qwen-Image OCR dataset with Ascend NPU

API Reference

• Trainer Interface
  • Base Ray Diffusion Trainer
  • Policy Gradient Ray Trainer
  • Direct Preference Ray Trainer
  • Entry Point
  • Diffusion Algorithms
  • Trainer Config
  • Metrics
• Workers Interface
  • Engine Workers
  • Diffusers FSDP Engine
  • Diffusers PPO FSDP Engine
  • LoRA Adapter Mixin
  • Loss Functions
  • Padding Utilities
  • Worker Configs
• Rollout & Agent Loop
  • Diffusion Agent Loop
  • Rollout Replica
  • vLLM-Omni Async Server
  • vLLM-Omni Utilities
• Reward Interface
  • Reward Manager
  • Default Score Dispatcher
  • Built-in Reward Scorers
• Pipelines Interface
  • Model Base
  • Pipeline Helpers
  • Schedulers
  • Built-in Pipelines
• Utilities
  • File System Utilities
  • FSDP Utilities
  • Dataset Utilities
  • Diffusion Padding Utilities
  • vLLM-Omni LoRA Hooks

Developer Guide

• Editing Agent Instructions
• How to Integrate a New Diffusion Model for FlowGRPO Training
• How to Integrate a New Policy-Gradient Algorithm for Diffusion Model
• How to Integrate a New Direct-Preference Algorithm for Diffusion Model
• Common Pitfalls

Contribution

VeRL-Omni is free software; you can redistribute it and/or modify it under the terms of the Apache License 2.0. We welcome contributions. Join us on GitHub for discussions.

See the 2026 Q2 roadmap for planned work.

Code Linting and Formatting

We use pre-commit to help improve code quality. To initialize pre-commit, run:

pip install pre-commit
pre-commit install

To resolve CI errors locally, you can also manually run pre-commit by:

pre-commit run

Adding CI tests

If possible, please add CI test(s) for your new feature. Pick the most relevant workflow from .github/workflows/:

Workflow	When to use
cpu_unit_tests.yml	New tests that run without a GPU (file name must end with _on_cpu.py)
gpu_smoke.yml	GPU-requiring tests for trainer, worker, rollout, or agent-loop changes
gpu_smoke_verl_latest.yml	Same as above, but pinned against the latest verl main (for upstream compatibility)
sanity.yml	Static / import-level checks under tests/special_sanity/

Steps:

1. Place your test file in the appropriate directory under tests/ (e.g. tests/trainer/, tests/workers/, tests/agent_loop/).

2. Open the chosen workflow yml and add any missing path patterns to its paths section so the workflow triggers on your changes.

3. Keep the test as lightweight as possible — use small models, reduced steps, and CPU where feasible (see existing *_on_cpu.py scripts for examples).