Welcome to VeRL-Omni’s documentation!

Last updated: 07/20/2026

VeRL-Omni is a general RL training framework focused on multimodal generative models, built on top of verl. It originated from the multi-modal generation RL effort in verl, and now has a dedicated home so it can evolve in a more focused way.

Scope

VeRL-Omni targets RL post-training for three families of generative models:

  1. Diffusion generative models for image, video, and audio — e.g., Qwen-Image, Wan2.2.

  2. Unified multimodal understanding + generation models — e.g., BAGEL, HunyuanImage-3.0.

  3. Omni-modality models that jointly handle text, image, audio, and video — e.g., Qwen3-Omni.

Key capabilities

  • Specialized rollout via vLLM-Omni for high-throughput diffusion and multimodal generation.

  • Flexible reward pipelines spanning rule-based rewards, model-based rewards, and multimodal reward computation.

  • Modular training backends that plug into existing parallelism (FSDP, USP) and other optimizations rather than rebuilding the stack from scratch.

  • End-to-end examples and benchmarks validating co-located sync and fully-async RL on the model families above.

  • High training throughput — on our reference Qwen-Image FlowGRPO setup, VeRL-Omni achieves up to ~25% higher end-to-end throughput than the diffusers-based flow_grpo reference implementation, driven by vLLM-Omni rollout, FSDP/USP training, and asynchronous reward computation on a dedicated GPU pool.

See Supported Models for the full model catalogue and which algorithms run on each model.

Getting Started

Examples

Advanced Features

Algorithms

Examples

Performance Tuning Guide

Hardware Support

API Reference

Developer Guide

Contribution

VeRL-Omni is free software; you can redistribute it and/or modify it under the terms of the Apache License 2.0. We welcome contributions. Join us on GitHub for discussions.

See the 2026 Q2 roadmap for planned work.

Code Linting and Formatting

We use pre-commit to help improve code quality. To initialize pre-commit, run:

pip install pre-commit
pre-commit install

To resolve CI errors locally, you can also manually run pre-commit by:

pre-commit run

Adding CI tests

If possible, please add CI test(s) for your new feature. See Testing Guide for the L1-L4 test taxonomy, placement rules, and coverage policy.

Pick the most relevant workflow from .github/workflows/:

Workflow

When to use

cpu_unit_tests.yml

New tests that run without a GPU (file name must end with _on_cpu.py)

gpu_smoke.yml

GPU-requiring tests for trainer, worker, rollout, or agent-loop changes

sanity.yml

Static / import-level checks under tests/special_sanity/

Steps:

  1. Place your test file in the appropriate directory under tests/ (e.g. tests/trainer/, tests/workers/, tests/agent_loop/).

  2. Open the chosen workflow yml and add any missing path patterns to its paths section so the workflow triggers on your changes.

  3. Keep the test as lightweight as possible — use small models, reduced steps, and CPU where feasible (see existing *_on_cpu.py scripts for examples).

For GPU smoke tests, see GPU Smoke Tests for how to register tests in the right group and run them locally.

Triggering CI on pull requests

Most PR workflows are label-driven. Add a label whose name contains ci to run the matching checks:

Label

Effect

ci-core

Core GPU smoke (2 GPUs) (training, reward, rollout modules)

ci-e2e-omni

Omni trainer e2e GPU smoke (2 GPUs)

ci-e2e-diffusion

Diffusion trainer e2e GPU smoke (4 GPUs)

ready-for-ci

Selective GPU smoke suite in parallel (Up to 8 GPUs)

Labels are removed automatically when new commits are pushed; re-apply the label after each update.