Rollout & Agent Loop

Last updated: Jun 05, 2026 (API docstrings are auto-generated).

VeRL-Omni rollout is built on top of vLLM-Omni for concurrent diffusion and multimodal generation. The agent loop streams per-sample generation requests to one or more rollout replicas, optionally fanning reward computation into asynchronous reward-loop workers.

Diffusion Agent Loop

Rollout Replica

vLLM-Omni Async Server

The async server adapters wire the verl.workers.rollout.vllm_rollout.vllm_async_server.vLLMHttpServer and verl.workers.rollout.vllm_rollout.vllm_async_server.vLLMReplica classes to vLLM-Omni’s diffusion-aware backend:

• verl_omni.workers.rollout.vllm_rollout.vllm_omni_async_server.vLLMOmniHttpServer: subclass of vLLM’s HTTP server that swaps the model config for DiffusionModelConfig, skips LLM-only validation, and exposes a PIL → tensor converter for image responses.

• verl_omni.workers.rollout.vllm_rollout.vllm_omni_async_server.vLLMOmniReplica: Ray actor wrapper that boots a vLLM-Omni engine per replica and forwards generation / weight-update / sleep & resume RPCs from the trainer.

These classes are heavy-weight wrappers that depend on running vLLM-Omni at import time, so they are not introspected by autodoc. See verl_omni/workers/rollout/vllm_rollout/vllm_omni_async_server.py for full source.

vLLM-Omni Utilities