Rollout & Agent Loop

Last updated: Oct 13, 2026 (API docstrings are auto-generated).

VeRL-Omni rollout is built on top of vLLM-Omni for concurrent diffusion and multimodal generation. The agent loop streams per-sample generation requests to one or more rollout replicas, optionally fanning reward computation into asynchronous reward-loop workers.

Diffusion Agent Loop

Variable-Length Reference Rows

Diffusion pipelines may return per-sample condition_video_rows and condition_audio_rows with different lengths. The Agent Loop validates each condition_*_row_count, pads both fields to max_prompt_embed_length, and emits condition_*_rows_mask tensors. This fixed transport shape lets the standard verl.experimental.agent_loop.AgentLoopManager concatenate outputs from multiple rollout workers without model-specific manager logic.

Before Actor training, verl_omni.workers.utils.padding.embeds_padding_2_no_padding() removes the global padding and stores the rows as jagged tensors. The DiffusionNFT engine restores only the minibatch-local maximum immediately before model input. This path raises an error for invalid tensor shapes, count or mask mismatches, or row counts above max_prompt_embed_length; it never truncates reference rows silently.

Rollout Replica

vLLM-Omni Async Server

The async server adapters wire the verl.workers.rollout.vllm_rollout.vllm_async_server.vLLMHttpServer and verl.workers.rollout.vllm_rollout.vllm_async_server.vLLMReplica classes to vLLM-Omni’s diffusion-aware backend:

• verl_omni.workers.rollout.vllm_rollout.vllm_omni_async_server.vLLMOmniHttpServer: subclass of vLLM’s HTTP server that swaps the model config for DiffusionModelConfig, skips LLM-only validation, and exposes a PIL → tensor converter for image responses.

• verl_omni.workers.rollout.vllm_rollout.vllm_omni_async_server.vLLMOmniReplica: Ray actor wrapper that boots a vLLM-Omni engine per replica and forwards generation / weight-update / sleep & resume RPCs from the trainer.

These classes are heavy-weight wrappers that depend on running vLLM-Omni at import time, so they are not introspected by autodoc. See verl_omni/workers/rollout/vllm_rollout/vllm_omni_async_server.py for full source.

vLLM-Omni Utilities

class verl_omni.utils.vllm_omni.utils.OmniTensorLoRARequest(lora_name: str, lora_int_id: int, lora_path: str = '', base_model_name: str | None = None, tensorizer_config_dict: dict | None = None, load_inplace: bool = False, is_3d_lora_weight: bool = False, peft_config: dict = None, lora_tensors: dict = None)

class verl_omni.utils.vllm_omni.utils.VLLMOmniHijack

Monkey-patches vLLM + vllm-omni internals to support in-memory LoRA tensors.

Applies verl’s base vLLM LoRA hijack (VLLMHijack.hijack()) first, then layers the vllm-omni diffusion-side patches on top, so callers only need a single VLLMOmniHijack.hijack() call.