Diffusion Rollout Batching

Last updated: 07/22/2026.

vLLM-Omni can batch diffusion rollouts in two different ways. They look similar in config (max_num_seqs) but they are different engines and are mutually exclusive.

Mode	What gets batched	Config switch
Step-wise continuous batching	Denoising steps across in-flight requests	`step_execution=true`
Request-level batching	Whole requests in one transformer forward	`step_execution=false` (default) + `supports_request_batch=True`
Serial	Nothing	`max_num_seqs=1` (or leave engine default when unsupported)

Note

Setting step_execution=true disables request-level packing inside the engine. Do not enable both modes at once.

Which mode should I use?

Prefer the faster mode for your model (benchmark gen / step time).
If tied or unknown and the adapter supports step-wise → use step-wise.
Else if the adapter supports request-level → use request-level.
Else → serial.

How to tell support

Mode	Adapter signal
Step-wise	Implements `prepare_encode`, `denoise_step`, `step_scheduler`, `post_decode`
Request-level	Sets `supports_request_batch = True` and handles `DiffusionRequestBatch` in `forward`

Example-script defaults today

Recipe	Default mode
Qwen-Image FlowGRPO (`run_qwen_image_ocr*.sh`)	Step-wise (`step_execution=true`, `max_num_seqs=256`)
SD3.5 FlowGRPO (`run_sd35_medium_ocr_lora.sh`)	Request-level (`step_execution=false`, `max_num_seqs=256`, `wait_ms=10`)

On Qwen-Image FlowGRPO e2e LoRA (32×16, 512²) the two modes were essentially tied (~106–108s gen). SD3.5 currently has request-level support only.

Step-wise continuous batching

Idea

Each request advances one denoising step at a time. The scheduler can mix steps from many requests in the same worker, so long-running generations overlap.

req A: encode → step0 → step1 → … → decode
req B:        encode → step0 → step1 → …
              ↑ steps from A and B can interleave

Enable

actor_rollout_ref.rollout.step_execution=true
++actor_rollout_ref.rollout.engine_kwargs.vllm_omni.max_num_seqs=256

Knob	Meaning
`step_execution`	Turns on the step lifecycle (`prepare_encode` → steps → `post_decode`).
`max_num_seqs`	Max concurrent requests in the step scheduler.

Implement for a new model

Add the step lifecycle methods to the existing rollout adapter (same algorithm= registration — no separate *_stepwise package). See integrating_a_stepwise_continuous_batching_model.md.

Request-level batching

Idea

Multiple complete requests are packed into one DiffusionRequestBatch and run through a single forward() (one packed transformer batch per denoising step of that fused group).

req A ─┐
req B ─┼─→ DiffusionRequestBatch → forward() → split per request
req C ─┘

Enable

actor_rollout_ref.rollout.step_execution=false
++actor_rollout_ref.rollout.engine_kwargs.vllm_omni.max_num_seqs=256
++actor_rollout_ref.rollout.engine_kwargs.vllm_omni.request_batch_max_wait_ms=10

Knob	Meaning
`step_execution=false`	Required (yaml default).
`max_num_seqs`	Max requests packed per forward. Engine default is effectively serial (`1`) unless raised.
`request_batch_max_wait_ms`	Optional admission wait before the first schedule of a wave. Default `0`; `10` is enough for typical training bursts.

Override with MAX_NUM_SEQS / REQUEST_BATCH_MAX_WAIT_MS in the example scripts, or Hydra as above.

Warning

Request-level packing uses more activation memory than step-wise for the same max_num_seqs. For Qwen-Image e2e LoRA + true CFG at 512², keep max_num_seqs≤32 on the request-level path; larger values can OOM. Step-wise Qwen examples can use 256.

Expected performance (request-level vs serial)

Gen time excludes step 1 (warmup).

Workload	serial (`max_num_seqs=1`)	request-level	Δ gen
Qwen-Image LoRA, 32×16, 512², `max_num_seqs=32`	226.4s	107.9s	−52%
SD3.5 LoRA, 8×8, 384² FA3, `max_num_seqs=256`	25.4s	22.3s	−12%

wait_ms has little effect on these recipes (0 vs 50 differs by ~2% on Qwen-Image). The gain comes from max_num_seqs > 1.

Implement for a new model

Set supports_request_batch = True on the pipeline class.
Accept OmniDiffusionRequest | DiffusionRequestBatch in forward.
Collate / split with helpers in verl_omni/pipelines/request_batch.py.

Details: integrating_a_diffusion_model.md.

Quick comparison

	Step-wise	Request-level
Unit of batching	Denoising step	Whole request
Adapter API	Step lifecycle methods	Batched `forward()`
`step_execution`	`true`	`false`
Memory shape	Usually lighter concurrency	Heavier packed forwards
Best when	Step methods exist and match or beat request-level	Step methods missing, or request packing is faster