Installation

Last updated: 07/13/2026

Requirements

For NVIDIA GPU:

Python: Version >= 3.10
CUDA: Version >= 12.8

For Ascend NPU:

Python: Version >= 3.10
CANN: Version >= 8.5.0

Install

git clone https://github.com/verl-project/verl-omni.git
cd verl-omni

Create a Python virtual environment

uv venv --python 3.12 --seed
source .venv/bin/activate

Install the platform backend

For NVIDIA GPU:

uv pip install -e ".[gpu]" --torch-backend=auto

This installs vllm for the CUDA PyTorch stack and kernels for FA3 backend.

For Ascend NPU:

uv pip install vllm==0.24.0
uv pip install "vllm-ascend @ git+https://github.com/vllm-project/vllm-ascend.git@$(cat .github/vllm_ascend_pin.txt)"

Install vLLM-Omni and VeRL-Omni

uv pip install "vllm-omni @ git+https://github.com/vllm-project/vllm-omni.git@$(cat .github/vllm_omni_pin.txt)"
uv pip install -e ".[train]"

This installs vllm-omni, then verl and verl-omni.

Extras

Extra	Adds	When
`gpu`	`vllm==0.24.0`, `kernels==0.14.1`, `liger-kernel`	CUDA rollout + actor FA3
`vllm-omni`	`vllm-omni==0.24.0`	Optional PyPI baseline only; CI/docs use the git pin above
`train`	`verl` @ `.github/verl_pin.txt`	RL training
`dev`	`pytest`, `pre-commit`, `Levenshtein`, …	Local development / CI
`ocr`	`Levenshtein`	OCR reward (FlowGRPO)

Optional Dependencies

Extra	Install	When needed
OCR reward	`uv pip install -e ".[ocr]"`	FlowGRPO training with OCR-based reward
Multimodal training	`pip install qwen-vl-utils math-verify`	Vision-language training (e.g. MMK12)
Dev tools	`uv pip install -e ".[dev]"`	Linting and unit tests
VeOmni engine backend	See Optional engine backends	VeOmni instead of default FSDP2

Flash Attention 3

The gpu extra pulls kernels==0.14.1 for Diffusers actor FA3 (attn_backend=_flash_3_varlen_hub). Defaults pair actor and rollout on the same Hub kernel backend:

actor_rollout_ref.model.attn_backend=_flash_3_varlen_hub
actor_rollout_ref.rollout.rollout_attn_backend=FLASH_ATTN_3_HUB

FLASH_ATTN_3_HUB is provided by vLLM-Omni (kernels-community/flash-attn3). The legacy FLASH_ATTN rollout path still uses local FA packages (fa3-fwd / flash-attn).

If FA3 deps are missing at runtime, training falls back to native/SDPA automatically. NPU recipes override with actor_rollout_ref.model.attn_backend=_native_npu.

Optional engine backends

VeRL-Omni defaults to FSDP2 as the training engine for the policy and reference models. The diffusion trainer can alternatively be switched to VeOmni. The engine is selected at the Hydra command line — see examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_veomni.sh for a complete recipe.

Installing VeOmni alongside vLLM 0.24.0

VeOmni 0.1.11’s gpu extra pins torch==2.9.1+cu129, which may conflict with the torch version pulled in by vllm==0.24.0. A plain uv pip install veomni[gpu,dit]==0.1.11 therefore fails dependency resolution.

VeOmni itself runs correctly on torch 2.11 — only the [gpu] extra’s pin is too strict. Install it without dependency resolution so the existing torch/vllm stack is preserved, and add the small set of runtime extras that the verl-omni VeOmni engine actually needs:

uv pip install veomni==0.1.11 --no-deps
uv pip install torchcodec librosa soundfile av

Verify the engine is importable:

python -c "import veomni; print('veomni', veomni.__version__)"
python -c "from veomni.distributed.offloading import load_model_to_gpu, load_optimizer, offload_model_to_cpu, offload_optimizer; print('VeOmni offloading helpers OK')"

If you want VeOmni’s full [gpu,dit] extras (flash-attn variants, liger-kernel, cuda-python, etc.), install them in a separate environment not pinned to vllm 0.24.0; verl-omni does not need them.

Post-Installation Verification

For NVIDIA GPU:

python -c "import torch; print('torch', torch.__version__, '| CUDA', torch.version.cuda)"
python -c "import vllm; print('vllm', vllm.__version__)"
python -c "import vllm_omni; print('vllm-omni OK')"
python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"

For Ascend NPU:

python -c "import torch; import torch_npu; print('torch', torch.__version__, '| NPU', torch.npu.is_available())"
python -c "import vllm; print('vllm', vllm.__version__)"
python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"

Build Your Own Docker Image

The repository provides Dockerfiles for both NVIDIA GPU and Ascend NPU environments:

CUDA Dockerfile: docker/Dockerfile.cuda
Ascend Atlas A2 NPU Dockerfile: docker/Dockerfile.a2.npu
Ascend Atlas A3 NPU Dockerfile: docker/Dockerfile.a3.npu

The CUDA image is intended for NVIDIA GPU training and rollout. The default CUDA base image uses CUDA 13.0.2 on Ubuntu 22.04. You can override the CUDA version with --build-arg CUDA_VERSION=... if needed.

The NPU images are split by Ascend hardware generation: Dockerfile.a2.npu is intended for Ascend 910B / Atlas A2, and Dockerfile.a3.npu is intended for Ascend Atlas A3. Both NPU images include CANN, torch-npu, vllm-ascend, and vllm-omni.

Build context is controlled by the repo-root .dockerignore; keep large local folders such as .venv, data/, and checkpoints/ out of the context.

CUDA Docker Image

Prerequisites

Docker with NVIDIA Container Toolkit

Build commands

From the repository root:

# Standard GPU training image (runtime target)
docker build -f docker/Dockerfile.cuda -t verl-omni:gpu .

# OCR reward (adds the `ocr` extra / Levenshtein)
docker build -f docker/Dockerfile.cuda --target ocr -t verl-omni:gpu-ocr .

# Local development tools (adds the `dev` extra)
docker build -f docker/Dockerfile.cuda --target dev -t verl-omni:gpu-dev .

The image bakes in verl_omni and its Python dependencies. Recipe scripts under examples/ are not copied into the image — mount the repository at runtime.

Launch with interactive session for development

Start an interactive shell with GPU access, shared memory for Ray/vLLM, and common host directories mounted:

export REPO=/path/to/verl-omni          # this repository
export WORKSPACE=$HOME                  # data, checkpoints, HF cache root

docker run --gpus all --shm-size=16g -it --rm \
  --name verl-omni-ocr \
  -v "$REPO:/workspace/verl-omni" \
  -v "$WORKSPACE/data:$WORKSPACE/data" \
  -v "$WORKSPACE/checkpoints:$WORKSPACE/checkpoints" \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -e WORKSPACE="$WORKSPACE" \
  -e HF_HOME=/root/.cache/huggingface \
  -e WANDB_API_KEY="${WANDB_API_KEY:-}" \
  -w /workspace/verl-omni \
  verl-omni:gpu-ocr \
  /bin/bash

Inside the container, confirm the installation using the same checks as Post-Installation Verification.

Notes:

--shm-size=16g — Ray and vLLM use shared memory; larger shared memory is needed for training.
Mount the repo — training recipes live in examples/; mounting $REPO lets you edit scripts locally and run them immediately in the container.
WORKSPACE — example scripts read datasets and write checkpoints under this path. The default is $HOME inside the container, i.e. /root unless overridden.
Hugging Face cache — mounting ~/.cache/huggingface avoids re-downloading Qwen/Qwen-Image and reward models on every run.

Ascend NPU Docker Image

Prerequisites

The Ascend NPU Docker image expects the host machine to provide the Ascend driver and device files.

Before launching the container, make sure the host has:

Ascend driver installed.
CANN-compatible runtime environment.
npu-smi available on the host.
Ascend device nodes under /dev, such as /dev/davinci0, /dev/davinci_manager, /dev/devmm_svm, and /dev/hisi_hdc.
Docker permission to pass NPU devices into the container.

The NPU container mounts the host driver directory:

-v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro

This allows the containerized CANN / torch-npu runtime to use the host Ascend driver.

Build commands

From the repository root, choose the Dockerfile that matches your Ascend hardware.

For Ascend Atlas A3:

docker build \
  -f docker/Dockerfile.a3.npu \
  -t verl-omni:npu-a3 \
  .

For Ascend Atlas A2 / 910B:

docker build \
  -f docker/Dockerfile.a2.npu \
  -t verl-omni:npu-a2 \
  .

When debugging dependency installation or making sure no old Docker layer is reused, add --no-cache:

# Atlas A3
docker build --no-cache \
  -f docker/Dockerfile.a3.npu \
  -t verl-omni:npu-a3 \
  .

# Atlas A2 / 910B
docker build --no-cache \
  -f docker/Dockerfile.a2.npu \
  -t verl-omni:npu-a2 \
  .

You may choose different image tags locally. If you do so, replace the image name in the docker run command accordingly.

Launch on Ascend Atlas A3, 16 NPU

Use this command on a 16-card Ascend Atlas A3 machine:

DEVICES=""
for i in $(seq 0 15); do
  DEVICES="$DEVICES --device=/dev/davinci$i"
done

docker run -it --rm \
  --name verl_omni_16npu \
  --network host \
  --ipc host \
  $DEVICES \
  --device=/dev/davinci_manager \
  --device=/dev/devmm_svm \
  --device=/dev/hisi_hdc \
  -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \
  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
  -v /mnt/data:/mnt/data \
  verl-omni:npu-a3 \
  bash

Launch on Ascend Atlas A2 / 910B, 8 NPU

Use this command on an 8-card Ascend Atlas A2 / 910B machine:

DEVICES=""
for i in $(seq 0 7); do
  DEVICES="$DEVICES --device=/dev/davinci$i"
done

docker run -it --rm \
  --name verl_omni_8npu \
  --network host \
  --ipc host \
  $DEVICES \
  --device=/dev/davinci_manager \
  --device=/dev/devmm_svm \
  --device=/dev/hisi_hdc \
  -v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi:ro \
  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
  -v /home:/home \
  verl-omni:npu-a2 \
  bash

Notes for NPU containers

--network host — useful for Ray, distributed training, and multi-process communication.
--ipc host — avoids shared-memory limitations during training and rollout.
/dev/davinci* devices — expose Ascend NPU cards to the container.
/dev/davinci_manager, /dev/devmm_svm, and /dev/hisi_hdc — required Ascend runtime device files.
/usr/local/Ascend/driver — mounted read-only from the host so the container can use the installed Ascend driver.
npu-smi — mounted from the host to inspect device status inside the container.
Atlas A3 16 NPU — exposes /dev/davinci0 through /dev/davinci15.
Atlas A2 / 910B 8 NPU — exposes /dev/davinci0 through /dev/davinci7.

Inside the container, confirm the NPU environment:

npu-smi info
python -c "import torch; import torch_npu; print('torch', torch.__version__, '| NPU', torch.npu.is_available())"
python -c "import vllm; print('vllm', vllm.__version__)"
python -c "import verl; print('verl', verl.__version__)"
python -c "import verl_omni; print('VeRL-Omni ready')"

Example: Qwen-Image FlowGRPO training in Docker

This walkthrough follows the FlowGRPO quickstart using the OCR dataset and examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh.

For CUDA, use the ocr image target (verl-omni:gpu-ocr) so the Levenshtein dependency is present.

For Ascend NPU, use the NPU image and the NPU-specific recipe options. NPU recipes should override the attention backend with:

actor_rollout_ref.model.attn_backend=_native_npu

1. Launch the interactive container

Use either the CUDA or NPU launch command above.

2. Prepare the OCR dataset inside the container

export WORKSPACE=${WORKSPACE:-$HOME}
mkdir -p $WORKSPACE/data/ocr

# Obtain raw train.txt / test.txt from the Flow-GRPO repo:
# https://github.com/yifan123/flow_grpo/tree/main/dataset/ocr
# Place them under $WORKSPACE/data/ocr/, then preprocess:

python3 examples/flowgrpo_trainer/data_process/qwenimage_ocr.py \
  --input_dir $WORKSPACE/data/ocr \
  --output_dir $WORKSPACE/data/ocr/qwen_image

3. Optional: Set W&B credentials

export WANDB_API_KEY=<your_wandb_api_key>

4. Run FlowGRPO training

For CUDA, the default OCR LoRA script uses 4 GPUs by default:

bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora.sh

For Ascend NPU, use the corresponding NPU recipe script if available in your checkout:

bash examples/flowgrpo_trainer/qwen_image/run_qwen_image_ocr_lora_npu.sh

If you run the training command manually, make sure to include the NPU-specific Hydra override:

actor_rollout_ref.model.attn_backend=_native_npu

The script launches python3 -m verl_omni.trainer.main_diffusion with FlowGRPO + vllm_omni rollout and OCR reward (compute_score_ocr). Checkpoints are written to:

checkpoints/flow_grpo/qwen_image_ocr_lora