Qwen-Image-Edit-2511 FlowGRPO training

Last updated: 08/26/2026

This guide shows how to prepare an image-edit dataset and train Qwen-Image-Edit-2511 with LoRA and FlowGRPO.

The current adapter is registered for QwenImageEditPlusPipeline. A replacement checkpoint must use that value in model_index.json::_class_name. The older QwenImageEditPipeline architecture is not supported by this recipe.

For implementation details or support for another image-edit architecture, see How to Integrate an Image-to-Image Diffusion Model.

Prerequisites

The example reward is PickScore. Its model weights are downloaded from Hugging Face the first time the reward workers start.

Prepare the Dataset

The example converter expects the following input layout:

my_image_edit_data/
|-- images/
|   |-- 000001.png
|   `-- 000002.png
|-- train.jsonl
`-- test.jsonl

Each line of train.jsonl and test.jsonl contains an editing instruction and an image path relative to images/:

{"prompt": "Change the background to blue", "image": "000001.png"}

Convert the data to training parquet files:

python examples/flowgrpo_trainer/qwen_image_edit/prepare_data.py \
    --input_dir my_image_edit_data \
    --output_dir data/qwen_image_edit \
    --image_size 512

The command writes:

data/qwen_image_edit/
|-- train.parquet
`-- test.parquet

The converter letterboxes each condition image onto a square canvas and stores it as PNG bytes. The fields relevant to training follow this structure:

{
    "prompt": [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Picture 1: <image>Change the background to blue"},
    ],
    "negative_prompt": [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Picture 1: <image> "},
    ],
    "images": [{"bytes": condition_image_png_bytes}],
    "data_source": "image_edit",
    "reward_model": {
        "style": "model",
        "ground_truth": "Change the background to blue",
    },
    "extra_info": {
        "instruction": "Change the background to blue",
        "image": "000001.png",
    },
}

The number of <image> placeholders must match the number of entries in images. Qwen-Image-Edit-2511 training currently requires exactly one condition image per sample.

Launch Training

The example recipe trains a LoRA adapter with PickScore:

WORKSPACE=$PWD \
NUM_GPUS_ACTOR_ROLLOUT_REWARD=8 \
bash examples/flowgrpo_trainer/qwen_image_edit/run_qwen_image_edit_lora.sh \
    trainer.logger=console

V1 sync trainer

A v1 counterpart launches main_diffusion_v1 in synchronous TransferQueue mode. It keeps the same parquet layout (images condition bytes), Hydra data paths, and model/reward knobs as the v0 LoRA recipe:

WORKSPACE=$PWD \
NUM_GPUS_ACTOR_ROLLOUT_REWARD=8 \
bash examples/flowgrpo_trainer/qwen_image_edit/run_qwen_image_edit_lora_v1.sh \
    trainer.logger=console

The v1 launcher differs only in the trainer entrypoint and:

python3 -m verl_omni.trainer.main_diffusion_v1
trainer.use_v1=true
trainer.v1.trainer_mode=sync

See Diffusion V1 training for TransferQueue prerequisites.

Ascend NPU

The NPU recipe uses the synchronous V1 diffusion trainer and a named native PickScore model. Its placement.devices list assigns one full PickScore model worker to each selected parent-pool bundle:

WORKSPACE=$PWD \
NUM_GPUS_ACTOR_ROLLOUT_REWARD=16 \
bash examples/flowgrpo_trainer/qwen_image_edit/run_qwen_image_edit_lora_v1_npu.sh

Its checked-in resource layout uses 16 NPUs, rollout tensor parallelism 4, and 16 native PickScore workers, creating four rollout replicas. The default native placement is [0, ..., 15]; these are native-subpool bundle indices, not physical NPU IDs or tensor-parallel ranks. Adjust ROLLOUT_TP for rollout topology and set NATIVE_REWARD_DEVICES explicitly when changing the number of native PickScore replicas. The model and parquet inputs use the same environment overrides listed below.

By default, the launcher reads:

$WORKSPACE/data/qwen_image_edit/train.parquet
$WORKSPACE/data/qwen_image_edit/test.parquet

Set TRAIN_FILES and VAL_FILES to use different parquet files.

Environment Overrides

Variable	Default	Purpose
`MODEL_PATH`	`Qwen/Qwen-Image-Edit-2511`	Hugging Face model ID or local compatible pipeline path.
`TRAIN_FILES`	`$WORKSPACE/data/qwen_image_edit/train.parquet`	Training parquet path.
`VAL_FILES`	`$WORKSPACE/data/qwen_image_edit/test.parquet`	Validation parquet path.
`NUM_GPUS_ACTOR_ROLLOUT_REWARD`	`8`	Total GPUs visible to actor, rollout, and reward workers.
`ACTOR_SP`	`1`	Actor Ulysses sequence-parallel size. Values greater than one require the attention overrides below.
`ROLLOUT_TP`	`1`	Rollout tensor-parallel size.
`REWARD_WORKERS`	`4`	Asynchronous reward worker count.
`IMAGE_RESOLUTION`	`512`	Square target output resolution.
`MAX_PROMPT_LENGTH`	`8192`	Token and prompt-embedding length limit.
`PICKSCORE_MODEL_PATH`	`yuvalkirstain/PickScore_v1`	PickScore checkpoint for the native reward model.
`NATIVE_REWARD_DEVICES`	`[0,1,2,3]` (CUDA) / `[0,...,15]` (NPU)	Native-subpool bundle indices; one full PickScore instance per entry.
`REWARD_OFFLOAD`	`true`	`true` wakes/sleeps around scoring; `false` keeps the reward model resident. The meaning is identical for engine and native models.

The launcher configures reward.models.pickscore.backend=native; the same-name reward.reward_functions.pickscore entry binds automatically. Native workers preserve the PickScore model’s local inference queue; they are not TP shards. The launcher explicitly configures the PickScore model class through executor.model; the generic native backend does not select model-specific behavior.

See Named Reward Models for native-only and engine-only configurations, mixed-model resource placement, lifecycle settings, and the custom-model extension contract.

Additional Hydra overrides can be appended to the command:

bash examples/flowgrpo_trainer/qwen_image_edit/run_qwen_image_edit_lora.sh \
    actor_rollout_ref.rollout.n=4 \
    data.train_batch_size=8 \
    trainer.total_training_steps=20 \
    trainer.logger=console

Keep actor_rollout_ref.rollout.n greater than one for group-relative advantages. When reducing GPU count, also reduce the training batch size, rollout count, micro-batch sizes, and the explicit native placement list to fit memory.

The launcher enables console, TensorBoard, and W&B logging by default. The first command overrides this with trainer.logger=console so it can run without W&B credentials. To use the launcher’s default loggers, export WANDB_API_KEY before starting training.

For ACTOR_SP > 1, use the SP-capable attention backends on both actor and rollout:

ACTOR_SP=2 \
bash examples/flowgrpo_trainer/qwen_image_edit/run_qwen_image_edit_lora.sh \
    actor_rollout_ref.model.attn_backend=native \
    actor_rollout_ref.rollout.rollout_attn_backend=TORCH_SDPA \
    trainer.logger=console

Important Configuration

The example launcher sets the model-specific fields required by the adapter:

actor_rollout_ref.model.algorithm=flow_grpo
actor_rollout_ref.rollout.name=vllm_omni
actor_rollout_ref.rollout.pipeline.true_cfg_scale=4.0
actor_rollout_ref.rollout.pipeline.height=512
actor_rollout_ref.rollout.pipeline.width=512
actor_rollout_ref.rollout.pipeline.num_inference_steps=12
actor_rollout_ref.rollout.algo.sde_type=sde

Do not remove the negative prompt while true_cfg_scale > 1. Positive and negative prompt encoding both consume the condition image.

PickScore measures instruction/image alignment but does not directly enforce source-image preservation. Use an edit-aware reward, or combine rewards, when the task requires strict preservation of identity, layout, or background.

Image and Sequence Constraints

The rollout and actor adapters validate the following constraints at their respective execution stages:

Each sample has exactly one condition image.
The condition-image aspect ratio matches the target output aspect ratio.
Samples in a rollout batch produce compatible packed latent lengths.
With actor sequence parallelism, the combined target and condition token count is divisible by ACTOR_SP.
true_cfg_scale > 1 has a negative prompt.

The example converter and launcher both use square images. For non-square training, use a custom converter that preserves the intended aspect ratio, then set actor_rollout_ref.rollout.pipeline.height and actor_rollout_ref.rollout.pipeline.width to that same ratio. Do not mix condition-image aspect ratios in one batch.

Qwen-Image-Edit checkpoints may contain processor/ without processor/config.json. The adapter creates the missing config before the driver loads the processor. The model directory must be writable when that file is absent.

Run the End-to-End Smoke Test

The special E2E test covers parquet loading, multimodal prompt processing, vLLM-Omni rollout, trajectory transport, reward computation, FSDP LoRA training, and weight synchronization:

CUDA_VISIBLE_DEVICES=0 NUM_GPUS=1 \
bash tests/special_e2e/run_flowgrpo_qwen_image_edit.sh

By default it expects a tiny random checkpoint at ~/models/tiny-random/qwen-image-edit-plus. Build it before the first run:

python tests/special_e2e/build_qwen_image_edit_plus_tiny_random.py \
    --output-dir ~/models/tiny-random/qwen-image-edit-plus

The builder creates the random weights, tokenizer, processor, and scheduler locally and does not require network access or a cached source checkpoint.

Set MODEL_PATH on the smoke-test command to use another compatible tiny checkpoint. A successful run ends with:

FlowGRPO Qwen-Image-Edit e2e test passed (training completed successfully).

Training logs and generated validation images are written below $WORKSPACE/outputs/qwen_image_edit_lora/ by the full example launcher.