Different configuration of "desc_prefix" for openimages dataset

[viviban]

why configuration "desc_prefix" for openimages dataset in ppt1_sd15.yaml is true but change to false in ppt2_bn.yaml ?

[zengyh1900]

Hi @viviban,

Since in v1, there is a U-Net finetuned model where there is only one U-Net, we concatenate the task prompt and image prompt together for training. However, in v2, there is a U-Net and a BrushNet. The U-Net in v2 takes the description as input, while the BrushNet takes only task prompts as input.

Therefore, we set desc_prefix to true to concatenate the task and description prompts in v1, while setting it to false in v2.

[yeluoo]

@zengyh1900 这里的prompt是整张图对应的caption，还是mask区域对应的caption，论文似乎说的是mask区域对应的反caption，那么问题来了，prompt不是训练t2i吗，那为什么是mask区域的caption呢