Clarification on Fine-tuning Strategy in PowerPaint-v2

[xiao10ma]

Hi, great work! I have a question regarding the fine-tuning process of PowerPaint-v2. Since the model adopts a Decomposed Dual-Branch Diffusion architecture—comprising BrushNet and the original UNet—should I fine-tune BrushNet, UNet, or both branches during training?

I'm trying to adapt the model to a new dataset, and I’d also like to confirm whether the task prompts such as P_ctxt and P_obj, as proposed in the paper, should be fine-tuned as well.