Updates

• Support LoRA for the instruction tuning stage of LLaVA-1.5 -- comparable performance to full-model finetuning, and reduced requirements on GPU VRAM. (ckpts/logs, script)
• Bring your own data and finetune LLaVA-1.5 to your own task. (instruction)
• Basic support for Windows. (instruction)
• Fix: the training behavior with gradient accumulation is the same as large-batch training.

Notes

• A new LoRA schedule for LLaVA-1.5 is used,
  • rank:128
  • alpha: 256
  • lr (LoRA): 2e-4
  • lr (projector): 2e-5