GPU out of memory

[Backdrop9019]

Hello, thank you for your excellent code.

I am trying to reproduce your results, but I keep encountering a GPU OOM (Out of Memory) error. I am using 16 A100 GPUs (each with 40GB of memory) for training. Even after reducing the batch size from 16 to 8, I still face CUDA OOM errors. Your paper mentions that you used 8 A100 GPUs for training. Could you please share the specific GPU settings you used?

Thank you.

[farewellthree]

Thank you for your interest in our work. We used A100 GPUs with 80GB of memory. You can try setting DeepSpeed to zero3, disabling BT-Adapter, reducing the number of frames, or using LoRA to manage memory usage.