How to eliminate multiple bounding boxes when predicting for one class only

[osamamer]

Hello,
I have a multi-class instance segmentation task. I'm using YOLOv5 to predict on test data, and I want to produce masks for each separate class for each image. Some of the detections have multiple assigned bounding boxes with different labels. I know I can make the prediction agnostic to eliminate this, but how do I eliminate this when predicting for each class separately? I.e. how do I take instances of class A only when class A has the highest confidence rate among all other classes?