allow mask to be propagated within the box during tracking

[AntonioConsiglio]

Description:

This pull request addresses the following changes:

1. ByteTrack Enhancement:

   • Modified the Detection updating functions to retain information about the mask [ByteTrack] - redesign update_with_detections to use IoU to match input Detections with tracker_id #754 .

2. Base Code Modification:

   • Made adjustments to the base code to resolve the issue.

Changes Made:

• byte_tracker.update_with_detections method update the tracker id of detections based on ByteTrack tracker.

Minor Changes Made:

• box_annotator.annotate method added a flag (only_tracked) to show only bbox with tracker id assigned

Testing:

colab: LinkToNotebook
Successfully tested the code with both object-detection and segmentation-models using the provided testing code:

import supervision as sv
from tqdm import tqdm
from ultralytics import YOLO

VIDEO_ROOT = "./walking_people.mp4"

class CustomDetections(sv.Detections):
    def __init__(self,*args,**kwargs):
        super().__init__(*args,**kwargs)
        self.class_name:list = []


if __name__ == "__main__":

    model = YOLO("yolov8x-seg.pt").cuda()  
    #model = YOLO("yolov8x.pt").cuda() 

    mask_annotator = sv.MaskAnnotator()
    box_annotator = sv.BoxAnnotator(thickness=1,
                                text_scale=0.2,
                                text_padding=4)
    track_annotator = sv.TraceAnnotator()

    video_info = sv.VideoInfo.from_video_path(video_path=VIDEO_ROOT)
    frame_generator = sv.get_video_frames_generator(source_path=VIDEO_ROOT)
    byte_tracker = sv.ByteTrack()

    with sv.VideoSink(f'test_box_and_mask_tracking.mp4', video_info=video_info) as sink:
  
        for frame in tqdm(frame_generator, total=video_info.total_frames):
            
            result = model.track(frame, imgsz = 640,verbose=False)[0]
            detections = CustomDetections.from_ultralytics(result)
            detections = byte_tracker.update_with_detections(detections)

            annotated_frame = box_annotator.annotate(
                scene=frame.copy(),
                detections=detections)
            annotated_frame = track_annotator.annotate(
                scene=annotated_frame.copy(),
                detections=detections)
            annotated_frame = mask_annotator.annotate(
                scene=annotated_frame.copy(),
                detections=detections)
           
            sink.write_frame(frame=annotated_frame)

[AntonioConsiglio]

Box-Tracking only result

Box-Tracking and Segmentation result

[SkalskiP]

Hi @AntonioConsiglio I just updated the description in #754. I hope that will help you understand my reasoning.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
5 out of 6 committers have signed the CLA.

✅ onuralpszr
✅ AdonaiVera
✅ SkalskiP
✅ Kadermiyanyedi
✅ AntonioConsiglio
❌ dependabot[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[AntonioConsiglio]

Hello @SkalskiP, the PR is ready to be reviewed.