Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Letterbox resolve to required resolution #1708

Closed
wants to merge 6 commits into from

Conversation

hardikdava
Copy link
Collaborator

Description

Currently, supervision has sv.letterbox_image() which resize images to get padded and with same aspect ratio image. But once this image used for detection task, then boxes has to be resized to original image in order to get correct scale and location of the boxes.

List any dependencies that are required for this change.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

YOUR_ANSWER

Any specific deployment considerations

For example, documentation changes, usability, usage/costs, secrets, etc.

Docs

  • Docs updated? What were the changes:

Script to reproduce results:

import numpy as np
from ultralytics import YOLO
from PIL import Image
import cv2
import supervision as sv


model = YOLO("yolo11n.pt")

image = Image.open("bus.jpg")

def detect_objects(image):
    results = model([image])[0]
    detections = sv.Detections.from_ultralytics(results)
    return detections


def annotate_img(image, dets):
    box_annotator = sv.BoxAnnotator()
    ann_img = box_annotator.annotate(image.copy(), dets)

    return ann_img


wo_letterbox_detections = detect_objects(image)
box_annotator = sv.BoxAnnotator()
wo_ann_img = box_annotator.annotate(image.copy(), wo_letterbox_detections)

## Process with letterbox
letterbox_resolution = (640, 640)
resolution_wh = (image.width, image.height)
letterboxed_image = sv.letterbox_image(image, letterbox_resolution)
letterbox_detections = detect_objects(letterboxed_image)
letterbox_detections.xyxy = sv.resolve_letterbox(letterbox_detections.xyxy, letterbox_resolution, resolution_wh)

box_annotator = sv.BoxAnnotator()
with_ann_img = box_annotator.annotate(image.copy(), letterbox_detections)
with_ann_img = sv.pillow_to_cv2(with_ann_img)


wo_ann_img = sv.pillow_to_cv2(wo_ann_img)
viz_img = np.hstack([wo_ann_img, with_ann_img])
cv2.imshow("Detection", viz_img)
cv2.waitKey(0)

@hardikdava hardikdava marked this pull request as ready for review December 3, 2024 12:14
@hardikdava hardikdava mentioned this pull request Dec 3, 2024
2 tasks
@LinasKo
Copy link
Contributor

LinasKo commented Dec 3, 2024

Thank you for a quick submission! I'll take a deeper look whenever I can.

Some preliminary notes:

  1. 'Resolve' is very general - undo_letterbox or revert_letterbox seems better.
  2. This seems separable into scale_to_fit or center_crop and sv.resize. This would introduce a general function we can use outside of unletterboxing use case. We'd then show in examples how letterboxing can be undone.

@hardikdava
Copy link
Collaborator Author

@LinasKo Thanks for a quick look. I have talked with @onuralpszr and he is suggesting different idea of integration. I will wait for him first.

@onuralpszr
Copy link
Collaborator

@hardikdava 👋 I talked with Linas and this is the new approach and he also asked me to add OBB and here is the PR #1711

@hardikdava
Copy link
Collaborator Author

@onuralpszr I saw that you already incorporated requirements for masks and oriented bounding box. Let's g with your PR. I am closing this one.

c.c. @LinasKo

@hardikdava hardikdava closed this Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants