Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failing tests #27

Open
xEnVrE opened this issue Aug 31, 2020 · 3 comments
Open

Fix failing tests #27

xEnVrE opened this issue Aug 31, 2020 · 3 comments

Comments

@xEnVrE
Copy link
Collaborator

xEnVrE commented Aug 31, 2020

Some of the tests are failing due to small mismatches between the ground truth rendering and the one generated by the test.

We should investigate whether this is due to the non repeatability of the rendering tasks.

@claudiofantacci
Copy link
Collaborator

How much is the mismatch error?

@xEnVrE
Copy link
Collaborator Author

xEnVrE commented Aug 31, 2020

It depends. For example, test_scissors reports a mismatch of only 1 unit for the green channel

[Utils][compareImages] Found non-equal pixel.
[Utils][compareImages] Pixel at (12,65) is [255, 128, 51].
[Utils][compareImages] Should be [255, 127, 51].

Adding a threshold in the compareImages util does not help since, after that, other mismatches were popping up in the order of 10 units.

By looking at the ground truth and generated files, it seems that there is a shift, that is noticeable if one alternates rapidly the figures using an image viewer, in the pixels.

ground truth

gt_scissors

generated

test_scissors

However, they seem pretty much the same to me!

@claudiofantacci
Copy link
Collaborator

The ground truth seems blurred.
I would try to generate the above image 10 times and see if the pipeline produces different results.
I would argue that and error of 1 could be tolerated (maybe some data conversion error?), but 10 seems way too off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants