Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CompatHelper: bump compat for ImageTransformations to 0.9, (keep existing compat) #101

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

github-actions[bot]
Copy link
Contributor

This pull request changes the compat entry for the ImageTransformations package from 0.5, 0.6, 0.7, 0.8 to 0.5, 0.6, 0.7, 0.8, 0.9.
This keeps the compat entries for earlier versions.

Note: I have not tested your package with this new compat entry.
It is your responsibility to make sure that your package tests pass before you merge this pull request.

@johnnychen94 johnnychen94 self-assigned this Aug 20, 2021
@johnnychen94
Copy link
Collaborator

This is a breaking release so needs more time to work on this, I believe there will be some test failures here and there. See https://github.com/JuliaImages/ImageTransformations.jl/blob/master/CHANGELOG.md for more information about this release.

@barucden
Copy link
Collaborator

barucden commented Sep 8, 2021

I was checking the new version and saw this:

julia> using ImageCore, ImageTransformations, Augmentor

julia> square = Gray{N0f8}[0.1 0.2 0.3; 0.4 0.5 0.6; 0.7 0.6 0.9]
3×3 Array{Gray{N0f8},2} with eltype Gray{N0f8}:
 Gray{N0f8}(0.102)  Gray{N0f8}(0.2)    Gray{N0f8}(0.298)
 Gray{N0f8}(0.4)    Gray{N0f8}(0.502)  Gray{N0f8}(0.6)
 Gray{N0f8}(0.698)  Gray{N0f8}(0.6)    Gray{N0f8}(0.902)

julia> WarpedView(square, inv(Augmentor.toaffinemap(Rotate90(), square, nothing)), ImageTransformations.autorange(square, Augmentor.toaffinemap(Rotate90(), square, nothing)))
3×3 WarpedView(::Array{Gray{N0f8},2}, AffineMap([6.123233995736766e-17 1.0; -1.0 6.123233995736766e-17], [-2.4492935982947064e-16, 4.0])) with eltype Gray{N0f8} with indices 1:3×1:3:
 Gray{N0f8}(0.0)    Gray{N0f8}(0.6)    Gray{N0f8}(0.902)
 Gray{N0f8}(0.2)    Gray{N0f8}(0.502)  Gray{N0f8}(0.6)
 Gray{N0f8}(0.102)  Gray{N0f8}(0.4)    Gray{N0f8}(0.698)

Gray{N0f8}(0.298) turned into Gray{N0f8}(0.0), which later makes our tests fail. Is this a result of any change in ImageTransformations? Can we adjust to it?

@timholy
Copy link

timholy commented Sep 8, 2021

It's a consequence of JuliaImages/ImageTransformations.jl#143. It only affects points exactly at the edge, and it's due to ulp-level roundoff errors causing a point that was judged as being interior now being judged as exterior and thus subject to the fill value. "reflect" or "replicate" boundary conditions would fix it. Alternatively we should modify Interpolations.jl to add a type that allows a tolerance.

EDIT: for the record, with ImageTransformations 0.8 you get errors like

MethodError: RotZ{Float64}(::NTuple{9, Float64}) is ambiguous. Candidates:
  (::Type{R})(t::NTuple{9, T} where T) where R<:RotZ in Rotations at /home/tim/.julia/packages/Rotations/QaTBi/src/euler_types.jl:30
  RotZ{T}(theta) where T in Rotations at /home/tim/.julia/packages/Rotations/QaTBi/src/euler_types.jl:21
Possible fix, define
  RotZ{T}(::NTuple{9, T} where T) where T

because not all transformations can be reconstructed with new values as defined. That why that PR was necessary.

@barucden barucden force-pushed the compathelper/new_version/2021-08-20-00-13-26-813-02800432892 branch 2 times, most recently from 654a78f to b12ae89 Compare September 9, 2021 11:21
@barucden
Copy link
Collaborator

barucden commented Sep 9, 2021

@johnnychen94 I wanted to help you adjusting the code but I am unable to do it. I think I got the low-hanging fruit but the more involved changes are still required. I pushed my changes to this branch with the idea that you can decide if you want to use them.

I think we should require (at least) v0.9 of ImageTransformations (in Project.toml) because that's what we need for #96.

@barucden
Copy link
Collaborator

I had some free time now so I got back to this PR. Essentially, I replaced all invocations of invwarpedview with the InvWarpedView(;fillvalue=Flat()) constructor. This change fixed many tests.

There are still many issues though. I tried to split them into multiple commits, each containing changes that were necessary for the tests to "pass".

  1. f8ea4ed: Several calls of @inferred fail now. This happens mostly in the case of Either. I don't know what is the cause of this.
  2. fac9447: Many @test_reference tests fail now. It seems that only rotation is affected.
  3. 3016d86: Some fields are not present in the new ImageTransformations.jl. I think this is not a problem, and we can just remove the commented lines.
  4. 643fa28: Many typeof(...) == typeof(...) tests fail now. It mostly involves InvWarpedView. It seems that now the type is more "nested" than before my changes. Furthermore, there are problems with sequences of rotations (e.g., here).

I can work on this further but I am not sure how to proceed. Any tips are greatly appreciated!

@johnnychen94
Copy link
Collaborator

This is tough... If you notice that it took more than 1 year to finally upgrade this package to be 1.0-compatible (#32)... I'm afraid I don't have more suggestions for this until I get some large trunk of free time to do the detective work.

Several calls of @inferred fail now. This happens mostly in the case of Either. I don't know what is the cause of this.

Does the @inferred test work in master? Since Julia 1.7 brings a lot of type-inference changes, there might be some odds that it's because of the Julia version changes.

@barucden
Copy link
Collaborator

Not sure if this is relevant at all, but the "nested" types come up when the fillvalue is set.

Example:

using ImageCore
using ImageTransformations
using CoordinateTransformations
using StaticArrays

const tinv1 = AffineMap(@SMatrix[1.0 0.0; 0. 1.], @SVector[0.0, 0.0])
const tinv2 = AffineMap(@SMatrix[1.0 0.0; 1.0 1.0], @SVector[0.0, -1.5])

function with_flat(img)
    iwv = InvWarpedView(rect, tinv1, fillvalue=ImageTransformations.Flat())
    iwv2 = InvWarpedView(iwv, tinv2, fillvalue=ImageTransformations.Flat())
    return typeof(iwv2)
end

function without_flat(img)
    iwv = InvWarpedView(rect, tinv1)
    iwv2 = InvWarpedView(iwv, tinv2)
    return typeof(iwv2)
end

rect = Gray{N0f8}[0.1 0.2 0.3; 0.4 0.5 0.6]
@show with_flat(rect)
@show without_flat(rect)

results in (line-breaks added for readability)

with_flat(rect) = 
InvWarpedView{Gray{N0f8}, 2, 
    InvWarpedView{
        Gray{N0f8}, 
        2, 
        Matrix{Gray{N0f8}}, 
        AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
        Tuple{UnitRange{Int64}, UnitRange{Int64}},
        AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
        Interpolations.Extrapolation{
            Gray{N0f8}, 
            2, 
            Interpolations.BSplineInterpolation{
                Gray{N0f8}, 
                2, 
                Matrix{Gray{N0f8}}, 
                Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}}, 
                Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}
            },
            Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
            Interpolations.Flat{Nothing}
        } 
    }, 
    AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
    Tuple{UnitRange{Int64}, UnitRange{Int64}},
    AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
    Interpolations.Extrapolation{
        Gray{N0f8}, 
        2, 
        Interpolations.BSplineInterpolation{
            Gray{N0f8}, 
            2, 
            InvWarpedView{
                Gray{N0f8}, 
                2, 
                Matrix{Gray{N0f8}},
                AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
                Tuple{UnitRange{Int64}, UnitRange{Int64}},
                AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
                Interpolations.Extrapolation{
                    Gray{N0f8}, 
                    2, 
                    Interpolations.BSplineInterpolation{
                        Gray{N0f8}, 
                        2, 
                        Matrix{Gray{N0f8}}, 
                        Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
                        Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}
                    }, 
                    Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
                    Interpolations.Flat{Nothing}
                }
            }, 
            Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
            Tuple{UnitRange{Int64}, UnitRange{Int64}}
        },
        Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
        Interpolations.Flat{Nothing}
    }
}

without_flat(rect) = 
InvWarpedView{
    Gray{N0f8},
    2,
    Matrix{Gray{N0f8}}, 
    AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}},
    Tuple{UnitRange{Int64}, UnitRange{Int64}},
    AffineMap{SMatrix{2, 2, Float64, 4}, SVector{2, Float64}}, 
    Interpolations.FilledExtrapolation{
        Gray{N0f8}, 
        2,
        Interpolations.BSplineInterpolation{
            Gray{N0f8}, 
            2,
            Matrix{Gray{N0f8}}, 
            Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
            Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}
        },
        Interpolations.BSpline{Interpolations.Linear{Interpolations.Throw{Interpolations.OnGrid}}},
        Gray{N0f8}
    }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants