Generic methods not handled properly

[ulysses4ever]

Turn on debug and try the following on e.g. bbf3dfa

julia> a(x::T) where T<:Integer = 1
a (generic function with 1 method)

julia> is_stable_function(a)
┌ Debug: is_stable_function: a
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:191
┌ Debug: is_stable_method: a(x::T) where T<:Integer in Main at REPL[6]:1
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:201
┌ Debug: all_subtypes: Any[T<:Integer]
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:396
┌ Debug: all_subtypes loop: Any[T<:Integer]
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:401
┌ Debug: all_subtypes: concrete!
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:404
1-element Vector{MethStCheck}:
 MethStCheck(a(x::T) where T<:Integer in Main at REPL[6]:1, Stb(1))

Notice how we go over a free type variable (all_subtypes loop: Any[T<:Integer]). This happens to give the right answer in this case but it's a
pure coincidence.

In more complicated cases we fail. E.g. the Flux package:

...
┌ Debug: is_stable_method: conv!(out::AbstractArray{T, 5}, in1::AbstractArray{T, 5}, in2::AbstractArray{T, 5}, cdims::C; kwargs...) where {T<:Union{Float32, Float64, ComplexF32, ComplexF64}, C<:ConvDims} in NNlib at /home/artem/.julia/packages/NNlib/TAcqa/src/conv.jl:182
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:201
┌ Debug: all_subtypes: Any[AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, C<:ConvDims]
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:396
┌ Debug: all_subtypes loop: Any[AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, C<:ConvDims]
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:401
┌ Debug: all_subtypes: abstract!
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:407
┌ Debug: 1
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:409
┌ Debug: direct_subtypes: Any[AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, AbstractArray{T<:Union{Float32, Float64, ComplexF32, ComplexF64}, 5}, C<:ConvDims]
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:431
┌ Debug: C<:ConvDims
└ @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:436
ERROR: TaskFailedException
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base ./task.jl:777
  [2] wait()
    @ Base ./task.jl:837
  [3] wait(c::Base.GenericCondition{ReentrantLock})
    @ Base ./condition.jl:123
  [4] take_unbuffered(c::Channel{Any})
    @ Base ./channels.jl:405
  [5] take!
    @ ./channels.jl:383 [inlined]
  [6] iterate(c::Channel{Any}, state::Nothing)
    @ Base ./channels.jl:466
  [7] iterate
    @ ./channels.jl:465 [inlined]
  [8] is_stable_method(m::Method, scfg::SearchCfg)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:211
  [9] (::StabilityCheck.var"#7#8"{SearchCfg})(m::Method)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:192
 [10] iterate
    @ ./generator.jl:47 [inlined]
 [11] collect_to!
    @ ./array.jl:782 [inlined]
 [12] collect_to_with_first!
    @ ./array.jl:760 [inlined]
 [13] _collect(c::Vector{Method}, itr::Base.Generator{Vector{Method}, StabilityCheck.var"#7#8"{SearchCfg}}, #unused#::Base.EltypeUnknown, isz::Base.HasShape{1})
    @ Base ./array.jl:754
 [14] collect_similar
    @ ./array.jl:653 [inlined]
 [15] map
    @ ./abstractarray.jl:2849 [inlined]
 [16] is_stable_function(f::Function, scfg::SearchCfg)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:192
 [17] is_stable_module(mod::Module, scfg::SearchCfg)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:177
 [18] is_stable_module(mod::Module)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:168
 [19] checkModule(m::Module, out::String) (repeats 2 times)
    @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:314
 [20] top-level scope
    @ REPL[3]:1
 [21] top-level scope
    @ ~/.julia/packages/CUDA/5jdFl/src/initialization.jl:52

    nested task error: TypeError: in <:, expected Type, got a value of type TypeVar
    Stacktrace:
     [1] #98
       @ ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:437 [inlined]
     [2] _any(f::StabilityCheck.var"#98#101"{TypeVar}, itr::Vector{DataType}, #unused#::Colon)
       @ Base ./reduce.jl:1110
     [3] any(f::Function, a::Vector{DataType}; dims::Function)
       @ Base ./reducedim.jl:899
     [4] any(f::Function, a::Vector{DataType})
       @ Base ./reducedim.jl:899
     [5] direct_subtypes(ts1::Vector{Any}, scfg::SearchCfg)
       @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:437
     [6] all_subtypes(ts::Vector{Any}, scfg::SearchCfg, result::Channel{Any})
       @ StabilityCheck ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:410
     [7] #10
       @ ~/research/julia-type-stability/sts/repo/src/StabilityCheck.jl:211 [inlined]
     [8] (::Base.var"#560#561"{StabilityCheck.var"#10#12"{SearchCfg, Vector{Any}}, Channel{Any}})()
       @ Base ./channels.jl:132

How To Fix

First thought: skip (but count) generic methods altogether. Otherwise, perhaps,
the binder could be preserved.

[ulysses4ever]

I looked into it again and it's painful to implement (in the current arch but also generally, I think). It looks like at the very top level of is_stable_method we need to start instantiating type variables of a generic method before doing a single thing (e.g. before splitting the method into a function object and a signature), So, in a nutshell, you rename the current is_stable_method to is_stable_method_mono and call it from the new is_stable_method that does the instantiation.