Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using callbacks degrades ComponentArray solutions to Arrays. #284

Open
dangirsh opened this issue Oct 5, 2020 · 3 comments
Open

Using callbacks degrades ComponentArray solutions to Arrays. #284

dangirsh opened this issue Oct 5, 2020 · 3 comments

Comments

@dangirsh
Copy link

dangirsh commented Oct 5, 2020

I’ve been solving a DAEProblem with IDA, and use a ComponentArray (from ComponentArrays.jl) to represent the solution. This works, until I add a DiscreteCallback. Then, the ComponentArray is degraded to an vanilla Array. AFAIK, this degradation breaks the intended type promises of the common solve interface, and is an undocumented issue.

The only potential workarounds I see are:

  1. Re-write my code to work on Arrays. This would be annoying, especially since it seems to be specific to Sundials.jl.
  2. Convert the solution back to a ComponentArray in the appropriate places. If anyone knows how to do this without causing fresh array allocations, please let me know.

This MWE fails on unpacking the (dummy) component u_ from u, because u is an Array. If you set the callback to nothing, it works (u remains a ComponentArray). If you change the solver to DImplicitEuler or DABDF2, it also works. I have not tested other callback types.

    using Sundials
    using ComponentArrays
    using DifferentialEquations
    using Parameters: @unpack
    
    function f(out,du,u,p,t)
        @unpack u_ = u
        out[1] = - 0.04u_[1]              + 1e4*u_[2]*u_[3] - du[1]
        out[2] = + 0.04u_[1] - 3e7*u_[2]^2 - 1e4*u_[2]*u_[3] - du[2]
        out[3] = u_[1] + u_[2] + u_[3] - 1.0
    end
    
    u₀ = ComponentVector(u_=[1.0, 0, 0])
    du₀ = (_ -> rand()).(copy(u₀))
    tspan = (0.0,100000.0)
    
    differential_vars = [true,true,false]
    prob = DAEProblem{true}(f,du₀,u₀,tspan,differential_vars=differential_vars)
    
    test_condition(u, t, integrator) = isapprox(u[1], 0.01, atol=0.01)
    test_cb = DiscreteCallback(test_condition, terminate!)
    # test_cb = nothing
    
    sol = solve(prob, IDA(), callback=test_cb)
    # sol = solve(prob, DABDF2(), callback=test_cb)
type Array has no field u_

Stacktrace:
 [1] getproperty(::Array{Float64,1}, ::Symbol) at ./Base.jl:33
 [2] unpack at /home/dan/.julia/packages/UnPack/EkESO/src/UnPack.jl:34 [inlined]
 [3] macro expansion at /home/dan/.julia/packages/UnPack/EkESO/src/UnPack.jl:101 [inlined]
 [4] f(::ComponentVector{Float64}, ::Array{Float64,1}, ::Array{Float64,1}, ::DiffEqBase.NullParameters, ::Float64) at ./In[74]:7
 [5] (::DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing})(::ComponentVector{Float64}, ::Vararg{Any,N} where N) at /home/dan/.julia/dev/DiffEqBase/src/diffeqfunction.jl:275
 [6] (::Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}})(::ComponentVector{Float64}, ::Array{Float64,1}, ::Array{Float64,1}, ::DiffEqBase.NullParameters, ::Float64) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/solve.jl:1045
 [7] initialize_dae!(::Sundials.IDAIntegrator{Array{Float64,1},Array{Float64,1},DiffEqBase.NullParameters,Sundials.Handle{Sundials.IDAMem},DAESolution{Float64,2,Array{ComponentVector{Float64},1},Nothing,Nothing,Nothing,Array{Float64,1},DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},IDA{:Dense,Nothing,Nothing},DiffEqBase.HermiteInterpolation{Array{Float64,1},Array{ComponentVector{Float64},1},Array{ComponentVector{Float64},1}},DiffEqBase.DEStats},IDA{:Dense,Nothing,Nothing},Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Sundials.FunJac{Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Nothing,Nothing,DiffEqBase.NullParameters,Nothing,Nothing,ComponentVector{Float64},Array{Float64,1},Nothing,Nothing},Nothing,Sundials.DEOptions{DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},CallbackSet{Tuple{},Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}},Float64,Float64,typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE)},Array{Float64,1},Tuple{Int64},Tuple{Int64},ComponentVector{Float64},Sundials.LinSolHandle{Sundials.Dense},Sundials.MatrixHandle{Sundials.DenseMatrix},Nothing}) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/integrator_utils.jl:198
 [8] handle_callback_modifiers!(::Sundials.IDAIntegrator{Array{Float64,1},Array{Float64,1},DiffEqBase.NullParameters,Sundials.Handle{Sundials.IDAMem},DAESolution{Float64,2,Array{ComponentVector{Float64},1},Nothing,Nothing,Nothing,Array{Float64,1},DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},IDA{:Dense,Nothing,Nothing},DiffEqBase.HermiteInterpolation{Array{Float64,1},Array{ComponentVector{Float64},1},Array{ComponentVector{Float64},1}},DiffEqBase.DEStats},IDA{:Dense,Nothing,Nothing},Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Sundials.FunJac{Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Nothing,Nothing,DiffEqBase.NullParameters,Nothing,Nothing,ComponentVector{Float64},Array{Float64,1},Nothing,Nothing},Nothing,Sundials.DEOptions{DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},CallbackSet{Tuple{},Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}},Float64,Float64,typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE)},Array{Float64,1},Tuple{Int64},Tuple{Int64},ComponentVector{Float64},Sundials.LinSolHandle{Sundials.Dense},Sundials.MatrixHandle{Sundials.DenseMatrix},Nothing}) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/integrator_utils.jl:129
 [9] handle_callbacks!(::Sundials.IDAIntegrator{Array{Float64,1},Array{Float64,1},DiffEqBase.NullParameters,Sundials.Handle{Sundials.IDAMem},DAESolution{Float64,2,Array{ComponentVector{Float64},1},Nothing,Nothing,Nothing,Array{Float64,1},DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},IDA{:Dense,Nothing,Nothing},DiffEqBase.HermiteInterpolation{Array{Float64,1},Array{ComponentVector{Float64},1},Array{ComponentVector{Float64},1}},DiffEqBase.DEStats},IDA{:Dense,Nothing,Nothing},Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Sundials.FunJac{Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Nothing,Nothing,DiffEqBase.NullParameters,Nothing,Nothing,ComponentVector{Float64},Array{Float64,1},Nothing,Nothing},Nothing,Sundials.DEOptions{DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},CallbackSet{Tuple{},Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}},Float64,Float64,typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE)},Array{Float64,1},Tuple{Int64},Tuple{Int64},ComponentVector{Float64},Sundials.LinSolHandle{Sundials.Dense},Sundials.MatrixHandle{Sundials.DenseMatrix},Nothing}) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/integrator_utils.jl:34
 [10] solve!(::Sundials.IDAIntegrator{Array{Float64,1},Array{Float64,1},DiffEqBase.NullParameters,Sundials.Handle{Sundials.IDAMem},DAESolution{Float64,2,Array{ComponentVector{Float64},1},Nothing,Nothing,Nothing,Array{Float64,1},DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},IDA{:Dense,Nothing,Nothing},DiffEqBase.HermiteInterpolation{Array{Float64,1},Array{ComponentVector{Float64},1},Array{ComponentVector{Float64},1}},DiffEqBase.DEStats},IDA{:Dense,Nothing,Nothing},Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Sundials.FunJac{Sundials.var"#82#86"{DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}},Tuple{Int64},Tuple{Int64}},Nothing,Nothing,DiffEqBase.NullParameters,Nothing,Nothing,ComponentVector{Float64},Array{Float64,1},Nothing,Nothing},Nothing,Sundials.DEOptions{DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},DataStructures.BinaryHeap{Float64,Base.Order.ForwardOrdering},CallbackSet{Tuple{},Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}},Float64,Float64,typeof(DiffEqBase.ODE_DEFAULT_PROG_MESSAGE)},Array{Float64,1},Tuple{Int64},Tuple{Int64},ComponentVector{Float64},Sundials.LinSolHandle{Sundials.Dense},Sundials.MatrixHandle{Sundials.DenseMatrix},Nothing}) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/solve.jl:1462
 [11] __solve(::DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}}, ::IDA{:Dense,Nothing,Nothing}, ::Array{Any,1}, ::Array{Any,1}, ::Array{Any,1}, ::Type{Val{true}}; kwargs::Base.Iterators.Pairs{Symbol,DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)},Tuple{Symbol},NamedTuple{(:callback,),Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}}}) at /home/dan/.julia/packages/Sundials/xAoTr/src/common_interface/solve.jl:15
 [12] #solve_call#456 at /home/dan/.julia/dev/DiffEqBase/src/solve.jl:65 [inlined]
 [13] solve_up(::DAEProblem{ComponentVector{Float64},ComponentVector{Float64},Tuple{Float64,Float64},true,DiffEqBase.NullParameters,DAEFunction{true,typeof(f),Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing,Nothing},Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Array{Bool,1}}, ::Nothing, ::ComponentVector{Float64}, ::DiffEqBase.NullParameters, ::IDA{:Dense,Nothing,Nothing}; kwargs::Base.Iterators.Pairs{Symbol,DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)},Tuple{Symbol},NamedTuple{(:callback,),Tuple{DiscreteCallback{typeof(test_condition),typeof(terminate!),typeof(DiffEqBase.INITIALIZE_DEFAULT)}}}}) at /home/dan/.julia/dev/DiffEqBase/src/solve.jl:86
 [14] #solve#457 at /home/dan/.julia/dev/DiffEqBase/src/solve.jl:74 [inlined]
 [15] top-level scope at In[74]:25
 [16] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091

Code from adapted from the DAE Example in the SciML docs.

FYI @jonniedie

@jonniedie
Copy link

Hm.. I'll give this a look. Thanks.

@jonniedie
Copy link

jonniedie commented Oct 5, 2020

So it looks it happens with other array types too. You get the same thing if you use LabelledArrays:

using Sundials
using DifferentialEquations
using LabelledArrays
using Parameters: @unpack

function f(out,du,u,p,t)
    @unpack u_ = u
    out[1] = - 0.04u_[1]              + 1e4*u_[2]*u_[3] - du[1]
    out[2] = + 0.04u_[1] - 3e7*u_[2]^2 - 1e4*u_[2]*u_[3] - du[2]
    out[3] = u_[1] + u_[2] + u_[3] - 1.0
end

u₀ = @LArray [1.0, 0, 0] (u_=1:3,)
du₀ = @LArray rand(3) (u_=1:3,)
tspan = (0.0,100000.0)

differential_vars = [true,true,false]
prob = DAEProblem{true}(f,du₀,u₀,tspan,differential_vars=differential_vars)

test_condition(u, t, integrator) = isapprox(u[1], 0.01, atol=0.01)
test_cb = DiscreteCallback(test_condition, terminate!)
# test_cb = nothing

sol = solve(prob, IDA(), callback=test_cb, jac_prototype=ones(3,3))
# sol = solve(prob, DABDF2(), callback=test_cb)

@ChrisRackauckas
Copy link
Member

It's interesting that a ComponentArray worked in the first place! Sundials as a C++ code does not compose with Julia abstractions. ComponentArrays seems to have slipped by and worked because Sundials grabbed the function pointer and used that, since it has an underlying array, but when it shows up in the DiscreteCallback it needs to be reinterpreted for the user. We should probably find a nice general way to handle this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants