PowerModelsDistributionStateEstimation process exit

[MartaVanin]

Hello,
moving here the issue comment previously posted in #228 .
I get the error in the title when I try to run this notebook: https://github.com/Electa-Git/PowerModelsDistributionStateEstimation.jl/blob/master/examples/PMDSE_intro.jl

My environment:

  [336ed68f] CSV v0.6.2
  [b6b21f68] Ipopt v0.6.3
  [4076af6c] JuMP v0.21.4
  [91a5bcdd] Plots v1.10.5
  [c3e4b0f8] Pluto v0.12.20
  [c36e90e8] PowerModels v0.17.4
  [d7431456] PowerModelsDistribution v0.10.1
  [d0713e65] PowerModelsDistributionStateEstimation v0.2.3

Basically, it is a notebook that explains how to use the PowerModelsDistributionStateEstimation package, which I made for version v0.1.2, but that in principle should work also for v0.2.x. However, with the updated package version it crashes, and it looks like a Pluto problem. On the REPL, I get:

Worker 2 terminated.Distributed.ProcessExitedException(2)Distributed.ProcessExitedException(2)
Stacktrace: [1] worker_from_id(::Distributed.ProcessGroup, ::Int64) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\cluster.jl:1074
 [2] worker_from_id at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\cluster.jl:1071 [inlined]
 [3] remotecall(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\remotecall.jl:376
 [4] remotecall at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\remotecall.jl:376 [inlined]
 [5] macro expansion at .\task.jl:403 [inlined]
 [6] macro expansion at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\macros.jl:224 [inlined]
 [7] macro expansion at .\task.jl:332 [inlined]
 [8] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Distributed\src\macros.jl:218
 [9] create_emptyworkspacemodule(::Int64) at C:\Users\mvanin\.julia\packages\Pluto\fxtGo\src\evaluation\WorkspaceManager.jl:99
 [10] delete_vars(::Tuple{Pluto.ServerSession,Pluto.Notebook}, ::Set{Symbol}, ::Set{Tuple{Base.UUID,Array{Symbol,1}}}, ::Set{Expr}; kwargs::Base.Iterators.Pairs{Symbol,Array{Pluto.Cell,1},Tuple{Symbol},NamedTuple{(:to_run,),Tuple{Array{Pluto.Cell,1}}}}) at C:\Users\mvanin\.julia\packages\Pluto\fxtGo\src\evaluation\WorkspaceManager.jl:275
 [11] run_reactive!(::Pluto.ServerSession, ::Pluto.Notebook, ::Pluto.NotebookTopology, ::Pluto.NotebookTopology, ::Array{Pluto.Cell,1}; deletion_hook::Function, persist_js_state::Bool) at C:\Users\mvanin\.julia\packages\Pluto\fxtGo\src\evaluation\Run.jl:65
 [12] run_reactive! at C:\Users\mvanin\.julia\packages\Pluto\fxtGo\src\evaluation\Run.jl:26 [inlined]
 [13] macro expansion at C:\Users\mvanin\.julia\packages\Pluto\fxtGo\src\evaluation\Run.jl:13 [inlined]
 [14] (::Pluto.var"#79#81"{Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},Pluto.ServerSession,Pluto.Notebook,Pluto.NotebookTopology,Pluto.NotebookTopology,Array{Pluto.Cell,1}})() at .\task.jl:356

Before the error, there does not seem to be anything suspicious (those warnings from PowerModels and PMDSE are regular stuff)

Opening http://localhost:1234/?secret=f1rIAFf8 in your default browser... ~ have fun!

Press Ctrl+C in this terminal to stop Pluto

      From worker 2:    [info | PowerModels]: Circuit has been reset with the "clear" on line 1 in "case3_unbalanced.dss"
      From worker 2:    [warn | PowerModels]: Command "calcvoltagebases" on line 35 in "case3_unbalanced.dss" is not supported, skipping.
      From worker 2:    [warn | PowerModels]: Command "solve" on line 37 in "case3_unbalanced.dss" is not supported, skipping.
      From worker 2:    [info | PowerModels]: updated generator 1 cost function with order 3 to a function of order 2: [0.5, 0.0]
      From worker 2:    [warn | PowerModelsDistributionStateEstimation]: Estimation criterion set to default value, edit data dictionary if you wish to change it.
Worker 2 terminated.Distributed
.ProcessExitedException(2)

[fonsp]

There are too many possibilities for us to debug, so you need to narrow the issue down:

• Can you find out exactly which versions of PowerModelsDistributionStateEstimation work and which don't? Try to narrow down to the git commit of PowerModelsDistributionStateEstimation that caused the problem.
• Can you run the notebook file without Pluto using julia PMDSE_intro.jl? i.e. Why you think that this is caused by Pluto?
• Did you run out of memory? Does it happen on every computer?
• Does the kernel crash when importing the package? When running code? What code? When displaying a graph? Try to make minimal examples of code that breaks and code that does not break. Investigate and narrow down the differences.
• Instead of posting a printout of your package environment (time-consuming and error-prone for us to reproduce), include a Manifest.toml file or use a "Pkg cell" like I do here with fixed versions to make it easy for us to run your code with the same versions. (Note that this reuses versions that are already installed, which I am pointing out because of this, which is not entirely true.)

[MartaVanin]

Thanks for the suggestions, they helped figure out what's going on.
At the end of the day, the problem is with the Ipopt package: version 0.6.3 of that is what makes the notebook crash. This is solved with ipopt>0.6.3, so from my end this issue could be closed, as I can easily just use ipopt 0.6.4 or whatever. I have created a minimal pluto example using just JuMP and Ipopt, in case you'd like to look at it for some reason (Ipopt is definitely much more mature and used than my own package, so might be interesting to know that there are some issues?):

### A Pluto.jl notebook ###
# v0.12.21

using Markdown
using InteractiveUtils

# ╔═╡ 095355e0-7a79-11eb-2a47-1fe7fe74c783
begin
	import Pkg
	Pkg.activate(mktempdir())
	Pkg.add([
			Pkg.PackageSpec(name="JuMP"), 
			Pkg.PackageSpec(name="Ipopt", version="0.6.3"), 
			])
	using JuMP, Ipopt
end

# ╔═╡ 911c771e-7a7b-11eb-35f1-5b46c4a40ecf
begin 
	model = JuMP.Model(Ipopt.Optimizer)
	JuMP.@variable(model, x)
	JuMP.@variable(model, y)
	JuMP.@constraint(model, x + 3y <= 3)
	JuMP.@constraint(model, x^2 + 3y^2 <= 6)
	JuMP.@objective(model, Max, x+y)	
	nothing
end

# ╔═╡ c19ff810-7a79-11eb-146a-6b9a09664e47
sol = JuMP.optimize!(model)

# ╔═╡ 35b2c0e0-7a7d-11eb-3b09-812b07e33f7f
display("x is $value(x)")

# ╔═╡ Cell order:
# ╠═095355e0-7a79-11eb-2a47-1fe7fe74c783
# ╠═911c771e-7a7b-11eb-35f1-5b46c4a40ecf
# ╠═c19ff810-7a79-11eb-146a-6b9a09664e47
# ╠═35b2c0e0-7a7d-11eb-3b09-812b07e33f7f

Specifically, both in the original PMDSE_intro notebook and here, it is when I call Ipopt to solve the JuMP model that it crashes, so sol = JuMP.optimize!(model) in the present case.
If I do include("thisminimalnotebook.jl") on the REPL, it also crashes, but if I run it line by line with an editor I get no error, so I am a bit puzzled. With PMDSE_intro it didn't crash with include(... which led me to think it was a Pluto problem, but probably was just because of the clumsy package version assignment, so it worked from the environment I was using from the REPL but not in general.
I only have a windows laptop, so I can't really see if it reproduces on other machines.

[fonsp]

This is great! Thanks for narrowing it down this far.

If I do include("thisminimalnotebook.jl") on the REPL, it also crashes

In that case, this is a great MWE (minimal working example) to reproduce the crash, that you can send to IpOpt.jl

About the include / not include: it could be that the error is inconsistent