Precompilation and restructuring linearSolver interface

[Pbellive]

I've been setting up precompilation for jInv and dependent packages. It seems to work but there is one unsatisfying aspect: currently the sparse direct solvers MUMPS.jl and Pardiso.jl are optional dependencies in jInv.LinearSolvers. This is implemented as follows (MUMPS example)

hasMUMPS=false
try
	using MUMPS
	hasMUMPS = true
	if myid()==1
		vMUMPS = Pkg.installed("MUMPS")
		if vMUMPS < minMUMPSversion;
			warn("MUMPS is outdated! Please update to version $(minMUMPSversion) or higher.")
		end
	end
catch
end

This works in general, including with precompilation, but there are two issues

1. If the user tries to use a solver they don't have installed, they'll get a no method error that may seem cryptic, rather than an error telling them they need to install MUMPS or Pardiso.
2. If jInv gets precompiled when the user doesn't have a particular solver installed and then they later install that solver, they'll have to manually clear the precompilation cache before the new solver becomes usable. I'm pretty sure this is an issue but I haven't actually tested it.

I don't know that the above two complaints actually merit fixing the problem. It wouldn't be too hard to implement a better solution but it would be a non-trivial amount of work and, more importantly, would break lots of user code. Having said that I thought I'd write down my thoughts to see what others think and just to have them documented.

I spent a little time looking at how this issue has been approached in other packages. Of particular interest was the juliaOpt organization. They face a similar but much more complicated problem in having a high level constrained optimization interface that supports many low level solvers. They solve the problem of supporting a great many solvers by adding an extra layer of abstraction. They provide a base abstract solver interface that lives in its own package. This does not depend on any of the specific low level solvers. Then there is a separate package for each low level solver that implements the general solver interface for that specific solver. Then when the user wants to define an optimization problem using the high level interface, they import the high level interface and the solver interface for their chosen solver. For jInv.LinearSolvers that might look like this:

jInv.LinearSolvers gets majorly slimmed down. It just defines the abstract type AbstractSolver and defines the functions solveLinearSystem!, clear!, and copySolver without adding any methods.

Then four new small packages would be needed:

1. jInvIterativeSolvers
2. jInvMUMPSSolver
3. jInvPardisoSolver
4. jInvJuliaSolver

These four packages would

1. import the abstract types and functions from jInv.LinearSolvers
2. define the appropriate concrete types and methods for their particular solvers.
3. Define constructors for their solver objects

Then if a user wants to setup a problem in jInv using MUMPS they would do

using <some jInv forward problem module>, jInvMUMPSSolver

pFor = getMyPfor(args,solver=getMUMPSSolver())

where getMUMPSSolver is defined in the jInvMUMPSSolver package.

As far as I understand that's the JuliaOpt approach to this issue. Plots.jl takes a different approach that I didn't understand from a quick pass through the source code.

As I said above, I don't think we need to change anything right now but if we want to do things in a more Julian way and eliminate the potential issues with the current approach, I leave the above as a place to start the conversation.

[dwfmarchant]

Thanks for looking into this Patrick, it would be really nice to get the pre-compilation working.

I really like this approach to the problem. The only change I might suggest would be to maybe leave the JuliaSolver in jinv.LinearSolvers.