Starting to develop New MSX API

[karrowood]

The new MSX API has a few specific goals that it will need to accomplish in order to be successful. First, it will not be dependent on the EPANET libraries as the current version of MSX is. Next, it will not use input files such as the .inp EPANET input file, or the .msx MSX input file. Instead it will take input through a REST API with higher level language bindings. In this way, users will not have to worry about following a specific particular input/output format.

In order to start development of this, it will likely be easiest if we have a test environment that calls the API in the ways we desire, and then move over the functional pieces implementing the core logic of MSX. For example, a simple pseudo-code:

MSX msx_proj = msx_init()
msx_build_network(&msx_proj, net_description)
msx_add_species(&msx_proj, chemical_stuff)
msx_add_source(&msx_proj, location_info, species, concentration, ...)
msx_set_option(&msx_proj, ... option data ...)
msx_init_q(&msx_proj)
msx_evolve(&msx_proj, hydraulic_state, timestep)
msx_get_values(...)
etc...

After this starts to become more and more functional, then it will be important to start building adaptors to allow the old API to work with the new one for full backwards compatibility.

Let me know if you have any thoughts!

[LRossman]

Here are a couple of questions:

1. What is the rationale behind changing the MSX API in this fashion?
2. Will this truly be a REST API that handles requests through HTTP? The sample API functions listed look more like those implemented in EPANET 2.2.
3. If it's not dependent on EPANET libraries then how are hydraulics calculated -- will EPANET's hydraulics code be duplicated within the new MSX?
4. Would the MSX community be better served by having this re-design be done in a separate GitHub project forked from MSX, similar to how epanet-dev (aka EPANET 3) was forked from the original EPANET?

[samhatchett]

@LRossman these are fantastic questions, and I thank you for biting!

First, to clarify: this type of lower-level API would be in addition to the current API. The motivation, as @karrowood mentioned, is to allow better extensibility (i.e., service-wrapping) of the core MSX functionality. A RESTful API would be a few layers up from here, and may not even belong in this project, but we do need to think about how to decouple the core propagation/reaction logic from any reliance on input files, binary files, or other libraries. Similar to the network-building API in EPANET, we are looking for a way to isolate a set of public functions that are safe mutators of the underlying data model.

So I would see the suggestion on the table as a re-arranging of code to allow separate compilation of the core logic apart from file I/O and epanet dependency. Does that make sense?

As to the question of forking into another project, I guess that could be put to a vote. But I don't see any fundamental changes to the architecture or legacy API/CLI resulting from this proposal.

[LRossman]

OK, I get the concept of EPANET2.2-ifying the MSX API by adding some project-building functions (it appears that the msx_proj pointer in @karrowood 's prototypes is already defined as MSXproject in msxtypes.h). And I suppose that the net_description and hydraulic_state arguments in the prototypes are replacements for having a hard link to EPANET -- which means that MSX could conceivably be used with any WDS simulation model as long as it supplies these two pieces of information (whose exact structure is yet to be determined).

[samhatchett]

Yep, that's it in a nutshell. This rearrangement and exposing of functionality has some benefits as described - like using with any simulation model (if the lower level API relies on state vector input), and potentially using this as a basis for backtracking.

so maybe the next step would be to propose a non-pseudo-code version of what a set of API functions would look like, using an opaque project pointer for the main struct, and an example of how the network topology and hydraulic state vectors would be represented?

[LRossman]

Here's all that's needed to describe network topology:

For each Link; Name  Up_Node  Down_Node  Diameter  Length  Roughness

Pumps and valves are identified by having no length. MSX also needs to know which nodes are tanks and what mixing model they use, but that could be accomplished with a msx_set_tank function.

For hydraulic state, it's basically a subset of what's in an EPANET hydraulics file:

TimeStamp
TimeStep (over which hydraulic values apply)
For each node: Name (if not using indexes)  Demand  Head
For each link: Name (if not using indexes)  Flow

There are additional hydraulic variables in an EPANET hydraulics file that MSX uses, like Reynolds number and friction factor, but these can be re-computed within MSX from knowing pipe flow and node head.