Skip to content

Commit

Permalink
Restructured package with functions and utils subpackages.
Browse files Browse the repository at this point in the history
  • Loading branch information
mcuntz committed Feb 13, 2020
1 parent 810cd5e commit 21dc795
Show file tree
Hide file tree
Showing 35 changed files with 261 additions and 181 deletions.
8 changes: 6 additions & 2 deletions AUTHORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,9 @@ Copyright (c) 2012-2020 Matthias Cuntz, Juliane Mai

## Main Authors

- [Matthias Cuntz](https://github.com/mcuntz), Email: mc (at) macu (dot) de
- [Juliane Mai](https://github.com/jmai)
- [Matthias Cuntz](https://github.com/mcuntz), E-mail: mc (at) macu (dot) de
- [Juliane Mai](https://github.com/julemai)

## Contributing Authors

- [Sebastian Müller](https://github.com/MuellerSeb)
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

All notable changes after its initial development up to January 2020 (v0.2) are documented in this file.

### v1.0
- Restructured package with functions and utils subpackages.

### v0.9
- Added mention to template of Sebastian Mueller in README.md and documentation.
- Renamed morris.py to morris_method.py.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ can then simply call it as `func(x)`, so that `x` is passed to
*pyeee* provides wrappers to use with partial.

```python
from pyeee import func_wrapper
from pyeee.utils import func_wrapper
args = [a, b]
kwargs = {}
func = partial(func_wrapper, ishigami, args, kwargs)
Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def setup(app):

# General information about the project.
project = "pyeee"
copyright = "2019, Matthias Cuntz"
copyright = "2019-2020, Matthias Cuntz"
author = "Matthias Cuntz"

# The version info for the project you're documenting, acts as replacement for
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.general_functions
=======================

.. automodule:: pyeee.general_functions
.. automodule:: pyeee.functions.general_functions
:members:
:undoc-members:
:inherited-members:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.sa_test_functions
=======================

.. automodule:: pyeee.sa_test_functions
.. automodule:: pyeee.functions.sa_test_functions
:members:
:undoc-members:
:inherited-members:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/utils.rst → docs/source/functions.utils.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.utils
=================

.. automodule:: pyeee.utils
.. automodule:: pyeee.functions.utils
:members:
:undoc-members:
:inherited-members:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ Function wrappers

.. code-block:: python
from pyeee import func_wrapper
from pyeee.utils import func_wrapper
args = [a, b]
kwargs = {}
func = partial(func_wrapper, ishigami, args, kwargs)
Expand Down
2 changes: 1 addition & 1 deletion docs/source/morris_method.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.morris
=================

.. automodule:: pyeee.morris
.. automodule:: pyeee.morris_method
:members:
:undoc-members:
:inherited-members:
Expand Down
14 changes: 7 additions & 7 deletions docs/source/package.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ pyeee API
.. toctree::

eee.rst
function_wrapper.rst
general_functions.rst
morris_method.rst
sa_test_functions.rst
screening.rst
std_io.rst
tee.rst
utils.rst
morris_method.rst
functions.general_functions.rst
functions.sa_test_functions.rst
functions.utils.rst
utils.function_wrapper.rst
utils.std_io.rst
utils.tee.rst
70 changes: 35 additions & 35 deletions docs/source/userguide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -274,11 +274,11 @@ finished and `x`, `a` and `b` are passed to `ishigami`.
``pyeee`` provides wrapper functions to work with
:any:`functools.partial`. `call_ishigami` can be replaced by the
wrapper function of ``pyeee``:
:func:`~pyeee.function_wrapper.func_wrapper`:
:func:`~pyeee.utils.function_wrapper.func_wrapper`:

.. code-block:: python
from pyeee import func_wrapper
from pyeee.utils import func_wrapper
arg = [a, b]
kwarg = {}
func = partial(func_wrapper, ishigami, arg, kwarg)
Expand All @@ -291,7 +291,7 @@ finally passes `x`, `arg` and `kwarg` to `func(x, *arg, **kwarg)`.
``pyeee`` provides also a wrapper function to work with masks as
above. To exclude the second parameter :math:`x_1` from screening of
the Ishigami-Homma function again, `x0` and `mask` must be given to
:func:`~pyeee.function_wrapper.func_mask_wrapper` as well. Then
:func:`~pyeee.utils.function_wrapper.func_mask_wrapper` as well. Then
Elementary Effects will be calculated only for the remaining
parameters, between `lb[mask]` and `ub[mask]`. All other
non-masked parameters will be taken as `x0`. Remember that `mask` is
Expand All @@ -300,7 +300,7 @@ an include-mask, i.e. all `mask==True` will be screened and all

.. code-block:: python
from pyeee import func_mask_wrapper
from pyeee.utils import func_mask_wrapper
func = partial(func_mask_wrapper, ishigami, x0, mask, arg, kwarg)
out = ee(func, lb[mask], ub[mask])
Expand Down Expand Up @@ -354,7 +354,7 @@ effects :func:`~pyeee.screening.ee`:
def ishigami(x, a, b):
return np.sin(x[0]) + a * np.sin(x[1])**2 + b * x[2]**4 * np.sin(x[0])
from pyeee import func_wrapper
from pyeee.utils import func_wrapper
arg = [a, b]
kwarg = {}
func = partial(func_wrapper, ishigami, arg, kwarg)
Expand All @@ -379,12 +379,12 @@ effects :func:`~pyeee.screening.ee`:
sensitive parameters and `False` for noninformative parameters. The
mask can be combined by `logical_and` with an incoming mask.

Note if you use :func:`~pyeee.function_wrapper.func_mask_wrapper`, `out`
Note if you use :func:`~pyeee.utils.function_wrapper.func_mask_wrapper`, `out`
has the dimension of the `mask==True` elements:

.. code-block:: python
from pyeee import func_mask_wrapper
from pyeee.utils import func_mask_wrapper
func = partial(func_mask_wrapper, ishigami, x0, mask, arg, kwarg)
out = eee(func, lb[mask], ub[mask])
Expand Down Expand Up @@ -469,7 +469,7 @@ reading in the three parameters :math:`x_0, x_1, x_2` from a
return np.sin(x[0]) + np.sin(x[1])**2 + x[2]**4 * np.sin(x[0])
# read parameters
from pyeee import standard_parameter_reader
from pyeee.utils import standard_parameter_reader
## from std_io import standard_parameter_reader
pfile = 'params.txt'
x = standard_parameter_reader(pfile)
Expand All @@ -490,12 +490,12 @@ This program can be called on the command line with:
python ishiexe.py
The external program can be used in ``pyeee`` with :any:`functools.partial` and the
wrapper function :func:`~pyeee.function_wrapper.exe_wrapper`:
wrapper function :func:`~pyeee.utils.function_wrapper.exe_wrapper`:

.. code-block:: python
from functools import partial
from pyeee import exe_wrapper, standard_parameter_writer, standard_objective_reader
from pyeee.utils import exe_wrapper, standard_parameter_writer, standard_objective_reader
ishi = ['python', 'ishiexe.py']
parameterfile = 'params.txt'
objectivefile = 'obj.txt'
Expand All @@ -509,10 +509,10 @@ wrapper function :func:`~pyeee.function_wrapper.exe_wrapper`:
from pyeee import ee
out = ee(func, lb, ub)
:func:`~pyeee.std_io.standard_parameter_reader` and
:func:`~pyeee.std_io.standard_parameter_writer` are convenience
:func:`~pyeee.utils.std_io.standard_parameter_reader` and
:func:`~pyeee.utils.std_io.standard_parameter_writer` are convenience
functions that read and write one parameter per line in a file without
a header. The function :func:`~pyeee.std_io.standard_objective_reader`
a header. The function :func:`~pyeee.utils.std_io.standard_objective_reader`
simply reads one value from a file without header. The empty directory
at the end will be explained below at `Further arguments of wrappers`_.

Expand All @@ -523,11 +523,11 @@ any compiled executable from C, Fortran or alike.
Exclude parameters from screening
---------------------------------

Similar to :func:`~pyeee.function_wrapper.func_mask_wrapper`, there is
Similar to :func:`~pyeee.utils.function_wrapper.func_mask_wrapper`, there is
also a wrapper to work with masks and external executables:
:func:`~pyeee.function_wrapper.exe_mask_wrapper`. To exclude the second parameter :math:`x_1` from screening of
:func:`~pyeee.utils.function_wrapper.exe_mask_wrapper`. To exclude the second parameter :math:`x_1` from screening of
the Ishigami-Homma function again, `x0` and `mask` must be given to
:func:`~pyeee.function_wrapper.exe_mask_wrapper` as well. Remember that `mask` is
:func:`~pyeee.utils.function_wrapper.exe_mask_wrapper` as well. Remember that `mask` is
an include-mask, i.e. all `mask==True` will be screened and all
`mask==False` will not be screened:

Expand Down Expand Up @@ -561,8 +561,8 @@ Further arguments of wrappers
-----------------------------

The user can pass further arguments to
:func:`~pyeee.function_wrapper.exe_wrapper` and
:func:`~pyeee.function_wrapper.exe_mask_wrapper` via a dictionary at
:func:`~pyeee.utils.function_wrapper.exe_wrapper` and
:func:`~pyeee.utils.function_wrapper.exe_mask_wrapper` via a dictionary at
the end of the call. Setting the key `shell` to `True` passes
`shell=True` to :func:`subprocess.check_output`, which makes
:func:`subprocess.check_output` open a shell for running the external
Expand Down Expand Up @@ -603,12 +603,12 @@ parameter value, 4. maximum parameter value, 5. parameter mask, e.g.:
3 1.0 -3.1415 3.1415 1
One can use
:func:`~pyeee.std_io.standard_parameter_reader_bounds_mask` in this
:func:`~pyeee.utils.std_io.standard_parameter_reader_bounds_mask` in this
case. Parameter bounds and mask can be passed via `pargs`:

.. code-block:: python
from pyeee import standard_parameter_reader_bounds_mask
from pyeee.utils import standard_parameter_reader_bounds_mask
ishi = ['python', 'ishiexe.py']
func = partial(exe_wrapper, ishi,
parameterfile, standard_parameter_reader_bounds_mask,
Expand All @@ -620,7 +620,7 @@ Or in case of exclusion of :math:`x_1`:

.. code-block:: python
from pyeee import standard_parameter_reader_bounds_mask
from pyeee.utils import standard_parameter_reader_bounds_mask
func = partial(exe_mask_wrapper, ishi, x0, mask,
parameterfile, standard_parameter_reader_bounds_mask,
objectivefile, standard_objective_reader,
Expand All @@ -641,13 +641,13 @@ the sampled parameter values. The parameterfile might look like:
x2 = 1.0
/
The function :func:`~pyeee.std_io.sub_names_params_files` (which is
identical to :func:`~pyeee.std_io.sub_names_params_files_ignorecase`)
The function :func:`~pyeee.utils.std_io.sub_names_params_files` (which is
identical to :func:`~pyeee.utils.std_io.sub_names_params_files_ignorecase`)
can be used and parameter names are passed via `pargs`:

.. code-block:: python
from pyeee import sub_names_params_files
from pyeee.utils import sub_names_params_files
pnames = ['x0', 'x1', 'x2']
func = partial(exe_wrapper, ishi,
parameterfile, sub_names_params_files,
Expand All @@ -656,12 +656,12 @@ can be used and parameter names are passed via `pargs`:
out = ee(func, lb, ub)
`parameterfile` can be a list of parameterfiles in case of
:func:`~pyeee.std_io.sub_names_params_files`. `pid` will be explained
:func:`~pyeee.utils.std_io.sub_names_params_files`. `pid` will be explained
in the next section. Note that `pargs` is set to `[pnames]`. Setting
`'pargs':pnames` would give `*pnames` to the parameterwriter, that
means each parameter name as an individual argument, which would be
wrong because it wants to have a list of parameter names. The
docstring of :func:`~pyeee.function_wrapper.exe_wrapper` states:
docstring of :func:`~pyeee.utils.function_wrapper.exe_wrapper` states:

.. code-block:: none
Expand All @@ -671,21 +671,21 @@ docstring of :func:`~pyeee.function_wrapper.exe_wrapper` states:
or if pid==True:
parameterwriter(parameterfile, pid, x, *pargs, **pkwargs)
And the definition of :func:`~pyeee.std_io.sub_names_params_files` is:
And the definition of :func:`~pyeee.utils.std_io.sub_names_params_files` is:

.. code-block:: python
def sub_names_params_files_ignorecase(files, pid, params, names):
so `*pargs` passes `*[pnames]` that means `pnames` as argument after the
parameters to :func:`~pyeee.std_io.sub_names_params_files`.
parameters to :func:`~pyeee.utils.std_io.sub_names_params_files`.

Excluding :math:`x_1` would then be achieved by simply excluding `x1`
from `pnames`:

.. code-block:: python
from pyeee import sub_names_params_files
from pyeee.utils import sub_names_params_files
pnames = ['x0', 'x2']
func = partial(exe_wrapper, ishi,
parameterfile, sub_names_params_files,
Expand All @@ -705,7 +705,7 @@ times in the same directory at the same time, all model runs would
read the same parameter file and overwrite the output of each
other.

:func:`~pyeee.function_wrapper.exe_wrapper` concatenates an individual
:func:`~pyeee.utils.function_wrapper.exe_wrapper` concatenates an individual
integer number to the function string (or list, see
:any:`subprocess`), adds the integer to call of `parameterwrite` and
appends the number to the `objectivefile`, like:
Expand Down Expand Up @@ -739,7 +739,7 @@ The `parameterwriter` is supposed to write `parameterfile+'.'+ipid`
return np.sin(x[0]) + np.sin(x[1])**2 + x[2]**4 * np.sin(x[0])
# read parameters
from pyeee import standard_parameter_reader
from pyeee.utils import standard_parameter_reader
## from std_io import standard_parameter_reader
pfile = 'params.txt'
if pid is not None:
Expand All @@ -757,12 +757,12 @@ The `parameterwriter` is supposed to write `parameterfile+'.'+ipid`
print(y, file=ff)
ff.close()
:func:`~pyeee.function_wrapper.exe_wrapper` would then be used with
:func:`~pyeee.utils.function_wrapper.exe_wrapper` would then be used with
`'pid':True` and one can use several parallel processes:

.. code-block:: python
from pyeee import exe_wrapper, standard_parameter_writer, standard_objective_reader
from pyeee.utils import exe_wrapper, standard_parameter_writer, standard_objective_reader
ishi = ['python3', 'ishiexe1.py']
parameterfile = 'params.txt'
objectivefile = 'obj.txt'
Expand All @@ -774,7 +774,7 @@ The `parameterwriter` is supposed to write `parameterfile+'.'+ipid`
ub = np.ones(npars) * np.pi
out = ee(func, lb, ub, processes=8)
Note that :func:`~pyeee.std_io.sub_names_params_files` writes
Note that :func:`~pyeee.utils.std_io.sub_names_params_files` writes
`parameterfile+'.'+ipid` and does not work with `'pid':False`.

If you cannot change your computational model, you can use, for
Expand Down Expand Up @@ -811,7 +811,7 @@ which would then be used:
.. code-block:: python
from functools import partial
from pyeee import exe_wrapper, standard_parameter_writer, standard_objective_reader
from pyeee.utils import exe_wrapper, standard_parameter_writer, standard_objective_reader
ishi = './ishiexe.sh'
parameterfile = 'params.txt'
objectivefile = 'obj.txt'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.function_wrapper
=======================

.. automodule:: pyeee.function_wrapper
.. automodule:: pyeee.utils.function_wrapper
:members:
:undoc-members:
:inherited-members:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/std_io.rst → docs/source/utils.std_io.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.std_io
=======================

.. automodule:: pyeee.std_io
.. automodule:: pyeee.utils.std_io
:members:
:undoc-members:
:inherited-members:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tee.rst → docs/source/utils.tee.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
pyeee.tee
=================

.. automodule:: pyeee.tee
.. automodule:: pyeee.utils.tee
:members:
:undoc-members:
:inherited-members:
Expand Down
Loading

0 comments on commit 21dc795

Please sign in to comment.