Survey & Simulation

This is a big release with many new features, and unfortunately not completely
backwards compatible. The main new features are the new Survey and
Simulation classes, as well as some initial work for optimization
(misfit, gradient). Also, a Model can now be a resistivity model, a
conductivity model, or the logarithm (natural or base 10) therefore. Receivers
can now be arbitrarily rotated, just as the sources. In addition to the
existing soft-dependencies empymod, discretize, and h5py there are
the new soft-dependencies xarray and tqm; discretize is now much tighter
integrated. For the new survey and simulation classes xarray is a required
dependency. However, the only hard dependency remain scipy and numba, if
you use emg3d purely as a solver. Data reading and writing has new a
JSON-backend, in addition to the existing HDF5 and NumPy-backends.

In more detail:

• Modules:

  • surveys (new; requires xarray):

    • Class surveys.Survey, which combines sources, receivers, and data.
    • Class surveys.Dipole, which defines electric or magnetic point dipoles
      and finite length dipoles.

  • simulations (new; requires xarray; soft-dependency tqdm):

    • Class simulations.Simulation, which combines a survey with a model. A
      simulation computes the e-field (and h-field) asynchronously using
      concurrent.futures. This class will include automatic, source- and
      frequency-dependent gridding in the future. If tqdm is installed it
      displays a progress bar for the asynchronous computation. Note that the
      simulation class has still some limitations, consult the class
      documentation.

  • models:

    • Model instances take new the parameters property_{x;y;z} instead of
      res_{x;y;z}. The properties can be either resistivity, conductivity, or
      log_{e;10} thereof. What is actually provided has to be defined with the
      parameter mapping. By default, it remains resistivity, as it was until
      now. The keywords res_{x;y;z} are deprecated, but still accepted at
      the moment. The attributes model.res_{x;y;z} are still available too,
      but equally deprecated. However, it is no longer possible to
      assign values to these attributes, which is a backwards
      incompatible change.
    • A model knows now how to interpolate itself from its grid to another grid
      (interpolate2grid).

  • maps:

    • New mappings for models.Model instances: The mappings take care of
      how to transform the investigation variable to conductivity and back, and
      how it affects its derivative.
    • New interpolation routine edges2cellaverages.

  • fields:

    • Function get_receiver_response (new), which returns the response
      for arbitrarily rotated receivers.

    • Improvements to Field and SourceField:

      • _sval and _smu0 not stored any longer, derived from _freq.
      • SourceField is now using the copy() and from_dict() from its
        parents class Field.

  • io:

    • File-format json (new), writes to a hierarchical, plain json file.
    • Deprecated the use of backend, it uses the file extension of
      fname instead.
    • This means .npz (instead of numpy), .h5 (instead of h5py), and
      new .json.
    • New parameter collect_classes, which can be used to switch-on
      collection of the main classes in root-level dictionaries. By default,
      they are no longer collected (changed).

  • meshes:

    • meshes.TensorMesh new inherits from discretize if installed.
    • Added __eq__ to models.TensorMesh to compare meshes.

  • optimize (new)

    • Functionalities related to inversion (data misfit, gradient, data
      weighting, and depth weighting). This module is in an early stage, and
      the API will likely change in the future. Current functions are misfit,
      gradient (using the adjoint-state method), and data_weighting. These
      functionalities are best accessed through the Simulation class.

• Dependencies:

  • empymod is now a soft dependency (no longer a hard dependency), only
    required for utils.Fourier (time-domain modelling).
  • Existing soft dependency discretize is now baked straight into meshes.
  • New soft dependency xarray for the Survey class (and therefore also for
    the Simulation class and the optimize module).
  • New soft dependency tqdm for nice progress bars in asynchronous
    computation.

• Deprecations and removals:

  • Removed deprecated functions data_write and data_read.
  • Removed all deprecated functions from utils.

• Miscellaneous:

  • Re-organise API-docs.
  • Much bookkeeping (improve error raising and checking; chaining errors,
    numpy types, etc).

Refactor

Grand refactor with new internal layout. Mainly splitting-up utils into
smaller bits. Most functionalities (old names) are currently retained in
utils and it should be mostly backwards compatible for now, but they are
deprecated and will eventually be removed. Some previously deprecated functions
were removed, however.

• Removed deprecated functions:

  • emg3d.solver.solver (use emg3d.solver.solve instead).
  • Aliases of emg3d.io.data_write and emg3d.io.data_read in emg3d.utils.

• Changes:

  • SourceField has now the same signature as Field (this might break your
    code if you called SourceField directly, with positional arguments, and
    not through get_source_field).

  • More functions and classes in the top namespace.

  • Replaced core.l2norm with scipy.linalg.norm, as SciPy 1.4 got the
    following PR: scipy/scipy#10397 (reason to raise
    minimum SciPy to 1.4).

  • Increased minimum required versions of dependencies to

    • scipy>=1.4.0 (raised from 1.1, see note above)
    • empymod>=2.0.0 (no min requirement before)
    • numba>=0.45.0 (raised from 0.40)

• New layout

  • njitted -> core.
  • utils split in fields, meshes, models, maps, and utils.

• Bugfixes:

  • Fixed to_dict, from_dict, and copy for the SourceField.
  • Fixed io for SourceField, that was not implemented properly.

Zero Source

• Bug fixes:

  • Checks now if provided source-field is zero, and exists gracefully if so, returning a zero electric field. Until now it failed with a division-by-zero error.

• Improvements:

  • Warnings: If verb=1 it prints a warning in case it did not converge (it finished silently until now).
  • Improvements to docs (figures-scaling; intersphinx).
  • Adjust Fields.pha and Fields.amp in accordance with empymod v2: .pha and .amp are now methods; uses directly empymod.utils.EMArray.
  • Adjust tests for empymod v2 (Fields, Fourier).

Data persistence

• New:

  • New functions emg3d.save and emg3d.load to save and load all sort of emg3d instances. The currently implemented backends are h5py for .h5-files (default, but requires h5py to be installed) and numpy for .npz-files.
  • Classes emg3d.utils.Field, emg3d.utils.Model, and emg3d.utils.TensorMesh have new methods .copy(), .to_dict(), and .from_dict().
  • emg3d.utils.Model: Possible to create new models by adding or subtracting existing models, and comparing two models (+, -, == and !=). New attributes shape and size.
  • emg3d.utils.Model does not store the volume any longer (just vnC).

• Deprecations:

  • Deprecated data_write and data_read.

• Internal and bug fixes:

  • All I/O-related stuff moved to its own file io.py.
  • Change from NUMBA_DISABLE_JIT to use py_func for testing and coverage.
  • Bugfix: emg3d.njitted.restrict did not store the {x;y;z}-field if sc_dir was {4;5;6}, respectively.

Sphinx gallery

• Rename solver.solver to solver.solve; load solve also into the main namespace as emg3d.solve.
• Adjustment to termination criterion for STAGNATION: The current error is now compared to the last error of the same cycle type. Together with this the workaround for sslsolver when called with an initial efield introduced in v0.8.0 was removed.
• Adjustment to utils.get_hx_h0 (this might change your boundaries): The calculation domain is now calculated so that the distance for the signal travelling from the source to the boundary and back to the most remote receiver is at least two wavelengths away. If this is within the provided domain, then now extra buffer is added around the domain. Additionally, the function has a new parameter max_domain, which is the maximum distance from the center to the boundary; defaults to 100 km.
• New parameter log for utils.grid2grid; if True, then the interpolation is carried out on a log10-scale.
• Change from the notebook-based emg3d-examples-repo to the sphinx-based emg3d-gallery-repo.

Complex sources

• Strength input for get_source_field can now be complex; it also stores now the source location and its strength and moment.

• get_receiver can now take entire Field instances, and returns in that case (fx, fy, fz) at receiver locations.

• Krylov subspace solvers:

  • Solver now finishes in the middle of preconditioning cycles if tolerance is reached.
  • Solver now aborts if solution diverges or stagnates also for the SSL solvers; it fails and returns a zero field.
  • Removed gmres and lgmres from the supported SSL solvers; they do not work nice for this problem. Supported remain bicgstab (default), cgs, and gcrotmk.

• Various small things:

  • New attribute Field.is_electric, so the field knows if it is electric or magnetic.

  • New verb-possibility: verb=-1 is a continuously updated one-liner, ideal to monitor large sets of calculations or in inversions.

  • The returned info dictionary contains new keys:

    • runtime_at_cycle: accumulated total runtime at each cycle;
    • error_at_cycle: absolute error at each cycle.

  • Simple __repr__ for TensorMesh, Model, Fourier, Time.

• Bugfixes:

  • Related to get_hx_h0, data_write, printing in Fourier.

VolumeModel

• New class VolumeModel; changes in Model:

  • Model now only contains resistivity, magnetic permeability, and electric permittivity.
  • VolumeModel contains the volume-averaged values eta and zeta; called from within emg3d.solver.solver.
  • Full wave equation is enabled again, via epsilon_r; by default it is set to None, hence diffusive approximation.
  • Model parameters are now internally stored as 1D arrays.
  • An {isotropic, VTI, HTI} initiated model can be changed by providing the missing resistivities.

• Bugfix: Up and till version 0.8.1 there was a bug. If resistivity was set with slices, e.g., model.res[:, :, :5]=1e10, it DID NOT update the corresponding eta. This bug was unintentionally fixed in 0.9.0, but only realised now.

• Various:

  • The log now lists the version of emg3d.
  • PEP8: internal imports now use absolute paths instead of relative ones.
  • Move from conda-channel prisae to conda-forge.
  • Automatic deploy for PyPi and conda-forge.

Fourier

• New routine:

  • emg3d.utils.Fourier, a class to handle Fourier-transform related stuff for time-domain modelling. See the example notebooks for its usage.

• Utilities:

  • Fields and returned receiver-arrays (EMArray) both have amplitude (.amp) and phase (.pha) attributes.
  • Fields have attributes containing frequency-information (freq, smu0).
  • New class SourceField; a subclass of Field, adding vector and v{x,y,z} attributes for the real valued source vectors.
  • The Model is not frequency-dependent any longer and does NOT take a freq-parameter any more (currently it still takes it, but it is deprecated and will be removed in the future).
  • data_write automatically removes _vol from TensorMesh instances and _eta_{x,y,z}, _zeta from Model instances. This makes the archives smaller, and they are not required, as they are simply reconstructed if needed.

• Internal changes:

  • The multigrid method, as implemented, only works for the diffusive approximation. Nevertheless, we always used \sigma-i\omega\epsilon, hence a complex number. This is now changed and \epsilon set to 0,
    leaving only \sigma.
  • Change time convention from exp(-iwt) to exp(iwt), as used in empymod and commonly in CSEM. Removed the parameter conjugate from the solver, to simplify.
  • Change own private class variables from __ to _.
  • res and mu_r are now checked to ensure they are >0; freq is checked to ensure !=0.

• New dependencies and maintenance:

  • empymod is a new dependency.
  • Travis now checks all the url's in the documentation, so there should be no
    broken links down the road. (Check is allowed to fail, it is visual QC.)

• Bugfixes:

  • Fixes to the setuptools_scm-implementation (MANIFEST.in).

setuptools_scm

• Implement setuptools_scm for versioning (adds git hashes for dev-versions).

Laplace

• Laplace-domain calculation: By providing a negative freq-value to utils.get_source_field and utils.Model, the calculation is carried out in the real Laplace domain s = freq instead of the complex frequency
  domain s = 2i*pi*freq.

• New meshing helper routines (particularly useful for transient modelling where frequency-dependent/adaptive meshes are inevitable):

  • utils.get_hx_h0 to get cell widths and origin for given parameters including a few fixed interfaces (center plus two, e.g. top anomaly, sea-floor, and sea-surface).
  • utils.get_cell_numbers to get good values of number of cells for given primes.

• Speed-up njitted.volume_average significantly thanks to @jcapriot.

• Bugfixes and other minor things:

  • Abort if l2-norm is NaN (only works for MG).
  • Workaround for the case where a sslsolver is used together with a provided initial efield.
  • Changed parameter rho to res for consistency reasons in utils.get_domain.
  • Changed parameter h_min to min_width for consistency reasons in utils.get_stretched_h.