diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 0000000..d9998d8 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,48 @@ + + +#### Summary + + +#### PR Checklist +* [ ] If this is a work in progress PR, set as a Draft PR +* [ ] General + * [ ] Notebook is compatible with latest SimPEG release + * [ ] No deprecated functions and/or input arguments are being used + * [ ] Coding cells linted according to the [style guides](https://docs.simpeg.xyz/latest/content/getting_started/contributing/code-style.html). +* [ ] Introduction is complete: + * [ ] Title and author added to notebook + * [ ] Admonitions for notebook difficulty and computational resources have been added + * [ ] Keywords list has been added + * [ ] Summary paragraph describing the tutorial has been added + * [ ] Learning objectives have been listed + * [ ] Hyperlinks to other tutorial notebooks added if necessary +* [ ] For sections and subsections: + * [ ] A header and short summary of what is being done + * [ ] Links to API documentation added for all classes and functions that are used + * [ ] Newly introduced functionality is explained, or links are provided to relevant materials + * [ ] The approach taken when choosing hyperparameter values is explained. The use of ad hoc values without explanation is discouraged + * [ ] All figures are legible and rendered appropriately +* [ ] Marked as ready for review (if this is was a draft PR), and converted to a pull request +* [ ] Tagged ``@simpeg/simpeg-developers`` when ready for review + + +#### Additional information + + + + diff --git a/index.md b/index.md index 9c9c17c..bc27217 100644 --- a/index.md +++ b/index.md @@ -106,3 +106,10 @@ Requires moderate computational resources. Run-times may exceed several minutes The computational resources required to execute this notebook exceed those provided by standard laptop computers. To execute the notebook, please deploy to a cluster computing environment. ``` +Want to Add a Tutorial? +----------------------- + +Please visit the [Contributing to User Tutorials](notebooks/contributing_index.md) page to find out how you can add a tutorial. + + + diff --git a/myst.yml b/myst.yml index 9db9d49..affeec0 100644 --- a/myst.yml +++ b/myst.yml @@ -115,6 +115,11 @@ project: children: - file: notebooks/12-pgi-inversion/plot_inv_1_joint_pf_pgi_full_info_tutorial.ipynb - file: notebooks/12-pgi-inversion/plot_inv_2_joint_pf_pgi_no_info_tutorial.ipynb + - file: notebooks/contributing_index.md + children: + - file: notebooks/contributing/build_website.md + - file: notebooks/contributing/formatting.md + - file: notebooks/contributing/pull_request.md site: template: book-theme diff --git a/notebooks/03-gravity/fwd_gravity_gradiometry_3d.ipynb b/notebooks/03-gravity/fwd_gravity_gradiometry_3d.ipynb index d9460f6..54c7a46 100644 --- a/notebooks/03-gravity/fwd_gravity_gradiometry_3d.ipynb +++ b/notebooks/03-gravity/fwd_gravity_gradiometry_3d.ipynb @@ -782,7 +782,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.15" + "version": "3.10.16" } }, "nbformat": 4, diff --git a/notebooks/contributing/build_website.md b/notebooks/contributing/build_website.md new file mode 100644 index 0000000..b55e146 --- /dev/null +++ b/notebooks/contributing/build_website.md @@ -0,0 +1,109 @@ +Building the User Tutorials Website Locally +=========================================== + +Here, we provide instructions for setting up an appropriate Python environment and building the SimPEG user tutorials website locally with [MyST](https://mystmd.org/). + +## Step 1: Cloning the GitHub Repository + +The URL for the SimPEG user tutorials GitHub repository is: https://github.com/simpeg/user-tutorials/. If using the Git Bash shell: + +```bash +git clone https://github.com/simpeg/user-tutorials +cd user-tutorials +``` + +## Step 2: Setting Up a Python Environment + +The Jupyter notebooks containing the tutorials are maintained to run properly using the [latest release of SimPEG](https://github.com/simpeg/simpeg/releases). +Notebooks may not run correctly if SimPEG is being imported from an earlier release or development branch. +The [mystmd][install-mystmd] and [nodejs](https://nodejs.org/api/packages.html) are also required to build the website locally. + +We advise building a Python environment from the `environment.yml` file in the root directory of the repository. +To create the `simpeg-user-tutorials` environment using conda: + +```bash +conda env create -f environment.yml +``` + +Once built, you can activate using: + +```bash +conda activate simpeg-user-tutorials +``` + +## Step 3: Build and Execution Commands + +The SimPEG user tutorials are a collection of [Jupyter Notebook](https://jupyter.org/) (and [Markdown](https://www.markdownguide.org/getting-started/)) files, +which [MyST][mystmd.org] builds into a website. Here, we describe the commands that contributors should be familiar with. + +:::{important} +The tutorial notebooks, including their states, are tracked by GitHub. When the SimPEG user tutorials repository was cloned, all notebooks had been run and saved. Therefore, you do not need to rerun all of the notebooks prior to building the website! +::: + +### Locally Build and Serve Website + +The following command will build the website and serve it locally. +This will allow you to preview the website and observe any changes to notebooks on the fly. +To locally build and serve the website: +the website: + +```bash +msyt start +``` + +Next, follow the instructions prompted by the command to launch the local build in your brower. + +### Build Only + +The following command will build the website and store the HTML files in +a new `_build` folder: + +```bash +msyt build --html +``` + +### Clean Cached Builds + +```bash +myst clean --all +``` + +### Rerunning Notebooks + +When making alterations and rerunning notebooks, we typically do so by launching Jupyter Notebooks: + +```bash +jupyter notebook +``` + +However, we can rerun a notebook and overwrite its output cells in place using `nbconvert`. +To rerun a single notebook, use: + +```bash +jupyter nbconvert --to notebook --execute --inplace notebook.ipynb +``` + +We may also want to rerun all notebooks in the repository. +To rerun all notebooks, use: + +```bash +jupyter nbconvert --to notebook --execute --inplace notebooks/**/*.ipynb +``` + +:::{danger} +Rerunning all notebooks is a computationally intensive task. Some notebooks +require significant amount of memory to allocate large sensitivity matrices. +::: + +> [!IMPORTANT] +> If you are using bash as your shell, make sure to run `shopt -s +> globstar` to enable the `globstar` feature that allows the use of `**` for +> filename expansion. + + + + +[install-mystmd]: https://mystmd.org/guide/quickstart +[jupyter]: https://jupyter.org +[mystmd.org]: https://mystmd.org + diff --git a/notebooks/contributing/formatting.md b/notebooks/contributing/formatting.md new file mode 100644 index 0000000..656a33d --- /dev/null +++ b/notebooks/contributing/formatting.md @@ -0,0 +1,291 @@ +Tutorial Structure and Formatting Requirements +============================================== + +To ensure quality and consistency, the [Jupyter Notebooks](https://jupyter.org/) containing tutorials must follow strict structure and formatting requirements. The contents of each notebooks must be parsed sensibly into a set of sections. Sufficient explanation and links to API documentation are required when presenting functionality. Each section within a tutorial contains a combination of [Markdown](https://www.markdownguide.org/getting-started/) and coding cells. Our notebooks use the functionality in [MyST Parser](https://myst-parser.readthedocs.io/en/latest/) to augment the contents of Markdown cells. Before moving forward, we advise the contributor to: + +* Be familiar with the structure and formatting that is used by most forward simulation and inversion tutorials. E.g. [3D Forward Simulation of Gravity Anomaly Data](../03-gravity/fwd_gravity_anomaly_3d.ipynb) and [3D Inversion of Gravity Anomaly Data](../03-gravity/inv_gravity_anomaly_3d.ipynb). + +* Refer to the Markdown syntax used by [MyST Parser](https://myst-parser.readthedocs.io/en/latest/). + +* Refer to the [style guide](https://docs.simpeg.xyz/latest/content/getting_started/contributing/code-style.html) for coding cells. + + +## Notebook Introduction + +Every tutorial notebook requires an introduction section. The Markdown cells required for the introduction are described below. + +### Cell 1: Title Cell + +The first cell contains the title and the author for the tutorial. It is formatted as follows. +The title is in quotations. You will need to add yourself as an author in the `myst.yml` file in the root directory of the repository. + +``` +--- +title: "My Notebook Title In Quotes" +authors: + - id: devincowan +--- +``` + +### Cell 2: Tutorial Difficulty + +Here, we indicate whether the notebook contains introductory, intermediate or advanced content. The author is required to choose +from one of the following [MyST admonitions](https://myst-parser.readthedocs.io/en/latest/syntax/admonitions.html), which is rendered when the webpage is built. + +``` +:::{admonition} Introductory notebook +:class: hint +This tutorial teaches basic functionality within SimPEG and is a good +entry point for new users. +::: +``` + +``` +:::{admonition} Intermediate notebook +:class: caution +This tutorial focusses on intermediate level functionality within SimPEG. +Basic functionality within SimPEG is not discussed in detail, as we assume +the user is already familiar. +::: +``` + +``` +:::{admonition} Advanced notebook +:class: danger +This tutorial focusses on advanced functionality within SimPEG. Basic and +intermediate level functionality are not discussed in detail, as we assume +the user is already an experienced SimPEG user. +::: +``` + +### Cell 3: Computational Requirements + +Here, we specify the computational requirements to run the notebook. Whereas some notebooks can be run easily on laptop computers, some notebook may need to be exported as Python files and run using cluster computing. The author is required to choose from one of the following [MyST admonitions](https://myst-parser.readthedocs.io/en/latest/syntax/admonitions.html), which is rendered when the webpage is built. + +``` +:::{admonition} Light-weight notebook +:class: hint +This tutorial requires minimal computational resources and can be executed +quickly in the background while other computer processes are running. +::: +``` + +``` +:::{admonition} Medium-weight notebook +:class: caution +Requires moderate computational resources. Run-times may exceed several +minutes and require up to 8 GB of available RAM. +::: +``` + +``` +:::{admonition} Computationally intensive notebook +:class: danger +The computational resources required to execute this notebook exceed those +provided by standard laptop computers. To execute the notebook, please deploy +to a cluster computing environment. +::: +``` + +### Cell 4: Keywords, Summary and Learning Objectives + +In this cell, the contributor must provide a set of relevant keywords. E.g. + +``` +**Keywords:** gravity inversion, sparse-norm inversion, integral formulation, tree mesh. +``` + +Next, the contributor must provide a summary of the tutorial. The summary should be roughly one paragraph. If your tutorial focusses on particular functionality within SimPEG, please link to the API documentation. Also, link to tutorials you feel the reader should have already worked through. E.g. + +``` +**Summary:** Here we invert gravity anomaly data to recover a density contrast model. +We demonstrate two approaches for recovering a density contrast model: + +1. Weighted least-squares inversion for a tensor mesh +2. Iteratively re-weighted least-squares (IRLS) inversion for a tree mesh + +The *weighted least-squares* approach is a great introduction to geophysical inversion +with SimPEG. One drawback however, is that it recovers smooth structures which may not +be representative of the true model. To recover sparse and/or blocky structures, we +also demonstrate the *iteratively re-weighted least-squares* approach. Because this +tutorial focusses primarily on inversion-related functionality, we urge the reader to +become familiar with functionality explained in the [3D Forward Simulation of Gravity +Anomaly Data](fwd_gravity_anomaly_3d.ipynb) tutorial before working through this one. +``` + +Finally, the contributor must provide learning objectives using a bullet list. E.g. + +``` +**Learning Objectives:** + +- How geophysical inversion is carried out using SimPEG. +- How to assign appropriate uncertainties to gravity anomaly data. +- How to design a suitable mesh for gravity inversion when using the integral formulation. +- How to choose and set parameters for the inversion. +- How to define directives that are applied and updated throughout the inversion. +- How to applying the sensitivity weighting generally used in 3D gravity inversion. +- How to invert data using weighted least-squares and sparse-norm regularization. +- How to analyse inversion results. +``` + +## Import Functionality + +Following the introduction section, the contributor must import all functionality used by the tutorial and briefly explain the functionality that is most important. + +### Cell 1: Header and Summary + +This Markdown cell uses the heading **Import Modules**. For functionality that relates +directly to the purpose of the tutorial, some context and a link to API documentation +should be provided. E.g. + +::: +## Import Modules + +Here, we import all of the functionality required to run the notebook for the tutorial +exercise. All of the functionality specific to simulating gravity data are imported from +[simpeg.potential_fields.gravity](xref:simpeg#simpeg.potential_fields.gravity). We also +import some useful utility functions from [simpeg.utils](xref:simpeg#simpeg.utils). +To simulate gravity data, we need to define our problem geometry on a numerical grid +(or mesh). To generate the mesh, we used the +[discretize](https://discretize.simpeg.xyz/en/main) package. +::: + +### Cell 2: Code Cell + +The coding cell where functionality is imported should be organized. E.g. + +```python +# SimPEG functionality +from simpeg.potential_fields import gravity +from simpeg.utils import plot2Ddata, model_builder +from simpeg import maps + +# discretize functionality +from discretize import TensorMesh +from discretize.utils import mkvc, active_from_xyz + +# Common Python functionality +import numpy as np +from scipy.interpolate import LinearNDInterpolator +import matplotlib as mpl + +mpl.rcParams.update({"font.size": 14}) +import matplotlib.pyplot as plt +import os +``` + +## Tutorial Sections (and Subsections) + +Each section (or subsection) is constructed using a combination of Markdown and code cells. The structure and formatting of these are discussed below. + +### Header and Summary + +Each section (or subsection) begins with a Markdown cell containing the header and a summary. +Here, the contributor must: + +* summarize what is being done in the section +* describe any new functionality that is being introduced +* provide links to all relevant API documentation +* describe the choices for hyperparameter values used in the tutorial + +E.g. + +``` +## Define the Survey + +Surveys within SimPEG generally require the user to create and connect three +types of objects: + +- [receivers](xref:simpeg#simpeg.potential_fields.gravity.receivers.Point): +which define the locations of field measurements and type of data being measured. +- [sources](xref:simpeg#simpeg.potential_fields.gravity.sources.SourceField): +the passive or active sources responsible for generating geophysical responses, +and their associated receivers. +- [survey](xref:simpeg#simpeg.potential_fields.gravity.survey.Survey): the object +which stores and organizes all of the sources and receivers. + +Here, we define the survey that will be used for the forward simulation. Gravity +surveys are simple to create. The user only needs an (N, 3) +[numpy.ndarray](xref:numpy#numpy.ndarray) to define the xyz locations of the +observation locations, and a list of field components +which are to be measured. For the tutorial simulation, the receivers are located +5 m above the surface topography and spaced 10 m apart. +``` + +If something has been thoroughly explained in an introductory tutorial, you can avoid repetition by linking to that tutorial. +However, the choice in values used in your tutorial should always be stated. E.g. + +``` +## Assign Uncertainties + +Approaches for applying reasonable uncertainties to normalized voltage and apparent +resistivity data were presented in the [2.5D Inversion](inv_dcr_2d.ipynb) tutorial. +Here, we apply uncertainties of 1e-7 V/A + 10 % to the normalized voltage data being +inverted. +``` + +### Code Block Cells + +Code blocks must follow the [style required by SimPEG](https://docs.simpeg.xyz/latest/content/getting_started/contributing/code-style.html). +Please provide comments when defining new objects. This is especially important when exposing the reader to new functionality. +Avoid lengthy code blocks that can be better parsed into multiple cells. Markdown cells can also be added between code cells +to provide additional explanation to the reader. + +```python +# Define the component(s) of the field we want to simulate as strings within +# a list. Here we simulate only the vertical component of the gravity anomaly. +components = ["gz"] + +# Use the observation locations and components to define receivers for the entire survey +# in one step. The set of receivers, even if it's only 1, are organized within a list. +receiver_list = gravity.receivers.Point(receiver_locations, components=components) +receiver_list = [receiver_list] + +# Defining the source. For gravity surveys, we simply need to specific the list of +# receivers associated with the source field. +source_field = gravity.sources.SourceField(receiver_list=receiver_list) + +# Defining the survey. +survey = gravity.survey.Survey(source_field) +``` + +### Plotting Requirements + +The contributor is required to plot data, models and other quantities that are best understood visually. +Due to its widespread use, figures should be generated using [matplotlib](https://matplotlib.org/) package. +When generating a figure, please ensure: + +* the figure renders at an appropriate size when building the website +* the code used to generate the figure is compact whenever possible +* the text and features within the plot are legible + + +## Check Style of Notebooks + +We can check the code style of our notebooks using [`ruff`][ruff]. +Simply run the following command to check the style of the notebooks: + +```bash +ruff check notebooks +``` + +You can run the following that the notebooks are correctly formatted: + +```bash +ruff format --check notebooks +``` + +And run this to autoformat them: + +```bash +ruff format --fix notebooks +``` + +Alternatively, you can use the targets we have in the `Makefile`, like `make +check` and `make format`. Read more information about the available targets +by running `make help`. + +[ruff]: https://astral.sh/ruff +[install-mystmd]: https://mystmd.org/guide/quickstart +[jupyter]: https://jupyter.org +[mystmd.org]: https://mystmd.org diff --git a/notebooks/contributing/pull_request.md b/notebooks/contributing/pull_request.md new file mode 100644 index 0000000..79d8026 --- /dev/null +++ b/notebooks/contributing/pull_request.md @@ -0,0 +1,66 @@ +GitHub Pull Request and Review Process +====================================== + +Here, we describe the pull request and review process for bring new tutorial notebooks into the SimPEG user tutorials repository. + +Requirements Checklist +---------------------- + +The following is a useful checklist for determining whether the tutorial notebook has satisfied formatting and structure requirements: + +* **General:** + * Notebook is compatible with latest SimPEG release + * No deprecated functions and/or input arguments are being used + * Coding cells linted according to the [style guides](https://docs.simpeg.xyz/latest/content/getting_started/contributing/code-style.html). +* **The introduction is complete:** + * Title and author added to notebook + * Admonitions for notebook difficulty and computational resources have been added + * Keywords list has been added + * Summary paragraph describing the tutorial has been added + * Learning objectives have been listed + * Hyperlinks to other tutorial notebooks added if necessary +* **For sections and subsections:** + * A header and short summary of what is being done + * Links to API documentation added for all classes and functions that are used + * Newly introduced functionality is explained, or links are provided to relevant materials + * The approach taken when choosing hyperparameter values is explained. The use of ad hoc values without explanation is discouraged + * All figures are legible and rendered appropriately + +Stages of a Pull Request +------------------------ + +When you are ready to submit your tutorial notebook review, you will make a +GitHub pull request (PR). If your pull request is not ready for a final review, +but you require feedback, please mark it as a draft pull request. Once you +feel the pull request is ready for a final review, you can convert the draft PR to +an open PR by selecting the ``Ready for review`` button at the bottom of the page. + +Once a pull request is in ``open`` status and you are ready for review, please +ping ``dccowan`` and ``santisoler`` in a github comment to +request a review. At minimum for a PR to be eligible to merge, we look for + +- Structure and formatting requirements are observed. +- The notebooks runs properly using the latest release of SimPEG. +- All reviewer comments (if any) have been addressed. +- A developer approves the PR. + +After all these steps are satisfied, a ``@simpeg/simpeg-admin`` will merge your +pull request into the main branch (feel free to ping one of us on Github). + +This being said, all SimPEG developers and admins are essentially volunteers +providing their time for the benefit of the community. This does mean that +it might take some time for us to get your PR. + +Merging a Pull Request +---------------------- + +The ``@simpeg/simpeg-admin`` will merge a Pull Request to the `main` branch +using the `Squash and Merge +`_ +strategy: all commits made to the PR branch will be _squashed_ to a single +commit that will be added to `main`. + +SimPEG admins will ensure that the commit message is descriptive and +comprehensive. Contributors can help by providing a descriptive and +comprehensive PR description of the changes that were applied and the reasons +behind them. This will be greatly appreciated. \ No newline at end of file diff --git a/notebooks/contributing_index.md b/notebooks/contributing_index.md new file mode 100644 index 0000000..8396c63 --- /dev/null +++ b/notebooks/contributing_index.md @@ -0,0 +1,40 @@ +Contributing to User Tutorials +============================== + +SimPEG welcomes contributions by the community to improve available tutorial materials. +However, tutorials must adhere to specific guidelines in order to ensure a high level of quality and consistency. +For those interested in contributing to SimPEG User Tutorials, please follow the instructions provided. + + +Before Creating a New Tutorial +------------------------------ + +```{admonition} WARNING! +:class: danger +Please ensure that the contents of your tutorial **DOES NOT** fall into any of the following categories: + +* the tutorial focusses on functionality that is not part of SimPEG +* the amount of SimPEG functionality NOT already covered in another tutorial is insufficient +* the tutorial focusses on a data-specific result and is not generalizable +* the tutorial was created for the purpose of self promotion + +Tutorials that fall into any of these categories will not be accepted! +``` + +Steps for Adding a Tutorial +--------------------------- + +As explained on the [SimPEG user tutorials](../index.md) landing page, the SimPEG user tutorials are a library of [Jupyter Notebooks](https://jupyter.org/) that have been published as a website using [MyST](https://mystmd.org/). To add a tutorial notebook to the project, you will need to complete the following steps: + +**Step 1: Building the User Tutorials Website Locally** + +First, you will need all of the functionality required to run the existing tutorial notebooks and build the website locally with [MyST][mystmd.org]. +Instructions for creating an appropriate Python environment and building the website locally are found on the [Building the User Tutorials Website Locally](contributing/build_website.md) page. + +**Step 2: Creating a Tutorial Notebook** + +Jupyter notebooks that contain tutorials must adhere to specific structure and formatting requirements. +These are provided on the [Tutorial Structure and Formatting Requirements](contributing/formatting.md) page. + +**Step 3: The Review Process** +