new environment for 2024-3 cycle (#316)

* MNT #215 * CI #316 suspend ruff lingint for now * MNT #318 change --force to -y * MNT update channels per Ts&Cs * DOC #319 add requested guidance * MNT #315 move most installs to pip * DOC #316 changes * MNT #316 drop defaults channel * DOC #316 remove the pin on sphinx version * ENV #316 * ENV #316 no pin on this python * ENV #316 pin py<3.12 is more flexible * DOC #215 update README
BCDA-APS · Oct 23, 2024 · e4f4f59 · e4f4f59
1 parent 5aedb8a
commit e4f4f59
Show file tree

Hide file tree

Showing 7 changed files with 279 additions and 119 deletions.
diff --git a/.github/workflows/code.yml b/.github/workflows/code.yml
@@ -32,10 +32,11 @@ jobs:
           set -vxeuo pipefail
           python -m pip install --upgrade pip
   
-      - name: Run ruff
-        uses: davidslusser/[email protected]
-        with:
-          python_version: "3.11"
+      # 2024-09-20,prj: disable until code is compliant
+      # - name: Run ruff
+      #   uses: davidslusser/[email protected]
+      #   with:
+      #     python_version: "3.11"
 
   tests:
     name: CI py${{ matrix.python-version }}

diff --git a/CHANGES.rst b/CHANGES.rst
@@ -21,10 +21,15 @@ Project `milestones <https://github.com/BCDA-APS/bluesky_training/milestones>`_
 describe future plans.
 
 ..
-   1.0.4
-   ******
+    1.0.4
+    ******
 
-   release expected by 2024-07-19
+    release expected by 2024-12-31?
+
+    Maintenance
+    ------------
+
+    * Change environment installation to prefer pip for most packages.
 
 1.0.3
 ******

diff --git a/bluesky/environments/README.md b/bluesky/environments/README.md
@@ -11,7 +11,8 @@ The repository is: https://github.com/BCDA-APS/bluesky_training/
 
 version | file
 --- | ---
-2024-2 (latest) | [`environment_2024_2.yml`](./environment_2024_2.yml)
+2024-2 (latest) | [`environment_2024_3.yml`](./environment_2024_3.yml)
+2024-1 | [`environment_2024_2.yml`](./environment_2024_2.yml)
 2024-1 | [`environment_2024_1.yml`](./environment_2024_1.yml)
 2023-3 | [`environment_2023_3.yml`](./archive/environment_2023_3.yml)
 2023-2 | [`environment_2023_2.yml`](./archive/environment_2023_2.yml)

diff --git a/bluesky/environments/environment_2024_3.yml b/bluesky/environments/environment_2024_3.yml
@@ -0,0 +1,129 @@
+name: bluesky_2024_3
+
+# download:
+#   wget https://raw.githubusercontent.com/BCDA-APS/bluesky_training/main/bluesky/environments/environment_2024_3.yml
+# create:
+#   conda env create -y -n bluesky_2024_3 -f ./environment_2024_3.yml --solver=libmamba
+# activate:
+#   conda activate bluesky_2024_3
+
+# Add/revise environment variable for default bluesky environment.
+# Add to ~/.bash_aliases (or if that does not exist, ~/.bashrc).
+# export BLUESKY_CONDA_ENV=bluesky_2024_3
+
+# For more advice about bash environment variables, see:
+# https://github.com/BCDA-APS/bluesky_training/blob/main/bluesky/environments/admin/bluesky.md#bash-environment-variables
+
+channels:
+  # - apsu  # Must be first for compatible epics-base, pyepics, pvapy
+  - conda-forge
+
+  # due to licensing Terms & Conditions,  don't use the defaults channel.
+  - nodefaults
+  # Note this advice: https://stackoverflow.com/questions/67695893/
+
+dependencies:
+  # Use conda to install only the basics and whatever pip can't install.
+  # This includes binaries such as Qt and hkl.
+  # Everything else by pip.
+
+  # Some Bluesky packages are not ready for 3.12+.
+  # Since this uses pip to install most packages, their
+  # requirements will not be handled until after the
+  # conda environment is installed.
+  - python <3.12
+  - pip
+  - apsu::pvapy
+  - apsu::aps-dm-api >=8   # linux-64  osx-64 -- ONLY at APS!
+  - hkl   # linux-64
+  - nbconvert-pandoc
+  - pandoc >3
+  - pyqt =5
+  - qt =5
+
+  - pip:
+
+    # APS-only
+    # - apsbss  # only works on APS subnet
+
+    # user-requested visualization packages
+    - c2dataviewer  # EPICS PVA images
+    - gemviz  # from tiled
+    - mdaviz  # MDA data files
+    - nexpy  # HDF5 and SPEC data files
+    - pymca  # HDF5 and SPEC data files
+
+    # --- user-requested packages
+    - https://github.com/BCDA-APS/adl2pydm/archive/main.zip
+    - ophyd-registry
+    - punx
+    # - pvmail
+    - pvview
+    - pyRestTable
+    - spec2nexus
+    - xrayutilities
+
+    # --- Python core packages
+    - ipython
+    - jupyter
+    - jupyterlab
+    - notebook
+    - numpy <1.999  # avoid any and all numpy 2.0 releases (includes 2.0.0aNNN)
+    - psutil
+
+    # --- general support packages
+    - apischema
+    - bitshuffle
+    - h5py
+    - hdf5plugin
+    # - httpie
+    - imagecodecs-lite
+    - ipympl
+    - lxml
+    - pandas
+    - pint
+    - pymongo
+    # - pysumreg
+    - scikit-image
+    - xlrd
+
+    # --- Bluesky framework packages
+    - apstools
+    - area-detector-handlers
+    - bluesky >=1.12
+    - bluesky-httpserver
+    - bluesky-kafka
+    - bluesky-live
+    - bluesky-queueserver
+    - bluesky-queueserver-api
+    - bluesky-widgets
+    - databroker <=1.999
+    - databroker-pack
+    - happi
+    - hklpy >=1.1.1
+    - httpx >=0.14  # issue #75
+    - ophyd >=1.9
+    - pydm
+    - tiled[all]
+    # https://github.com/pydata/xarray/issues/6818
+    - xarray !=2022.6.0  # incompatible with numpy=1.20
+
+    # --- testing and quality assurance
+    - black
+    - flake8
+    - pylint
+    - pytest
+    - pytest-notebook
+    - ruff
+
+    # --- packaging and publishing
+    - coverage
+    - myst-parser
+    - nbsphinx
+    - pydata-sphinx-theme
+    - pygithub
+    - sphinx
+    - sphinx-design
+    - sphinx-tabs
+    - sphinxcontrib-napoleon
+    - twine
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -8,6 +8,6 @@ nbsphinx
 pydata-sphinx-theme
 pygments >=2.12
 pygments-ipython-console
-sphinx <6
+sphinx
 sphinx-design
 sphinx-tabs
diff --git a/docs/source/reference/_create_conda_env.rst b/docs/source/reference/_create_conda_env.rst
@@ -104,7 +104,7 @@ bash? <https://bcda-aps.github.io/bluesky_training/reference/_FAQ.html#faq-bash>
          <pre>
          $ <b>source /PATH/TO/CONDA/bin/activate</b>
          </pre>
-         
+
       However, if the command ``which conda`` does not return anything, or if
       you are getting an error message (``bash: conda: command not found`` or
       ``bash: activate: No such file or directory``), conda is not installed on
@@ -167,11 +167,11 @@ bash? <https://bcda-aps.github.io/bluesky_training/reference/_FAQ.html#faq-bash>
 
          # update ~/.bashrc to activate base environment on login
          conda init
-         </code></pre>     
-         
+         </code></pre>
+
          </details>
          <br />
-      
+
 
       If you still encounter the same error message after installing conda or
       miniconda, you may need to add the conda installation directory to your
@@ -216,7 +216,7 @@ prompt is also prefixed with the environment name, as mentioned above.
     <pre>
     $ <b>conda config --show channels</b>
     </pre>
-    
+
     To add more channels:
 
     <pre>
@@ -244,7 +244,7 @@ prompt is also prefixed with the environment name, as mentioned above.
     href="https://conda.github.io/conda-libmamba-solver/">conda-libmamba-solver</a>
     package. If you have installed miniconda using the instructions for Linux
     above, this package is already installed and configured as the default solver.
-    
+
     You can check your default solver with the following command:
 
     <pre>
@@ -261,7 +261,7 @@ prompt is also prefixed with the environment name, as mentioned above.
     <li>Set libmamba as the default solver:</li>
     <pre>
     $ <b>conda config --set solver libmamba</b>
-    </pre>      
+    </pre>
     <li>Confirm libmanda is now your default solver:</li>
     <pre>
     $ <b>conda config --show solver</b>
@@ -281,7 +281,15 @@ The following commands install the ``bluesky_2023_3`` environment inside the
 (see `setup a bluesky instrument
 <https://bcda-aps.github.io/bluesky_training/instrument/_install_new_instrument.html#setup-a-bluesky-instrument>`__).
 
+.. hint:: Software requirements evolve.  Use the newest ``environment*.yml``
+   file in the ``environments/`` directory.  This command gives a directory
+   listing in chronological order:
 
+   .. raw:: html
+
+      <pre>
+      $ <b>ls -larth /environments/environment*.yml</b>
+      </pre>
 
 Note that the installation takes several minutes.
 
@@ -302,17 +310,17 @@ Note that the installation takes several minutes.
     <li>The <code>-y</code> option will replace any existing environment by this
     name without asking for confirmation. Remove this option if you wish.</li>
     <li>The <code>-n bluesky_2023_3</code> sets the name of the conda environment to be
-    created.</li> 
-    <li>The <code>-f ./environments/environment_2023_3.yml</code> option names the 
+    created.</li>
+    <li>The <code>-f ./environments/environment_2023_3.yml</code> option names the
     YAML file to be used. We create different versions of the YAML file, named
     for the APS operating cycle (2021-1, 2023-2, …), as the suite of packages
     for a working installation may change over time. By keeping all these files
     in the environments subdirectory, we can restore any of these environments
     with a simple command.</li>
-    <li>The <code>--solver=libmamba</code> option will use the conda-libmamba-solver. 
-    The <code>--solver</code> option can be removed but its use results in a much 
-    faster installation of the bluesky environment. The <code>libmamba</code> installation 
-    is described in the previous section.</li> 
+    <li>The <code>--solver=libmamba</code> option will use the conda-libmamba-solver.
+    The <code>--solver</code> option can be removed but its use results in a much
+    faster installation of the bluesky environment. The <code>libmamba</code> installation
+    is described in the previous section.</li>
     </ul>
 
     </details>
@@ -348,35 +356,35 @@ are configuration files for your bash shell. Here's a simple step-by-step guide:
 
    <ol>
    <li>Open a terminal.</li>
-   <li>Open the <code>~/.bashrc</code> and <code>~/.bash_aliases</code> files with your prefered text editor, 
+   <li>Open the <code>~/.bashrc</code> and <code>~/.bash_aliases</code> files with your prefered text editor,
    <i>e.g.</i>:
    <pre>
    $ <b> gedit ~/.bashrc ~/.bash_aliases </b>
    </pre>
-   If any of those files do not exist, this command will create blank ones. 
+   If any of those files do not exist, this command will create blank ones.
    </li>
-   <li> In <code>~/.bashrc</code>, scroll down to the end of the file or find 
+   <li> In <code>~/.bashrc</code>, scroll down to the end of the file or find
    a suitable place to add the following lines:
    <pre>
    source ~/.bash_aliases
    </pre>
    <b>Note:</b> this line may already be included in your <code>~/.bashrc</code>.
    </li>
-   <li>In <code>~/.bash_aliases</code>, scroll down to the end of the file or find 
-   a suitable place to add your alias. 
+   <li>In <code>~/.bash_aliases</code>, scroll down to the end of the file or find
+   a suitable place to add your alias.
    On a new line, type:
    <pre>
    export BLUESKY_CONDA_ENV=bluesky_2023_3
    alias become_bluesky='conda activate ${BLUESKY_CONDA_ENV}'
    </pre>
-   </li>  
+   </li>
    <li>Save your changes.</li>
    <li>Type <code>bash</code> and press enter, or open a new terminal windows to
    make the new alias available.</li>
 
    </ol>
    You can now use the alias <code>become_bluesky</code> to activate the bluesky
-   environment. 
+   environment.