Binder examples sometimes crash during build

[joschkabirk]

Hi

just wanted to have a look at the examples using the binder link :)

I noticed that the build crashes during the pip install.

Looks to me like the problem is the sklearn requirement in neos, since sklearn is now deprecated and should be replaced by scikit-learn.

Last few lines of the logs:

Collecting sklearn (from neos->-r examples/requirements.txt (line 3))
  Downloading sklearn-0.0.post7.tar.gz (3.6 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.
      
      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error

I think atm this exception is only raised in certain time windows. Just tried it outside of these time windows - then everything works fine and the notebook starts without issues 👍

Click me for full logs

Waiting for build to start...
Picked Git content provider.
Cloning into '/tmp/repo2dockerlnh9arue'...
HEAD is now at 2bff240 Merge pull request #59 from gradhep/remove-mac-ci
Python version unspecified, using current default Python version 3.10. This will change in the future.Building conda environment for python=3.10
Using PythonBuildPack builder
Step 1/49 : FROM docker.io/library/buildpack-deps:bionic
 ---> 0f137f206fa8
Step 2/49 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Using cache
 ---> 9a60f92d78f4
Step 3/49 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends locales > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 950fb7d941f4
Step 4/49 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen &&     locale-gen
 ---> Using cache
 ---> aad2c1ce6dff
Step 5/49 : ENV LC_ALL=en_US.UTF-8     LANG=en_US.UTF-8     LANGUAGE=en_US.UTF-8
 ---> Using cache
 ---> cb934374e1b5
Step 6/49 : ENV SHELL=/bin/bash
 ---> Using cache
 ---> 60a0cfa7ff62
Step 7/49 : ARG NB_USER
 ---> Using cache
 ---> 27c992051258
Step 8/49 : ARG NB_UID
 ---> Using cache
 ---> 626362bf33c6
Step 9/49 : ENV USER=${NB_USER}     HOME=/home/${NB_USER}
 ---> Using cache
 ---> 24abacc0f0c6
Step 10/49 : RUN groupadd         --gid ${NB_UID}         ${NB_USER} &&     useradd         --comment "Default user"         --create-home         --gid ${NB_UID}         --no-log-init         --shell /bin/bash         --uid ${NB_UID}         ${NB_USER}
 ---> Using cache
 ---> 5ba34ac3f7e1
Step 11/49 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends        gettext-base        less        unzip        > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 1a113221114c
Step 12/49 : EXPOSE 8888
 ---> Using cache
 ---> 43ece9cde4f1
Step 13/49 : ENV APP_BASE=/srv
 ---> Using cache
 ---> 5657b39a0484
Step 14/49 : ENV CONDA_DIR=${APP_BASE}/conda
 ---> Using cache
 ---> f8f78313702d
Step 15/49 : ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/notebook
 ---> Using cache
 ---> 1c67ea477fb9
Step 16/49 : ENV NPM_DIR=${APP_BASE}/npm
 ---> Using cache
 ---> 4e5d9aece07c
Step 17/49 : ENV NPM_CONFIG_GLOBALCONFIG=${NPM_DIR}/npmrc
 ---> Using cache
 ---> b86f9c4bccc3
Step 18/49 : ENV NB_ENVIRONMENT_FILE=/tmp/env/environment.lock
 ---> Using cache
 ---> 80ae064b87a3
Step 19/49 : ENV MAMBA_ROOT_PREFIX=${CONDA_DIR}
 ---> Using cache
 ---> 4cafd175b65f
Step 20/49 : ENV MAMBA_EXE=${CONDA_DIR}/bin/mamba
 ---> Using cache
 ---> a2fd0ffb89ec
Step 21/49 : ENV CONDA_PLATFORM=linux-64
 ---> Using cache
 ---> e0bfd399f34a
Step 22/49 : ENV KERNEL_PYTHON_PREFIX=${NB_PYTHON_PREFIX}
 ---> Using cache
 ---> 3cc71cfd4d10
Step 23/49 : ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
 ---> Using cache
 ---> 7ca083140a37
Step 24/49 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e10-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-44e764 /etc/profile.d/activate-conda.sh
 ---> Using cache
 ---> 0cbd98e041a6
Step 25/49 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e10-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e10-2dlinux-2d64-2elock-388e0b /tmp/env/environment.lock
 ---> Using cache
 ---> 3242406b91b0
Step 26/49 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e10-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dbase-2denv-2ebash-e5509f /tmp/install-base-env.bash
 ---> Using cache
 ---> fbe47a0ddf87
Step 27/49 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-base-env.bash' && rm -rf /tmp/install-base-env.bash /tmp/env
 ---> Using cache
 ---> bee2cdf4ee28
Step 28/49 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}
 ---> Using cache
 ---> 95f7e95b6a04
Step 29/49 : USER root
 ---> Using cache
 ---> 2120b46739b8
Step 30/49 : ARG REPO_DIR=${HOME}
 ---> Using cache
 ---> be02981960cc
Step 31/49 : ENV REPO_DIR=${REPO_DIR}
 ---> Using cache
 ---> 7c41df45494f
Step 32/49 : RUN if [ ! -d "${REPO_DIR}" ]; then         /usr/bin/install -o ${NB_USER} -g ${NB_USER} -d "${REPO_DIR}";     fi
 ---> Using cache
 ---> 7208f01def75
Step 33/49 : WORKDIR ${REPO_DIR}
 ---> Using cache
 ---> e3f692b0f6ac
Step 34/49 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}
 ---> Using cache
 ---> f50acf41adcd
Step 35/49 : ENV PATH=${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}
 ---> Using cache
 ---> 7e582363e01c
Step 36/49 : ENV CONDA_DEFAULT_ENV=${KERNEL_PYTHON_PREFIX}
 ---> Using cache
 ---> 2ffbb645f077
Step 37/49 : USER root
 ---> Using cache
 ---> c89f8406064c
Step 38/49 : COPY --chown=1000:1000 src/ ${REPO_DIR}/
 ---> 181e826a777e
Step 39/49 : LABEL repo2docker.ref="2bff24096f06068eba9b6ec75fdb0ab69a1243bc"
 ---> Running in 0b4cb45ab5cc
Removing intermediate container 0b4cb45ab5cc
 ---> 8802b8283483
Step 40/49 : LABEL repo2docker.repo="https://github.com/gradhep/relaxed"
 ---> Running in 6b91ee52741b
Removing intermediate container 6b91ee52741b
 ---> 71c8d6b3bd7c
Step 41/49 : LABEL repo2docker.version="2023.06.0+8.gd414e99"
 ---> Running in d2e96d9d058b
Removing intermediate container d2e96d9d058b
 ---> b448795a2949
Step 42/49 : USER ${NB_USER}
 ---> Running in 757681d13d85
Removing intermediate container 757681d13d85
 ---> d5e4d0de2a4a
Step 43/49 : RUN chmod +x binder/postBuild
 ---> Running in 3eb088b12b00
Removing intermediate container 3eb088b12b00
 ---> 60719b70cd24
Step 44/49 : RUN ./binder/postBuild
 ---> Running in 553490bd21bb
Processing /home/jovyan
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting equinox (from relaxed==0.3.0)
  Downloading equinox-0.10.10-py3-none-any.whl (132 kB)
     ��������������������������������������� 132.3/132.3 kB 4.0 MB/s eta 0:00:00
Collecting jaxopt>=0.7 (from relaxed==0.3.0)
  Downloading jaxopt-0.7-py3-none-any.whl (151 kB)
     ��������������������������������������� 151.2/151.2 kB 9.7 MB/s eta 0:00:00
Collecting optax>=0.1.2 (from relaxed==0.3.0)
  Downloading optax-0.1.5-py3-none-any.whl (164 kB)
     ��������������������������������������� 164.6/164.6 kB 8.3 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=4.6 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from relaxed==0.3.0) (4.6.3)
Collecting jax>=0.2.18 (from jaxopt>=0.7->relaxed==0.3.0)
  Downloading jax-0.4.13.tar.gz (1.3 MB)
     ���������������������������������������� 1.3/1.3 MB 11.4 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting jaxlib>=0.1.69 (from jaxopt>=0.7->relaxed==0.3.0)
  Downloading jaxlib-0.4.13-cp310-cp310-manylinux2014_x86_64.whl (71.6 MB)
     ���������������������������������������� 71.6/71.6 MB 17.8 MB/s eta 0:00:00
Collecting numpy>=1.18.4 (from jaxopt>=0.7->relaxed==0.3.0)
  Downloading numpy-1.25.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
     ���������������������������������������� 17.6/17.6 MB 26.7 MB/s eta 0:00:00
Collecting scipy>=1.0.0 (from jaxopt>=0.7->relaxed==0.3.0)
  Downloading scipy-1.11.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36.3 MB)
     ���������������������������������������� 36.3/36.3 MB 20.8 MB/s eta 0:00:00
Collecting absl-py>=0.7.1 (from optax>=0.1.2->relaxed==0.3.0)
  Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
     �������������������������������������� 126.5/126.5 kB 27.1 MB/s eta 0:00:00
Collecting chex>=0.1.5 (from optax>=0.1.2->relaxed==0.3.0)
  Downloading chex-0.1.82-py3-none-any.whl (94 kB)
     ���������������������������������������� 94.8/94.8 kB 14.9 MB/s eta 0:00:00
Collecting jaxtyping>=0.2.20 (from equinox->relaxed==0.3.0)
  Downloading jaxtyping-0.2.20-py3-none-any.whl (24 kB)
Collecting toolz>=0.9.0 (from chex>=0.1.5->optax>=0.1.2->relaxed==0.3.0)
  Downloading toolz-0.12.0-py3-none-any.whl (55 kB)
     ���������������������������������������� 55.8/55.8 kB 14.2 MB/s eta 0:00:00
Collecting ml-dtypes>=0.1.0 (from jax>=0.2.18->jaxopt>=0.7->relaxed==0.3.0)
  Downloading ml_dtypes-0.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
     ���������������������������������������� 1.0/1.0 MB 34.2 MB/s eta 0:00:00
Collecting opt-einsum (from jax>=0.2.18->jaxopt>=0.7->relaxed==0.3.0)
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     ���������������������������������������� 65.5/65.5 kB 16.1 MB/s eta 0:00:00
Collecting typeguard>=2.13.3 (from jaxtyping>=0.2.20->equinox->relaxed==0.3.0)
  Downloading typeguard-4.0.0-py3-none-any.whl (33 kB)
Building wheels for collected packages: relaxed, jax
  Building wheel for relaxed (pyproject.toml): started
  Building wheel for relaxed (pyproject.toml): finished with status 'done'
  Created wheel for relaxed: filename=relaxed-0.3.0-py3-none-any.whl size=11175 sha256=cd8e19cd2b66a59d6cf5432de41165cc43dc2d17515b54d43953a1f557b04ef3
  Stored in directory: /tmp/pip-ephem-wheel-cache-_bcw9wfc/wheels/bc/01/de/a88c3880a4c27c802b27f000f770a2fe391bdc11e738d133f7
  Building wheel for jax (pyproject.toml): started
  Building wheel for jax (pyproject.toml): finished with status 'done'
  Created wheel for jax: filename=jax-0.4.13-py3-none-any.whl size=1518708 sha256=6cfd4cac9a45e34a4f2d3365df1d7f4bbdcd08190684b728e71547c16f7629cb
  Stored in directory: /home/jovyan/.cache/pip/wheels/f3/7a/25/f297f69029b5e4064e4736a0c4b3996a44cc27781c120bcb99
Successfully built relaxed jax
Installing collected packages: typeguard, toolz, numpy, absl-py, scipy, opt-einsum, ml-dtypes, jaxtyping, jaxlib, jax, jaxopt, equinox, chex, optax, relaxed
Successfully installed absl-py-1.4.0 chex-0.1.82 equinox-0.10.10 jax-0.4.13 jaxlib-0.4.13 jaxopt-0.7 jaxtyping-0.2.20 ml-dtypes-0.2.0 numpy-1.25.1 opt-einsum-3.3.0 optax-0.1.5 relaxed-0.3.0 scipy-1.11.1 toolz-0.12.0 typeguard-4.0.0
Collecting git+http://github.com/phinate/pyhf.git@feat/jax-model-building (from -r examples/requirements.txt (line 2))
  Cloning http://github.com/phinate/pyhf.git (to revision feat/jax-model-building) to /tmp/pip-req-build-pog4fdb3
�[91m  Running command git clone --filter=blob:none --quiet http://github.com/phinate/pyhf.git /tmp/pip-req-build-pog4fdb3
�[0m�[91m  warning: redirecting to https://github.com/phinate/pyhf.git/
�[0m�[91m  Running command git checkout -b feat/jax-model-building --track origin/feat/jax-model-building
�[0m�[91m  warning: redirecting to https://github.com/phinate/pyhf.git/
�[0m�[91m  Switched to a new branch 'feat/jax-model-building'
�[0m�[91m  Branch 'feat/jax-model-building' set up to track remote branch 'feat/jax-model-building' from 'origin'.
�[0m  Resolved http://github.com/phinate/pyhf.git to commit b30a8854c8f6a240964f2a0406fb688720e443a9
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting celluloid (from -r examples/requirements.txt (line 1))
  Downloading celluloid-0.2.0-py3-none-any.whl (5.4 kB)
Collecting neos (from -r examples/requirements.txt (line 3))
  Downloading neos-0.3.0-py3-none-any.whl (7.0 kB)
Collecting plothelp (from -r examples/requirements.txt (line 4))
  Downloading plothelp-0.2-py3-none-any.whl (6.1 kB)
Collecting matplotlib (from celluloid->-r examples/requirements.txt (line 1))
  Downloading matplotlib-3.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
     ���������������������������������������� 11.6/11.6 MB 13.7 MB/s eta 0:00:00
Requirement already satisfied: scipy>=1.1.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2)) (1.11.1)
Collecting click>=8.0.0 (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2))
  Downloading click-8.1.6-py3-none-any.whl (97 kB)
     ���������������������������������������� 97.9/97.9 kB 7.9 MB/s eta 0:00:00
Collecting tqdm>=4.56.0 (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2))
  Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
     ���������������������������������������� 77.1/77.1 kB 7.6 MB/s eta 0:00:00
Requirement already satisfied: jsonschema>=3.0.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2)) (4.17.3)
Collecting jsonpatch>=1.15 (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2))
  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Requirement already satisfied: pyyaml>=5.1 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from pyhf==0.6.4.dev209->-r examples/requirements.txt (line 2)) (6.0)
Requirement already satisfied: relaxed>=0.2.0 in /srv/conda/envs/notebook/lib/python3.10/site-packages (from neos->-r examples/requirements.txt (line 3)) (0.3.0)
Collecting sklearn (from neos->-r examples/requirements.txt (line 3))
  Downloading sklearn-0.0.post7.tar.gz (3.6 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
�[91m  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.
      
      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error
      
      More information is available at
      https://github.com/scikit-learn/sklearn-pypi-package
      
      If the previous advice does not cover your use case, feel free to report it at
      https://github.com/scikit-learn/sklearn-pypi-package/issues/new
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
�[0m�[91merror: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
�[0mRemoving intermediate container 553490bd21bb
The command '/bin/sh -c ./binder/postBuild' returned a non-zero code: 1