RDkit 2020.09.4 release issues with Conda-forge build and other modules (Python3-saml) #92
Comments
|
Python3-saml_fail_conda_conflict_report_Feb2021.txt The above is the output on a failed install, details the conflicts - hopefully this saves some testing time. |
|
I just tried the simple experiment of creating a python 3.8 environment, installing the rdkit from conda-forge (gets me v2020.09.4 of the RDKit), and then trying to install python3-saml. I can confirm that this then wants to downgrade the RDKit to v2019.09.3 Similarly if I start by installing python3-saml and then try to install the rdkit, it goes with 2019.09.3 I'm just guessing, because untangling the rats' nest of dependencies is tricky, but it looks like the problem is caused by the fact that python3-saml comes from the main channel (not conda-forge) and depends on some other packages which are only available from conda main... one that seems particularly problematic seems to be libxmlsec1. Those then bring in a build of libxml2 from main and it looks like that may break something that more recent rdkit installs are looking for. ack... it's a mess I'll see if there's any way to reduce the number of explicit dependencies that the rdkit conda-forge package has, but I'm not terribly optimistic about that. @jaimergp: maybe this is something we can look at together? I suspect I'll reach the end of my ability to really figure this out before the full problem is solved. |
|
You don't want to mix
|
|
The environment is solvable if you specify everything at the same time, but indeed goes back to Unfortunately,
So, in short, |
|
In case you were wondering, the actual conflict for |
|
yeah, I thought it was boost at first too, but I don't see how that can be true if python3-saml doesn't install boost: |
|
more data, ignoring the rdkit entirely, python3-saml will not install with |
It might be a secondary (or n-ary) dependency of their dependencies 🤷 Nice find about |
If that were the case, wouldn't boost still end up being installed if I install python3-saml? Because that's not what happens. |
|
That would be the natural thing, but some packages use
I don't think that's what's happening, but untangling this is a rabbit hole and anyway we will need a conda-forge package in the end... |
|
Added terse part of the issue here to conda-forge/staged-recipes and tagged in @jaimergp |
|
PR here conda-forge/staged-recipes#13880 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What:
Fresh install on Ubuntu 20.04 with miniconda3, python 3.7 RDKit installs ok but Python3-saml has become incompatible.
First noted with 2020.09.04 but after experiments found first version where break occurred - 2020.03.6
Terse:
Demonstrated with
python3-saml
Some issues with rpy2 but:
appears to install and work ok on Ubuntu 20.04 (not Mac OSX)
TERSE:
Up to, and including rdkit==2019.09.3 python3-saml==1.9.0 would install and work together
With a python 3.7 condo env:
Python3-saml failed to install with multiple conflicts
- there are references to Python 3.8 - did we miss something and RDkit is Python3.8 now?
rdkit==2020.09.3 and rdkit==2020.09.4 also are incompatible
Good News - There is compatibility with (this is a section of our build script)
For addition clarity:
These are the results on Ubuntu 20, on Mac OSX Catalina things are worse (that will required another day of experimentation - Rdkit 2018 appears stable here with all other module combinations)
Verbose (-vvv)
These are the experiments I performed focussing on just RDkit and Python3-saml for reproducibility.
Experiment details:
All builds were:
- performed on Amazon Web Services (AWS) Ubuntu 20.04 instances (unless otherwise stated), using t2.large
- MINICONDA="Miniconda3-4.5.12-Linux-x86_64.sh"
- with Python 3.7 environments
- as root user
After each experiment the AWS instance was terminated and a new instance used each time.
Control - 1 - establish older version of RDkit installs with
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2019.03.1.0 -y
conda install -c conda-forge python3-saml
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.8.3
latest version: 4.9.2
Please update conda by running
Package Plan
environment location: /root/miniconda3/envs/test_rdkit
added / updated specs:
- python3-saml
The following packages will be downloaded:
The following NEW packages will be INSTALLED:
defusedxml conda-forge/noarch::defusedxml-0.6.0-py_0
isodate conda-forge/noarch::isodate-0.6.0-py_1
libgcrypt conda-forge/linux-64::libgcrypt-1.8.7-h36c2ea0_0
libgpg-error pkgs/main/linux-64::libgpg-error-1.41-h2531618_0
libtool conda-forge/linux-64::libtool-2.4.6-h58526e2_1007
libxmlsec1 pkgs/main/linux-64::libxmlsec1-1.2.29-h8fe84ac_0
libxslt pkgs/main/linux-64::libxslt-1.1.34-hc22bd24_0
lxml pkgs/main/linux-64::lxml-4.6.2-py37h9120a33_0
python3-saml pkgs/main/noarch::python3-saml-1.9.0-py_0
xmlsec pkgs/main/linux-64::xmlsec-1.3.3-py37h0e8eec6_0
The following packages will be UPDATED:
libxml2 conda-forge::libxml2-2.9.9-h13577e0_2 --> pkgs/main::libxml2-2.9.10-hb55368b_3
....lots
conda list
...
python-dateutil 2.8.1 py_0 conda-forge
python3-saml 1.9.0 py_0
python_abi 3.7 1_cp37m conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
rdkit 2019.03.1 py37h9c20d5c_0 conda-forge
readline 8.1 h27cfd23_0
...
SUCCESSFUL build***
Control 2 - newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2019.09.3 -y
conda install -c conda-forge python3-saml
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.8.3
latest version: 4.9.2
Please update conda by running
Package Plan
environment location: /root/miniconda3/envs/test_rdkit
added / updated specs:
- python3-saml
The following packages will be downloaded:
The following NEW packages will be INSTALLED:
defusedxml conda-forge/noarch::defusedxml-0.6.0-py_0
gettext conda-forge/linux-64::gettext-0.19.8.1-h5e8e0c9_1
isodate conda-forge/noarch::isodate-0.6.0-py_1
libgcrypt conda-forge/linux-64::libgcrypt-1.8.7-h36c2ea0_0
libgpg-error conda-forge/linux-64::libgpg-error-1.39-he1b5a44_0
libtool conda-forge/linux-64::libtool-2.4.6-h58526e2_1007
libxmlsec1 pkgs/main/linux-64::libxmlsec1-1.2.29-h8fe84ac_0
libxslt pkgs/main/linux-64::libxslt-1.1.33-h7d1a2b0_0
lxml conda-forge/linux-64::lxml-4.4.1-py37h7ec2d77_0
python3-saml pkgs/main/noarch::python3-saml-1.9.0-py_0
xmlsec pkgs/main/linux-64::xmlsec-1.3.3-py37h0e8eec6_0
The following packages will be DOWNGRADED:
boost 1.72.0-py37h48f8a5e_1 --> 1.70.0-py37h9de70de_1
boost-cpp 1.72.0-h9359b55_3 --> 1.70.0-ha2d47e9_1
cairo 1.16.0-h3fc0475_1005 --> 1.16.0-h18b612c_1001
fontconfig 2.13.1-h7e3eb15_1002 --> 2.13.1-he4413a7_1000
icu 67.1-he1b5a44_0 --> 58.2-hf484d3e_1000
libxml2 2.9.10-h68273f3_2 --> 2.9.9-h13577e0_2
rdkit 2019.09.3-py37he9dd633_1 --> 2019.09.3-py37hb31dc5d_0
conda list
...
python-dateutil 2.8.1 py_0 conda-forge
python3-saml 1.9.0 py_0
python_abi 3.7 1_cp37m conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
rdkit 2019.09.3 py37hb31dc5d_0 conda-forge
readline 8.1 h27cfd23_0
setuptools 52.0.0 py37h06a4308_0
...
SUCCESSFUL build***
Expt 1 - moving newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.03.6 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
Conda detects conflicts between RDkit install and Python3-SAML
Expt 2 - newer version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
#Conda init bash
#
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.09.3 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
--> Fails to install python3-saml and reports a huge amount of output - see included file.
Expt 3 - latest version of RDkit and Python3-saml
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge rdkit==2020.09.4 -y
conda install -c conda-forge python3-saml
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
--> Fails to install python3-saml and reports a huge amount of output - see included file.
Expt 4 - Install Python3-saml first then RDkit - check if it works the other way
Ubuntu 20.04
export MINICONDA="Miniconda3-py37_4.8.3-Linux-x86_64.sh"
wget https://repo.continuum.io/miniconda/$MINICONDA
bash $MINICONDA -b -p /root/miniconda3
export PATH="/root/miniconda3/bin:$PATH"
conda create -n test_rdkit python=3.7 -y
source activate test_rdkit
pip install --upgrade pip
conda install -c conda-forge python3-saml -y
conda install -c conda-forge rdkit==2020.09.4 -y
!!!!!!!!UNSUCCESSFUL INSTALL!!!!!!!!
The text was updated successfully, but these errors were encountered: