Upgrade pyarrow #1317
Upgrade pyarrow #1317
Conversation
http://b/310037300
|
This could be a problem: |
Honestly, I think we can live with that. Very few metrics use GPUs and most of those are processing model files and so won't be affected by PyArrow versions one way or the other. There should be so few remaining GPU metrics that we can reasonably just plan on not enabling parquet support for them. |
Dockerfile.tmpl
Outdated
| @@ -114,7 +114,7 @@ RUN pip install spacy && \ | |||
|
|
|||
| # b/232247930: uninstall pyarrow to avoid double installation with the GPU specific version. | |||
| RUN pip uninstall -y pyarrow && \ | |||
| mamba install -y cudf cuml && \ | |||
| mamba install -y "pyarrow>=14.0.1" cudf cuml && \ | |||
There was a problem hiding this comment.
Let's see if this passes, if it does we'll upgrade both, if not, then we'll switch to only upgrading the CPU-image.
There was a problem hiding this comment.
Note that without this change, it does look like tests pass:
https://ci.kaggle.net/blue/organizations/jenkins/kernels%2Fdocker-python/detail/upgrade-pyarrow/1/pipeline/58/
So cudf is complaining, but it might work anyway...? Though we might not want to risk that.
There was a problem hiding this comment.
I'd prefer not to risk it.
http://b/310037300