Build support for Google Colab

[darribas]

It'd be good to have the book run on Google Colab.

[jeffcsauer]

@darribas was casually looking into this and the Google Colab launch buttons are working as intended. The problem seems to be that an automatically generated Google Colab for a given notebook will usually require the installation of geopandas, pysal, and others.

For example, trying to run Choropleth Mapping on a fresh Google Colab runtime will require the installation of geopandas, pysal. Depending on the order of the installation (i.e. geopandas before pysal), a full runtime restart will be required. This can be avoided by installing pysal before geopandas.

Is there any way to pass arguments to the automatically generated Google Colab such that certain modules will already be installed?

A similar question was raised in bookdown here.

Perhaps another solution would be to add the following caveat paragraph to each .ipynb:

If you are using Google Colab, please run the following code chunk to ensure that specialty packages like pysal are installed:

try:
  import pysal, geopandas [additional packages]
except:
  !pip install pysal geopandas [additional packages]
  import pysal, geopandas [additional packages]

[darribas]

Thanks for this Jeff. Yes, I think Collab opens fine from the buttons but on a "blank" stack with no geo libs installed. I suspect that'd create a bunch of confusion as you expect it to "just work" as it does with Binder for example. I haven't spent enough time on Collab to get a good solution up and running but would very much welcome solutions. Ideally, there'd be a way to pull a .yml file to set up a conda environment (or better a Docker image like we do on Binder), but I'm not sure it's that straightforward...

In my experience (although I haven't tried in a while), pip install things like geopandas usually doesn't go smooth... I might be wrong though.

[jeffcsauer]

Totally! Sounds good! A .yml file makes much more sense then reinstalling each time. I will continue to look into this - a previous course I took distributed Colab files with custom modules, but I don't think that would work in this setup.

If you need a 'crude' solution, I tested a few versions of the following lines on a 'blank' Google Colab runtime...

try:
  import pysal, geopandas [additional packages]
except:
  !pip install pysal geopandas [additional packages]
  import pysal, geopandas [additional packages]

...the above successfully installs pysal, geopandas, and their dependencies without needing to restart the runtime, but it is very sensitive to the ordering of the modules.

Will ping you if I find anything!

[jeffcsauer]

@darribas might have some progress. Adapted from this SO post. Was able to get the following lines to install a fresh conda environment in a new Google Colab runtime.

# Download latest version of miniconda installer for linux. This places the file in the general
# Google Colab directory.
!wget -c https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

# Change permissions on the download such that it can be insatlled to the 'local' 
# Google Colab runtime. 
!chmod +x Miniconda3-latest-Linux-x86_64.sh
!bash ./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/local

# Grab the desired conda environment that would include all necessary packages for GDS
# The one below does not actually contain pysal or geopandas...
!wget https://raw.githubusercontent.com/gdsbook/book/master/infrastructure/ga_environment.yml

# Create the conda environment
!conda-env create -f ga_environment.yml

# Activate the conda environment
!activate book

# code

As noted above, the .../infrastructure/ga_environment.yml doesn't actually include pysal or geopandas, so I can't be sure this 100% solves the problem. However, if the .yml file was updated with the necessary packages we could test it!

[darribas]

Dropping here this project which might make our life easier:

https://github.com/conda-incubator/condacolab

It seems that all would be required be:

!pip install -q condacolab
import condacolab
condacolab.install()

[darribas]

This seems important (URL):

#################################################################################
# INSTALL MAMBA ON GOOGLE COLAB
#################################################################################
! wget -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-py37_4.10.3-Linux-x86_64.sh
! chmod +x miniconda.sh
! bash ./miniconda.sh -b -f -p /usr/local
! rm miniconda.sh
! conda config --add channels conda-forge
! conda install -y mamba
! mamba update -qy --all
! mamba clean -qafy
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')