Merge pull request #51 from CUAHSI/subsetter_argo

Subsetter argo
CUAHSI · Nov 16, 2023 · d453c7f · d453c7f
2 parents b70c986 + 5bbfad0
commit d453c7f
Show file tree

Hide file tree

Showing 136 changed files with 11,990 additions and 122 deletions.
diff --git a/.github/workflows/frontend-gh-pages.yml b/.github/workflows/frontend-gh-pages.yml
@@ -0,0 +1,58 @@
+# Simple workflow for deploying static content to GitHub Pages
+name: Deploy static content to Pages
+
+on:
+  push:
+  workflow_dispatch:
+
+# Sets the GITHUB_TOKEN permissions to allow deployment to GitHub Pages
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+# Allow one concurrent deployment
+concurrency:
+  group: 'pages'
+  cancel-in-progress: true
+
+defaults:
+  run:
+    working-directory: './app/frontend'
+
+jobs:
+  # Single deploy job since we're just deploying
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Set up Node
+        uses: actions/setup-node@v3
+        with:
+          node-version: 20
+          cache: 'npm'
+          cache-dependency-path: '**/package-lock.json'
+      - name: create env file
+        run: |
+          touch .env
+          echo VITE_APP_API_URL=${{ secrets.VITE_APP_API_URL }} >> .env
+          echo VITE_APP_URL=${{ secrets.VITE_APP_URL }} >> .env
+          cp .env ../.env
+      - name: Install dependencies
+        run: npm install
+      - name: Build
+        run: npm run build
+      - name: Setup Pages
+        uses: actions/configure-pages@v3
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v1
+        with:
+          # Upload dist repository
+          path: './app/frontend/dist'
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v1
diff --git a/README.md b/README.md
@@ -1,136 +1,22 @@
 # CONUS Subsetting
 
-This project is an experimental web service for subsetting model domain files at CONUS scales.
+This is a CUAHSI service for providing access to static domain datasets for CONUS hydrologic models.
 
-## Installation
+## Run Service
 
-This instructions are for a completely fresh install on a CentOS 7 Linux box.
+This service uses Docker to containerize dependencies for the subsetting operations. As such, we recommend that you use one of the pre-configured `docker-compose` files to run the service.
 
-1. Update Centos7 packages, and install git, vim, screen
-```
-yum update
-yum install -y vim screen libcurl-devel openssl-devel libjpeg*devel* netcdf*devel* hdf-devel gdal gdal-devel proj proj-devel proj-epsg proj-nad udunits2-devel libxml2-devel nco libpng-devel
-
-yum groupinstall 'Development Tools'
-```
-
-2. Install Anaconda or Miniconda
-```
-wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
-sh Miniconda3-latest-Linux-x86_64.sh
-rm Miniconda3-latest-Linux-x86_64.sh
-source ~/.bashrc
-conda config --set auto_activate_base false
-conda config --add channels conda-forge
-```
-
-4. create the environment for running the subsetter
-```
-conda env create -f environment.yml
-```
-
-5. Move proj dat into conda directory
-```
-# define the conda directory, this will be specific to your installation
-CONDA_ENV_DIR=/home/acastronova/.conda/envs/subsetting-server/share/proj
-
-cp /usr/share/proj/proj_def.dat $CONDA_ENV_DIR
-```
-
-5. Prepare data directories
-```
-VERSION=nwm.v1.2.4
-mkdir /share/appdata
-mkdir /share/appdata/output 
-
-cd /share
-wget -r -np -nH -R "index.html*" --cut-dirs=5 https://www.nco.ncep.noaa.gov/pmb/codes/nwprod/$VERSION/parm/domain/
-mv domain $VERSION
-```
-
-6. Install R 
-```
-yum install epel-release
-yum install R
-```
-
-Search for CRAN repositories in the USA
-```
-https://cran.r-project.org/mirrors.html
-```
+### Development Mode
 
-Choose a mirror and set it to the defualt using the following command
 ```
-URL=https://mirrors.nics.utk.edu/cran/
-echo "options(repos=structure(c(CRAN=\"$URL\")))" > ~/.Rprofile
+docker-compose -f docker/docker-compose.dev up -d
 ```
 
-7. Install R libraries for WRF-Hydro subsetting
-```
-R
-> install.packages(c('devtools', 'rgdal', 'RNetCDF', 'ggmap', 'dataRetrieval', 'data.table'), dependencies=TRUE)
-> devtools::install_github("NCAR/rwrfhydro")
-```
 
-# Configure Environment Settings
+### Production Mode
 
+The production version of this service is configured with SSL.
 
 ```
-vim nwm_subsetting/tornado-app/environment.py
-
-# adjust settings, e.g. :
-
-address="0.0.0.0"
-port="80"
-
-debug=True
-static_path = os.path.join(os.path.dirname(__file__), "static")
-template_path = os.path.join(os.path.dirname(__file__), "templates")
-
-# multiprocessing settings
-worker_count = 4
-
-# logging settings
-log_dir = os.path.join(os.getcwd(), 'logs')
-log_file_size = 1024 * 20 * 1000
-log_count = 10
-
-# levels: DEBUG, INFO, WARNING, ERROR, CRITICAL
-access_level = 'INFO'
-application_level = 'INFO'
-general_level = 'INFO'
-
-# WRF-HYDRO settings
-wrfdata = '/share/nwm.v1.2.4'
-
-# Output subset directory
-output_dir = '/share/appdata/output'
-
-# Location of jobs database
-sqldb = '/share/appdata/jobs.db'
+docker-compose -f docker/docker-compose.prod up -d
 ```
-
-# Build Bounding Box Lookup Database
-
-todo
-
-# Start the Server
-
-There are many ways to run the server, one simple method is shown below.
-```
-screen -S server
-cd nwm_subsetting/tornado-app/
-conda activate subset
-./app.py
-```
-
-# Setup
-
-1. Install R 3.3
-
-2. Install Python 3
-
-3. Open firewall ports
-   - `$ sudo iptables -I INPUT 1 -i eth0 -p tcp --dport 8000 -j ACCEPT`
-
-
diff --git a/app/.gitignore b/app/.gitignore
@@ -0,0 +1,160 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
diff --git a/app/LICENSE b/app/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 CUAHSI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.