Skip to content

Commit

Permalink
Update install docs; switch to GH Actions (#235)
Browse files Browse the repository at this point in the history
* REL: better, more formal conda install

Installing pygraphviz with pip (but graphviz with conda) seems to
break stuff. this fixes things

(The README and environment.yml setup were taken from strainFlye,
which is ... code that I wrote)

* DEV/STY: Improve submodule introspection; style

* MNT: improve py test cov reports

* TST: switch from travis to gh actions

Closes #214. I doubt that this will work perfectly, but let's see
what goes wrong and fix accordingly

* STY/DEV: fix py style; update gitignore

* TST: fix Python CI (forgor to upd8 env name)
  • Loading branch information
fedarko authored Jun 24, 2022
1 parent 5a36246 commit 30b23bc
Show file tree
Hide file tree
Showing 10 changed files with 278 additions and 55 deletions.
35 changes: 35 additions & 0 deletions .github/workflows/js.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Based on
# https://github.com/fedarko/strainFlye/blob/main/.github/workflows/main.yml +
# https://github.com/biocore/empress/blob/master/.github/workflows/main.yml
name: MgSc JavaScript CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [14]

steps:

# https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
- name: Check out code
uses: actions/checkout@v3

- name: Set up Node.js
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}

- name: Install MetagenomeScope's Node.js development dependencies
run: npm install -g mocha-headless-chrome nyc [email protected] jshint

- name: Lint and stylecheck the JS code
run: make jsstylecheck

- name: Run JS tests
run: make jstest

- name: Upload JS code coverage information to Codecov
uses: codecov/codecov-action@v2
37 changes: 37 additions & 0 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Based on
# https://github.com/fedarko/strainFlye/blob/main/.github/workflows/main.yml
name: MgSc Python CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest

strategy:
matrix:
python-version: [3.6]

steps:

# https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
- name: Check out code
uses: actions/checkout@v3

# https://github.com/conda-incubator/setup-miniconda#example-3-other-options
- name: Install conda dependencies
uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: mgsc
environment-file: environment.yml
python-version: ${{ matrix.python-version }}

- name: Install MetagenomeScope (and its pip dependencies)
run: conda run -n mgsc pip install -e .[dev]

- name: Lint and stylecheck the Python code
run: conda run -n mgsc make pystylecheck

- name: Run Python tests
run: conda run -n mgsc make pytest

- name: Upload Python code coverage information to Codecov
uses: codecov/codecov-action@v2
139 changes: 134 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,138 @@
*.pyc
# (This is the basic Python gitignore from GitHub.)
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
.python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# vim
*.swp

# Extra MgSc-specific stuff
metagenomescope/tests/output/
spqrtests/
testgraphs/
Expand All @@ -8,17 +141,13 @@ viewer/demotests/
node_modules/
.DS_STORE
.DS_Store
*.egg-info/
*.bak
.coverage
js_coverage.json
metagenomescope/tests/js_tests/instrumented_js/*
dist/
big-list-of-naughty-strings/
*.gv
*.xdot
*.db
*.agp
mg2/
coverage/
metagenomescope/tests/input/extras/sheepgut_g1217.gfa
35 changes: 0 additions & 35 deletions .travis.yml

This file was deleted.

19 changes: 10 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
# based on the initial version of this Makefile, anyway, so it's a silly
# chicken-and-egg thing).

.PHONY: pytest jstest test
.PHONY: pytest jstest test pystylecheck jsstylecheck stylecheck pystyle jsstyle style demo

PYTEST_COMMAND = python3 -B -m pytest metagenomescope/tests/ --cov
PYTEST_COMMAND = python3 -B -m pytest metagenomescope/tests/ --cov-report xml --cov-report term --cov metagenomescope
PYLOCS = metagenomescope/ setup.py
JSLOCS = metagenomescope/support_files/js/*.js metagenomescope/tests/js_tests/*.js docs/js/extra_functionality.js .jshintrc
HTMLCSSLOCS = metagenomescope/support_files/index.html metagenomescope/tests/js_tests/*.html metagenomescope/support_files/css/viewer_style.css docs/404.html docs/index.html docs/css/mgsc_docs_style.css
Expand All @@ -35,25 +35,26 @@ jstest:

test: pytest jstest

stylecheck:
pystylecheck:
flake8 --ignore=E203,W503,E266,E501 $(PYLOCS)
black --check -l 79 $(PYLOCS)

jsstylecheck:
jshint $(JSLOCS)
prettier --check --tab-width 4 $(JSLOCS) $(HTMLCSSLOCS)

style:
stylecheck: pystylecheck jsstylecheck

pystyle:
black -l 79 $(PYLOCS)
@# To be extra safe, do a dry run of prettier and check that it hasn't
@# changed the code's abstract syntax tree (AST). (Black does this sort of
@# thing by default.)
prettier --debug-check --tab-width 4 $(JSLOCS) $(HTMLCSSLOCS)
prettier --write --tab-width 4 $(JSLOCS) $(HTMLCSSLOCS)

jsstyle:
@# Shorthand, for when I'm developing JS code and don't want to waste time
@# with python/HTML stuff
prettier --write --tab-width 4 $(JSLOCS)

style: pystyle jsstyle

demo:
@# makes a simple demo with a tiny graph that lets us test out the viewer
@# interface
Expand Down
18 changes: 15 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# MetagenomeScope
[![Build Status](https://travis-ci.org/marbl/MetagenomeScope.svg?branch=master)](https://travis-ci.org/marbl/MetagenomeScope) [![Code Coverage](https://codecov.io/gh/marbl/MetagenomeScope/branch/master/graph/badge.svg)](https://codecov.io/gh/marbl/MetagenomeScope)

[![MgSc Python CI](https://github.com/marbl/Metagenomescope/actions/workflows/python.yml/badge.svg)](https://github.com/marbl/MetagenomeScope/actions/workflows/python.yml)
[![MgSc JavaScript CI](https://github.com/marbl/Metagenomescope/actions/workflows/js.yml/badge.svg)](https://github.com/marbl/MetagenomeScope/actions/workflows/js.yml)
[![Code Coverage](https://codecov.io/gh/marbl/MetagenomeScope/branch/master/graph/badge.svg)](https://codecov.io/gh/marbl/MetagenomeScope)

![Screenshot of MetagenomeScope's standard mode, showing an example assembly graph from Nijkamp et al. 2013](https://user-images.githubusercontent.com/4177727/100696036-6aa7ab80-3347-11eb-8017-f693aae08aa2.png "Screenshot of MetagenomeScope showing an example assembly graph from Nijkamp et al. 2013.")
<div align="center">
Expand Down Expand Up @@ -40,8 +43,17 @@ Probably the easiest way to install MetagenomeScope is using a
[conda](https://docs.conda.io/en/latest/) environment:

```bash
conda create -n mgsc python=3.6 graphviz numpy
source activate mgsc
# Download the YAML file describing the conda packages we'll install
wget https://raw.githubusercontent.com/marbl/MetagenomeScope/main/environment.yml

# Create a new conda environment based on this YAML file
# (by default, it'll be named "mgsc")
conda env create -f environment.yml

# Activate this conda environment
conda activate mgsc

# Install the actual MetagenomeScope software
pip install git+https://github.com/marbl/MetagenomeScope.git
```

Expand Down
12 changes: 12 additions & 0 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: mgsc
channels:
- conda-forge
- defaults
dependencies:
- python = 3.6
- pip
- numpy
- graphviz
# version 1.3 gives me errors when accessing bounding boxes sometimes:
# github.com/pygraphviz/pygraphviz/issues/113#issuecomment-298631567
- pygraphviz >= 1.6
23 changes: 23 additions & 0 deletions metagenomescope/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,26 @@
#
# You should have received a copy of the GNU General Public License
# along with MetagenomeScope. If not, see <http://www.gnu.org/licenses/>.

# Import submodules so they're easy to see from REPL
from . import (
arg_utils,
assembly_graph_parser,
config,
file_utils,
input_node_utils,
layout_utils,
msg_utils,
)

# ... And explicitly declare them in __all__. This will stop flake8 from
# yelling at us about these imports being unused.
__all__ = [
"arg_utils",
"assembly_graph_parser",
"config",
"file_utils",
"input_node_utils",
"layout_utils",
"msg_utils",
]
9 changes: 9 additions & 0 deletions metagenomescope/tests/input/mod_scaffs.agp
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
aosdfi-scaffold 1 39949 1 W 66 1 39949 +
aosdfi-scaffold 39950 49953 2 W -8 1 5003 -
aosdfi-scaffold 49954 59955 3 W 6 1 10001 +
aosdfi-scaffold 59956 64959 4 W 58 1 5003 +
aosdfi-scaffold 64960 71522 5 W 94 1 6562 +
aosdfi-scaffold 71523 76533 6 W 111 1 5010 +
aosdfi-scaffold 76534 81583 7 W -44 1 5049 -
lol 1 6360 1 W 84 1 6360 +
lol 6361 11698 2 W -79 1 5337 -
Loading

0 comments on commit 30b23bc

Please sign in to comment.