Add Docker docs #157

machawk1 · 2024-05-22T18:26:46Z

The repo has a Dockerfile. How does one use it? Let's add documentation. If it's unstable/unreliable, it would be worthwhile to mention this as well.

Obtain yamz.sql dump that aligns with current production / main branch code (see issue-157 branch)

machawk1 · 2024-05-22T18:28:09Z

For example, it references a yamz.sql, which does not exist in the repo. I believe I discussed this file with @cr625 in the past.

machawk1 · 2024-05-22T18:30:04Z

An old commit https://github.com/metadata-research/yamz/tree/778566e10eadf2e2eeddd7ad859f96df11b423c8 has a yamz.sql file but it has since been deleted.

machawk1 · 2024-05-22T18:36:47Z

Updated the config file reference in 286e5ba . Pushed straight to main, as it should not affect anything else.

Even with these changes, the container fails to build (macOS M1) when installing the requirements.

output

machawk1@Mat-mini yamz % docker build -t yamz .
[+] Building 15.1s (16/18)                                                                                                                                                  
 => [internal] load build definition from Dockerfile                                                                                                                   0.0s
 => => transferring dockerfile: 82B                                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                                      0.0s
 => => transferring context: 35B                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal                                                                                                        0.2s
 => [ 1/14] FROM docker.io/library/ubuntu:focal@sha256:874aca52f79ae5f8258faff03e10ce99ae836f6e7d2df6ecd3da5c1cad3a912b                                                0.0s
 => [internal] load build context                                                                                                                                      0.0s
 => => transferring context: 343B                                                                                                                                      0.0s
 => CACHED [ 2/14] RUN apt-get update && apt-get install -y tzdata python3 python3-pip git                                                                             0.0s
 => CACHED [ 3/14] RUN apt-get install -y postgresql-12                                                                                                                0.0s
 => CACHED [ 4/14] RUN git clone https://github.com/metadata-research/yamz.git                                                                                         0.0s
 => CACHED [ 5/14] RUN chown -R postgres /yamz                                                                                                                         0.0s
 => CACHED [ 6/14] RUN echo "host all  all    0.0.0.0/0  md5" >> /etc/postgresql/12/main/pg_hba.conf                                                                   0.0s
 => CACHED [ 7/14] RUN echo "listen_addresses='*'" >> /etc/postgresql/12/main/postgresql.conf                                                                          0.0s
 => CACHED [ 8/14] RUN /etc/init.d/postgresql start &&    psql --command "ALTER USER postgres WITH SUPERUSER PASSWORD 'PASS';" &&    createdb -O postgres yamz         0.0s
 => CACHED [ 9/14] WORKDIR /yamz                                                                                                                                       0.0s
 => CACHED [10/14] COPY _config.py /yamz/config.py                                                                                                                     0.0s
 => CACHED [11/14] COPY requirements.txt .                                                                                                                             0.0s
 => ERROR [12/14] RUN python3 -m pip install -r requirements.txt                                                                                                      14.8s
------                                                                                                                                                                      
 > [12/14] RUN python3 -m pip install -r requirements.txt:                                                                                                                  
#16 0.722 Collecting alembic==1.8.1                                                                                                                                         
#16 0.764   Downloading alembic-1.8.1-py3-none-any.whl (209 kB)
#16 0.831 Collecting beautifulsoup4==4.11.1
#16 0.845   Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)
#16 0.884 Collecting bleach==5.0.1
#16 0.899   Downloading bleach-5.0.1-py3-none-any.whl (160 kB)
#16 0.936 Collecting blinker==1.5
#16 0.948   Downloading blinker-1.5-py2.py3-none-any.whl (12 kB)
#16 0.979 Collecting certifi==2022.9.14
#16 0.993   Downloading certifi-2022.9.14-py3-none-any.whl (162 kB)
#16 1.125 Collecting charset-normalizer==2.1.1
#16 1.137   Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
#16 1.172 Collecting click==8.1.3
#16 1.185   Downloading click-8.1.3-py3-none-any.whl (96 kB)
#16 1.216 Collecting dnspython==2.2.1
#16 1.232   Downloading dnspython-2.2.1-py3-none-any.whl (269 kB)
#16 1.281 Collecting email-validator==1.3.0
#16 1.292   Downloading email_validator-1.3.0-py2.py3-none-any.whl (22 kB)
#16 1.327 Collecting Flask==2.2.2
#16 1.338   Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
#16 1.371 Collecting Flask-Admin==1.6.0
#16 1.389   Downloading Flask-Admin-1.6.0.tar.gz (6.6 MB)
#16 2.042 Collecting Flask-Login==0.6.2
#16 2.053   Downloading Flask_Login-0.6.2-py3-none-any.whl (17 kB)
#16 2.076 Collecting Flask-Mail==0.9.1
#16 2.087   Downloading Flask-Mail-0.9.1.tar.gz (45 kB)
#16 2.209 Collecting Flask-Migrate==3.1.0
#16 2.225   Downloading Flask_Migrate-3.1.0-py3-none-any.whl (20 kB)
#16 2.247 Collecting Flask-PageDown==0.4.0
#16 2.259   Downloading Flask_PageDown-0.4.0-py3-none-any.whl (5.5 kB)
#16 2.282 Collecting Flask-Reuploaded==1.2.0
#16 2.294   Downloading Flask_Reuploaded-1.2.0-py3-none-any.whl (14 kB)
#16 2.324 Collecting Flask-SQLAlchemy==2.5.1
#16 2.336   Downloading Flask_SQLAlchemy-2.5.1-py2.py3-none-any.whl (17 kB)
#16 2.365 Collecting Flask-WTF==1.0.1
#16 2.377   Downloading Flask_WTF-1.0.1-py3-none-any.whl (12 kB)
#16 2.607 Collecting greenlet==1.1.3
#16 2.625   Downloading greenlet-1.1.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (158 kB)
#16 2.657 Collecting html5lib==1.1
#16 2.674   Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB)
#16 2.706 Collecting idna==3.4
#16 2.718   Downloading idna-3.4-py3-none-any.whl (61 kB)
#16 2.775 Collecting importlib-metadata==4.12.0
#16 2.791   Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB)
#16 2.842 Collecting importlib-resources==5.9.0
#16 2.858   Downloading importlib_resources-5.9.0-py3-none-any.whl (33 kB)
#16 2.891 Collecting itsdangerous==2.1.2
#16 2.911   Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
#16 2.941 Collecting Jinja2==3.1.2
#16 2.956   Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
#16 3.273 Collecting lxml==4.9.1
#16 3.295   Downloading lxml-4.9.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl (6.6 MB)
#16 3.540 Collecting Mako==1.2.2
#16 3.553   Downloading Mako-1.2.2-py3-none-any.whl (78 kB)
#16 3.590 Collecting Markdown==3.4.1
#16 3.609   Downloading Markdown-3.4.1-py3-none-any.whl (93 kB)
#16 3.700 Collecting MarkupSafe==2.1.1
#16 3.713   Downloading MarkupSafe-2.1.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26 kB)
#16 4.011 Collecting numpy==1.23.3
#16 4.028   Downloading numpy-1.23.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
#16 4.656 Collecting orcid==1.0.3
#16 4.669   Downloading orcid-1.0.3.tar.gz (10 kB)
#16 5.003 Collecting pandas==1.5.0
#16 5.034   Downloading pandas-1.5.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.5 MB)
#16 5.649 Collecting psycopg2-binary==2.9.3
#16 5.664   Downloading psycopg2-binary-2.9.3.tar.gz (380 kB)
#16 5.841 Collecting python-dateutil==2.8.2
#16 5.857   Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#16 5.931 Collecting pytz==2022.2.1
#16 5.945   Downloading pytz-2022.2.1-py2.py3-none-any.whl (500 kB)
#16 6.013 Collecting rauth==0.7.3
#16 6.026   Downloading rauth-0.7.3.tar.gz (16 kB)
#16 6.160 Collecting requests==2.28.1
#16 6.171   Downloading requests-2.28.1-py3-none-any.whl (62 kB)
#16 6.326 Collecting simplejson==3.17.6
#16 6.340   Downloading simplejson-3.17.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (134 kB)
#16 6.370 Collecting six==1.16.0
#16 6.380   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
#16 6.409 Collecting soupsieve==2.3.2.post1
#16 6.420   Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
#16 6.898 Collecting SQLAlchemy==1.4.41
#16 6.924   Downloading SQLAlchemy-1.4.41-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.6 MB)
#16 7.082 Collecting urllib3==1.26.12
#16 7.095   Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
#16 7.128 Collecting webencodings==0.5.1
#16 7.138   Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
#16 7.181 Collecting Werkzeug==2.2.2
#16 7.196   Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
#16 7.232 Collecting WTForms==3.0.1
#16 7.246   Downloading WTForms-3.0.1-py3-none-any.whl (136 kB)
#16 7.287 Collecting zipp==3.8.1
#16 7.300   Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
#16 7.309 Building wheels for collected packages: Flask-Admin, Flask-Mail, orcid, psycopg2-binary, rauth
#16 7.310   Building wheel for Flask-Admin (setup.py): started
#16 8.539   Building wheel for Flask-Admin (setup.py): finished with status 'done'
#16 8.544   Created wheel for Flask-Admin: filename=Flask_Admin-1.6.0-py3-none-any.whl size=7496260 sha256=5358c120cc7f98b0d09f83ed0237f298ecb3a942e08e1836e503c31b0b0a2e48
#16 8.544   Stored in directory: /var/lib/postgresql/.cache/pip/wheels/29/3b/fd/4efe7f1799ddc495661193f39fc73b86af6bc0b5e69db47f2c
#16 8.545   Building wheel for Flask-Mail (setup.py): started
#16 8.650   Building wheel for Flask-Mail (setup.py): finished with status 'done'
#16 8.650   Created wheel for Flask-Mail: filename=Flask_Mail-0.9.1-py3-none-any.whl size=7568 sha256=e9ce7a225de09ffe5a8416143ccb59b0b76418a6e6f170644741a52cbb97ed46
#16 8.650   Stored in directory: /var/lib/postgresql/.cache/pip/wheels/98/bc/8c/34c329e4d7efeaf7b9886db0c76d0b23170e54de443f688e3c
#16 8.651   Building wheel for orcid (setup.py): started
#16 8.767   Building wheel for orcid (setup.py): finished with status 'done'
#16 8.768   Created wheel for orcid: filename=orcid-1.0.3-py3-none-any.whl size=10440 sha256=750f815846214985e34f2bbf2e258a3d7385b2d1899ddbf60f3592cff1d9dd49
#16 8.768   Stored in directory: /var/lib/postgresql/.cache/pip/wheels/6d/b9/f6/f0090699772788f01e8cc4cc76b69aa9f91f08a76ed2f8fca8
#16 8.769   Building wheel for psycopg2-binary (setup.py): started
#16 8.916   Building wheel for psycopg2-binary (setup.py): finished with status 'error'
#16 8.916   ERROR: Command errored out with exit status 1:
#16 8.916    command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6sw8zm1z
#16 8.916        cwd: /tmp/pip-install-kn4zfkyu/psycopg2-binary/
#16 8.916   Complete output (40 lines):
#16 8.916   running bdist_wheel
#16 8.916   running build
#16 8.916   running build_py
#16 8.916   creating build
#16 8.916   creating build/lib.linux-aarch64-3.8
#16 8.916   creating build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/_ipaddress.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/__init__.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/errorcodes.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/_json.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/_range.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/pool.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/errors.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/extras.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/sql.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/extensions.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   copying lib/tz.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 8.916   warning: build_py: byte-compiling is disabled, skipping.
#16 8.916   
#16 8.916   running build_ext
#16 8.916   building 'psycopg2._psycopg' extension
#16 8.916   creating build/temp.linux-aarch64-3.8
#16 8.916   creating build/temp.linux-aarch64-3.8/psycopg
#16 8.916   aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.9.3 (dt dec pq3 ext lo64) -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=120018 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -I/usr/include/libxml2 -I/usr/include/mit-krb5 -c psycopg/psycopgmodule.c -o build/temp.linux-aarch64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
#16 8.916   In file included from psycopg/psycopgmodule.c:28:
#16 8.916   ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
#16 8.916      36 | #include 
#16 8.916         |          ^~~~~~~~~~~~
#16 8.916   compilation terminated.
#16 8.916   
#16 8.916   It appears you are missing some prerequisite to build the package from source.
#16 8.916   
#16 8.916   You may install a binary package by installing 'psycopg2-binary' from PyPI.
#16 8.916   If you want to install psycopg2 from source, please install the packages
#16 8.916   required for the build and try again.
#16 8.916   
#16 8.916   For further information please check the 'doc/src/install.rst' file (also at
#16 8.916   ).
#16 8.916   
#16 8.916   error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
#16 8.916   ----------------------------------------
#16 8.916   ERROR: Failed building wheel for psycopg2-binary
#16 8.916   Running setup.py clean for psycopg2-binary
#16 8.984   Building wheel for rauth (setup.py): started
#16 9.092   Building wheel for rauth (setup.py): finished with status 'done'
#16 9.092   Created wheel for rauth: filename=rauth-0.7.3-py3-none-any.whl size=16057 sha256=678f378b0e2f024b70260779558199ea9ec40a47e4b3ed760464222b9fa23ac1
#16 9.092   Stored in directory: /var/lib/postgresql/.cache/pip/wheels/12/ba/46/fba7d3f89b3f871b076908875f0b61fa6fba64af01c7b53036
#16 9.093 Successfully built Flask-Admin Flask-Mail orcid rauth
#16 9.093 Failed to build psycopg2-binary
#16 9.201 Installing collected packages: greenlet, SQLAlchemy, zipp, importlib-resources, importlib-metadata, MarkupSafe, Mako, alembic, soupsieve, beautifulsoup4, six, webencodings, bleach, blinker, certifi, charset-normalizer, click, dnspython, idna, email-validator, Jinja2, Werkzeug, itsdangerous, Flask, WTForms, Flask-Admin, Flask-Login, Flask-Mail, Flask-SQLAlchemy, Flask-Migrate, Flask-PageDown, Flask-Reuploaded, Flask-WTF, html5lib, lxml, Markdown, numpy, urllib3, requests, simplejson, orcid, pytz, python-dateutil, pandas, psycopg2-binary, rauth
#16 9.669   WARNING: The script mako-render is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.669   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 9.740   WARNING: The script alembic is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.740   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 9.881   WARNING: The script normalizer is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 9.881   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.04   WARNING: The script email_validator is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.04   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.18   WARNING: The script flask is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.18   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 10.93   WARNING: The script markdown_py is installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 10.93   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 11.96   WARNING: The scripts f2py, f2py3 and f2py3.8 are installed in '/var/lib/postgresql/.local/bin' which is not on PATH.
#16 11.96   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
#16 14.21     Running setup.py install for psycopg2-binary: started
#16 14.36     Running setup.py install for psycopg2-binary: finished with status 'error'
#16 14.36     ERROR: Command errored out with exit status 1:
#16 14.36      command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ty6vci5/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /var/lib/postgresql/.local/include/python3.8/psycopg2-binary
#16 14.36          cwd: /tmp/pip-install-kn4zfkyu/psycopg2-binary/
#16 14.36     Complete output (40 lines):
#16 14.36     running install
#16 14.36     running build
#16 14.36     running build_py
#16 14.36     creating build
#16 14.36     creating build/lib.linux-aarch64-3.8
#16 14.36     creating build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/_ipaddress.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/__init__.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/errorcodes.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/_json.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/_range.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/pool.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/errors.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/extras.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/sql.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/extensions.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     copying lib/tz.py -> build/lib.linux-aarch64-3.8/psycopg2
#16 14.36     warning: build_py: byte-compiling is disabled, skipping.
#16 14.36     
#16 14.36     running build_ext
#16 14.36     building 'psycopg2._psycopg' extension
#16 14.36     creating build/temp.linux-aarch64-3.8
#16 14.36     creating build/temp.linux-aarch64-3.8/psycopg
#16 14.36     aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.9.3 (dt dec pq3 ext lo64) -DPSYCOPG_DEBUG=1 -DPG_VERSION_NUM=120018 -DHAVE_LO64=1 -DPSYCOPG_DEBUG=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -I/usr/include/libxml2 -I/usr/include/mit-krb5 -c psycopg/psycopgmodule.c -o build/temp.linux-aarch64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
#16 14.36     In file included from psycopg/psycopgmodule.c:28:
#16 14.36     ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
#16 14.36        36 | #include 
#16 14.36           |          ^~~~~~~~~~~~
#16 14.36     compilation terminated.
#16 14.36     
#16 14.36     It appears you are missing some prerequisite to build the package from source.
#16 14.36     
#16 14.36     You may install a binary package by installing 'psycopg2-binary' from PyPI.
#16 14.36     If you want to install psycopg2 from source, please install the packages
#16 14.36     required for the build and try again.
#16 14.36     
#16 14.36     For further information please check the 'doc/src/install.rst' file (also at
#16 14.36     ).
#16 14.36     
#16 14.36     error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
#16 14.36     ----------------------------------------
#16 14.50 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"'; __file__='"'"'/tmp/pip-install-kn4zfkyu/psycopg2-binary/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ty6vci5/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /var/lib/postgresql/.local/include/python3.8/psycopg2-binary Check the logs for full command output.
------
executor failed running [/bin/sh -c python3 -m pip install -r requirements.txt]: exit code: 1

machawk1 · 2024-05-22T20:22:38Z

psycopg2-binary installation seems to be an issue. I updated the requirements file to use the latest (v. 2.9.9) of the module and the Dockerfile builds.

Re:#157

cr625 · 2024-05-22T20:23:13Z

The outstanding issue involves the yamz.sql file, which is a backup of the yamz terms exported from the database (as described in the readme). Without that file there is just an empty copy of the flask app that runs but doesn't do anything so we were going to decide what the docker distro would include as terms.

machawk1 · 2024-05-22T20:24:48Z

@cr625 I believe it would be good for the project to supply some sample data just to allow the user to test the base functionality of the app -- even if it's not a dump from the prod db.

machawk1 · 2024-05-22T20:32:53Z

Sourcing the old yamz.sql file is a little problematic. After I build the image:

docker build -t yamz .

...then run the container with port mapping (5000 is a conflict):

docker run -p 5001:5000 yamz

...click browse then a term, I get an error when accessing the term on the web:

output

[2024-05-22 16:28:50,784] ERROR in app: Exception on /term/alternates/user community [GET]
Traceback (most recent call last):
  File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column terms.search_vector does not exist
LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...
                                                             ^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app

response = self.full_dispatch_request()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request

rv = self.dispatch_request()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request

return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)

File "/yamz/app/term/views.py", line 130, in show_alternate_terms

return render_template(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 147, in render_template

return _render(app, template, context)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 130, in _render

rv = template.render(context)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1301, in render

self.environment.handle_exception()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 936, in handle_exception

raise rewrite_traceback_stack(source=source)

File "/yamz/app/term/templates/term/display_terms.jinja", line 1, in top-level template code

{% extends './base.jinja' %}

File "/yamz/app/templates/base.jinja", line 115, in top-level template code

{% block content %}

File "/yamz/app/term/templates/term/display_terms.jinja", line 22, in block 'content'

{% for selected_term in selected_terms  %}

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2900, in iter

return self._iter().iter()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2907, in _iter

result = self.session.execute(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1712, in execute

result = conn._execute_20(statement, params or {}, execution_options)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20

return meth(self, args_10style, kwargs_10style, execution_options)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection

return connection._execute_clauseelement(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement

ret = self._execute_context(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context

self.handle_dbapi_exception(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in handle_dbapi_exception

util.raise(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise

raise exception

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context

self.dialect.do_execute(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute

cursor.execute(statement, parameters)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column terms.search_vector does not exist

LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...

^
[SQL: SELECT terms.class AS terms_class, terms.id AS terms_id, terms.ark_id AS terms_ark_id, terms.shoulder AS terms_shoulder, terms.naan AS terms_naan, terms.owner_id AS terms_owner_id, terms.created AS terms_created, terms.modified AS terms_modified, terms.term_string AS terms_term_string, terms.definition AS terms_definition, terms.definition_html AS terms_definition_html, terms.examples AS terms_examples, terms.examples_html AS terms_examples_html, terms.concept_id AS terms_concept_id, terms.status AS terms_status, terms.search_vector AS terms_search_vector

FROM terms

WHERE terms.term_string = %(term_string_1)s AND terms.status = %(status_1)s]

[parameters: {'term_string_1': 'user community', 'status_1': 'published'}]

(Background on this error at: https://sqlalche.me/e/14/f405)

Exception on /term/alternates/user community [GET]

Traceback (most recent call last):

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context

self.dialect.do_execute(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute

cursor.execute(statement, parameters)

psycopg2.errors.UndefinedColumn: column terms.search_vector does not exist

LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...

^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app

response = self.full_dispatch_request()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request

rv = self.handle_user_exception(e)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request

rv = self.dispatch_request()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request

return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)

File "/yamz/app/term/views.py", line 130, in show_alternate_terms

return render_template(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 147, in render_template

return _render(app, template, context)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/flask/templating.py", line 130, in _render

rv = template.render(context)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 1301, in render

self.environment.handle_exception()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/jinja2/environment.py", line 936, in handle_exception

raise rewrite_traceback_stack(source=source)

File "/yamz/app/term/templates/term/display_terms.jinja", line 1, in top-level template code

{% extends './base.jinja' %}

File "/yamz/app/templates/base.jinja", line 115, in top-level template code

{% block content %}

File "/yamz/app/term/templates/term/display_terms.jinja", line 22, in block 'content'

{% for selected_term in selected_terms  %}

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2900, in iter

return self._iter().iter()

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2907, in _iter

result = self.session.execute(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1712, in execute

result = conn._execute_20(statement, params or {}, execution_options)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20

return meth(self, args_10style, kwargs_10style, execution_options)

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection

return connection._execute_clauseelement(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement

ret = self._execute_context(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context

self.handle_dbapi_exception(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in handle_dbapi_exception

util.raise(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 208, in raise

raise exception

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context

self.dialect.do_execute(

File "/var/lib/postgresql/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute

cursor.execute(statement, parameters)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column terms.search_vector does not exist

LINE 1: ...S terms_concept_id, terms.status AS terms_status, terms.sear...

^
[SQL: SELECT terms.class AS terms_class, terms.id AS terms_id, terms.ark_id AS terms_ark_id, terms.shoulder AS terms_shoulder, terms.naan AS terms_naan, terms.owner_id AS terms_owner_id, terms.created AS terms_created, terms.modified AS terms_modified, terms.term_string AS terms_term_string, terms.definition AS terms_definition, terms.definition_html AS terms_definition_html, terms.examples AS terms_examples, terms.examples_html AS terms_examples_html, terms.concept_id AS terms_concept_id, terms.status AS terms_status, terms.search_vector AS terms_search_vector

FROM terms

WHERE terms.term_string = %(term_string_1)s AND terms.status = %(status_1)s]

[parameters: {'term_string_1': 'user community', 'status_1': 'published'}]

(Background on this error at: https://sqlalche.me/e/14/f405)

172.17.0.1 - - [22/May/2024 16:28:50] "GET /term/alternates/user%20community HTTP/1.1" 500 -

172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/css/site.css HTTP/1.1" 304 -

172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/images/yamz_purple_logo_square_small.png HTTP/1.1" 304 -

172.17.0.1 - - [22/May/2024 16:28:50] "GET /static/images/favicon.ico HTTP/1.1" 304

Perhaps this is attributed to the code expecting some different DB structure than what is provided in the old yamz.sql. A fresh, usable dump of yamz.sql would be good to have for testing. Can this be generated from production and perhaps pared down to test the fundamentals?

cr625 · 2024-05-22T20:48:00Z

Perhaps this is attributed to the code expecting some different DB structure than what is provided in the old yamz.sql. A fresh, usable dump of yamz.sql would be good to have for testing. Can this be generated from production and perhaps pared down to test the fundamentals?

We did change the columns, you're right to add search_vector. We could generate the dump file from a subset. I think the simplest way is to pick a filter and export only those terms. For example, we could choose the NASA helio data as our sample data or something from material science. The other option I think would be to generate a test set but that is time consuming, I'm sure.

machawk1 · 2024-05-22T21:13:09Z

@cr625 Extracting a subset from production (e.g., helio) would be a good start. If it's not to much trouble, please do so.

machawk1 · 2024-06-25T18:57:00Z

@cr625 Can you supply the command you were using on the prod server to obtain the backup?

I do not believe I have credentials for that server. Obtaining a dump that aligns with the current main branch code is a blocker for this GitHub issue.

machawk1 · 2024-06-25T19:31:57Z

I began stripping out a yamz.sql based on the above just to get up and running and have added that to the issue-157 branch. It would be far less laborious and more accurate to the implementation to have a modern dump that aligns with the prod db schema.

machawk1 added Idea A nice to have feature or request. Has not yet been approved to be worked on. New Issue labels May 22, 2024

machawk1 added a commit that referenced this issue May 22, 2024

Update psycopg2-binary to latest for building docker img

d185085

Re:#157

jkunze removed the New Issue label Jun 12, 2024

machawk1 added the BLOCKED label Jun 25, 2024

machawk1 added a commit that referenced this issue Jun 25, 2024

Add some Docker docs for #157

09f1d91

jkunze removed the BLOCKED label Jun 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Docker docs #157

Add Docker docs #157

machawk1 commented May 22, 2024 •

edited

Loading

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024 •

edited

Loading

machawk1 commented May 22, 2024

cr625 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024 •

edited

Loading

cr625 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented Jun 25, 2024

machawk1 commented Jun 25, 2024

Add Docker docs #157

Add Docker docs #157

Comments

machawk1 commented May 22, 2024 • edited Loading

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024 • edited Loading

machawk1 commented May 22, 2024

cr625 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented May 22, 2024 • edited Loading

cr625 commented May 22, 2024

machawk1 commented May 22, 2024

machawk1 commented Jun 25, 2024

machawk1 commented Jun 25, 2024

machawk1 commented May 22, 2024 •

edited

Loading

machawk1 commented May 22, 2024 •

edited

Loading

machawk1 commented May 22, 2024 •

edited

Loading