Skip to content

Commit

Permalink
A lot of changes for containerized development
Browse files Browse the repository at this point in the history
  • Loading branch information
monotasker committed Mar 29, 2024
1 parent 13e5eee commit 8bb8ef3
Show file tree
Hide file tree
Showing 47 changed files with 5,394 additions and 57 deletions.
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ __pycache__/
# Node module files
node_modules/

# untracked assets files (e.g. compiled assets)
static/admin/
static/bootstrap/
static/css/
static/dist/
static/js/
static/templates/
static/vendor/
static/xsl/

# Distribution / packaging
.Python
env/
Expand Down
75 changes: 66 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,27 +17,84 @@ From there installation involves these steps and commands:
1. `git clone [email protected]:MESH-Research/knowledge-commons-repository.git`
2. `cd knowledge-commons-repository`
3. create and configure the .env file in this folder
4. `pip install invenio-cli`
5. `invenio-cli install`
4. `pip install invenio-cli` (optional?)
5. `invenio-cli install` (optional?)
6. start docker if it's not already running
6. `docker-compose up -d`
7. `invenio-cli services setup`
7. `invenio-cli containers start --setup` (`invenio-cli services setup` for local dev install)
- or `docker exec -it knowledge-commons-repository-web-ui-1 bash` and then the following commands
- `invenio db init create`
- `invenio files location create --default default-location ${INVENIO_INSTANCE_PATH}/data`
- `invenio roles create admin`
- `invenio access allow superuser-access role admin`
- `invenio index init`
- `invenio rdm-records custom-fields create`
- `invenio communities custom-fields create`
- `invenio rdm-records fixtures`
- `invenio rdm fixtures`
- `pybabel compile --directory={project_path / translation_folder}`
- `invenio queues declare`
8. `bash kcr-startup.sh`

You can then create an admin user. From the command line, run
You can then create an admin user. From the command line inside the `web-ui` container, run
```console
pipenv run invenio users create <email> --password <password>
pipenv run invenio users activate <email>
pipenv run invenio access allow administration-access user <email>
```

The application instance and its services can be started and stopped using single commands:
```console
bash kcr-startup.sh
The application instance and its services can be started and stopped by starting and stopping the docker-compose project:

```shell
docker-compose --file docker-compose.dev.yml up -d
```
```console
bash kcr-shutdown.sh
```shell
docker-compose --file docker-compose.dev.yml stop
```

[!WARNING]
Do not use the `docker-compose down` command unless you want the containers to be destroyed. This will destroy all data in your database and all OpenSearch indexes. YOU DO NOT WANT TO DO THIS!

### Updating the instance with changes

#### Changes to html template files

Changes to html template files will be visible immediately in the running Knowledge Commons Repository instance. You simply need to refresh the page.

#### Changes to invenio.cfg

Changes to the invenio.cfg file will only take effect after the instance uwsgi processes are restarted. This can be done by running the following command inside the `web-ui` container:
```shell
uwsgi --reload /tmp/kcr_ui.pid
```

#### Changes to theme (CSS) and javascript files

#### Changes to static files

Changes to static files like images will require running the collect command to copy them to the central static folder. This can be done by running the following command inside the `web-ui` container:
```shell
invenio collect -v
```
You will then need to restart the uwsgi processes as described above.

#### Changes to python code in the `site` folder

#### Changes to external python modules (including Invenio modules)

Changes to other python modules (including Invenio modules) will require rebuilding the main knowledge-commons-works container. Additions to the python requirements should be added to the `Pipfile` in the knowledge-commons-works folder and committed to the Github repository. You should then request that the knowledge-commons-works container be rebuilt.

In the meantime, required python packages can be installed directly in the `web-ui` container. First, enter the container:
```shell
docker exec -it knowledge-commons-repository-web-ui-1 bash
```
Then install the required package using pipenv:
```shell
pipenv install <package-name>
```

### Digging deeper

What follows is a step-by-step walk through this process. Note that these instructions do not support installation under Windows. Windows users should emulate a Linux environment using WSL2.

## Install Python and Required Python Tools
Expand Down
43 changes: 33 additions & 10 deletions assets/less/site/globals/site.overrides
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ textarea {
}
}
}
.mobile.hidden {
@media all and (max-width: @largestMobileScreen) {
display: none !important;
}
}

// FIXME: hidden for now

Expand Down Expand Up @@ -96,10 +101,30 @@ textarea {
box-shadow: none;
}
}
.rdm-logo {
width: auto;
@media screen and (max-width: @largestMobileScreen) {
max-height: 3rem;
.item.logo {
.logo-link {
display: flex;
align-items: center;
.rdm-logo {
width: auto;
@media screen and (max-width: @largestMobileScreen) {
max-height: 3rem;
}
}
.title-wrapper {
h1 {
margin-top: 0;
margin-left: 1rem;
text-transform: uppercase;
letter-spacing: 0.5rem;
}
.subtitle {
font-size: 0.75rem;
font-weight: bold;
text-transform: uppercase;
letter-spacing: 0.25rem;
}
}
}
}
}
Expand Down Expand Up @@ -1439,26 +1464,25 @@ textarea {
color: @textColor;
padding-left: 0.75rem;
}

}
}

.form.communities-profile {
&> .ui.grid > .row:first-child {
& > .ui.grid > .row:first-child {
padding-top: 0;
}
.accordion.ui.inverted.invenio-accordion-field {
&:nth-child(2) {
.content.active {
.content.active {
.ui.container {
margin-left: 0rem !important;
margin-right: 0rem !important;
}
}
}
}

&:nth-child(3) {
.content.active {
.content.active {
.ui.container {
margin-left: 1rem !important;
margin-right: 1rem !important;
Expand All @@ -1474,7 +1498,6 @@ textarea {
}
}
}

}

.logo-upload-column {
Expand Down
26 changes: 15 additions & 11 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#
# Usage::
#
# $ docker-compose up --file docker-compose.full.yml -d
# $ docker-compose up --file docker-compose.dev.yml -d
#
# Following services are included:
# - Frontend: Nginx (exposed ports: 80 and 443)
Expand Down Expand Up @@ -63,8 +63,8 @@ services:
volumes:
- ./docker/nginx_containerized/nginx.conf:/etc/nginx/nginx.conf
- ./docker/nginx_containerized/conf.d:/etc/nginx/conf.d
# - ./static:/opt/invenio/var/instance/static
- static_data:/opt/invenio/var/instance/static
- ./static:/opt/invenio/var/instance/static
# - static_data:/opt/invenio/var/instance/static
depends_on:
- web-ui
- web-api
Expand All @@ -83,6 +83,7 @@ services:
command: ["uwsgi /opt/invenio/var/instance/uwsgi_ui.ini"]
image: monotasker/knowledge-commons-repository:latest
environment:
- FLASK_ENV=development
- INVENIO_INSTANCE_PATH=/opt/invenio/var/instance
- INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL=redis://cache:6379/4
ports:
Expand All @@ -91,12 +92,13 @@ services:
- "host.docker.internal:host-gateway"
volumes:
- ./invenio.cfg:/opt/invenio/var/instance/invenio.cfg
- ${INVENIO_LOCAL_INSTANCE_PATH}/app_data:/opt/invenio/var/instance/app_data
- ${INVENIO_LOCAL_INSTANCE_PATH}/assets:/opt/invenio/var/instance/assets
- ${INVENIO_LOCAL_INSTANCE_PATH}/templates:/opt/invenio/var/instance/templates
- ./app_data:/opt/invenio/var/instance/app_data
# - ${INVENIO_LOCAL_INSTANCE_PATH}/assets:/opt/invenio/var/instance/assets
- ./static:/opt/invenio/var/instance/static
- ./templates:/opt/invenio/var/instance/templates
- ./translations:/opt/invenio/var/instance/translations
- .:/opt/invenio/src
- static_data:/opt/invenio/var/instance/static
# - static_data:/opt/invenio/var/instance/static
- archived_data:/opt/invenio/var/instance/archive
- uploaded_data:/opt/invenio/var/instance/data
- /${MIGRATION_SERVER_LOCAL_DATA_DIR}:${MIGRATION_SERVER_DATA_DIR}
Expand All @@ -123,17 +125,19 @@ services:
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- FLASK_ENV=development
- INVENIO_INSTANCE_PATH=/opt/invenio/var/instance
- INVENIO_COMMUNITIES_IDENTITIES_CACHE_REDIS_URL=redis://cache:6379/4
volumes:
# TODO: in production copy these files rather than mounting?
- ./invenio.cfg:/opt/invenio/var/instance/invenio.cfg
- ${INVENIO_LOCAL_INSTANCE_PATH}/app_data:/opt/invenio/var/instance/app_data
- ${INVENIO_LOCAL_INSTANCE_PATH}/assets:/opt/invenio/var/instance/assets
- ${INVENIO_LOCAL_INSTANCE_PATH}/static:/opt/invenio/var/instance/static
- ${INVENIO_LOCAL_INSTANCE_PATH}/templates:/opt/invenio/var/instance/templates
- ./app_data:/opt/invenio/var/instance/app_data
# - ${INVENIO_LOCAL_INSTANCE_PATH}/assets:/opt/invenio/var/instance/assets
- ./static:/opt/invenio/var/instance/static
- ./templates:/opt/invenio/var/instance/templates
- ./translations:/opt/invenio/var/instance/translations
- .:/opt/invenio/src
# - static_data:/opt/invenio/var/instance/static
- archived_data:/opt/invenio/var/instance/archive
- uploaded_data:/opt/invenio/var/instance/data
- /${MIGRATION_SERVER_LOCAL_DATA_DIR}:${MIGRATION_SERVER_DATA_DIR}
Expand Down
10 changes: 5 additions & 5 deletions docker-services.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
version: '2.2'
services:
app:
# build:
# context: ./
# args:
# - ENVIRONMENT=DEV
build:
context: ./
args:
- ENVIRONMENT=DEV
image: monotasker/knowledge-commons-repository:latest
restart: "unless-stopped"
environment:
Expand All @@ -21,7 +21,7 @@ services:
- "INVENIO_CELERY_BROKER_URL=amqp://guest:guest@mq:5672/"
- "INVENIO_CELERY_RESULT_BACKEND=redis://cache:6379/2"
- "INVENIO_SEARCH_HOSTS=['search:9200']"
- "INVENIO_SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://knowledge-commons-repository:knowledge-commons-repository@db/knowledge-commons-repository"
- "INVENIO_SQLALCHEMY_DATABASE_URI=${INVENIO_SQLALCHEMY_DATABASE_URI}"
- "INVENIO_WSGI_PROXIES=2"
- "INVENIO_RATELIMIT_STORAGE_URI=redis://cache:6379/3"
- "INVENIO_RATELIMIT_STORAGE_URL=redis://cache:6379/3"
Expand Down
1 change: 1 addition & 0 deletions docker/uwsgi/uwsgi_rest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ manage-script-name = true
wsgi-disable-file-wrapper = true
single-interpreter = true
buffer-size = 32768
pidfile = /tmp/uwsgi_rest.pid
3 changes: 2 additions & 1 deletion docker/uwsgi/uwsgi_ui.ini
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,5 @@ processes = 2
threads = 2
single-interpreter = true
buffer-size = 32768
wsgi-disable-file-wrapper = true
wsgi-disable-file-wrapper = true
pidfile = /tmp/uwsgi_ui.pid
64 changes: 48 additions & 16 deletions invenio.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -375,10 +375,10 @@ I18N_LANGUAGES = [
# See https://invenio-theme.readthedocs.io/en/latest/configuration.html

# Frontpage title
THEME_FRONTPAGE_TITLE = "Knowledge Commons Repository"
THEME_FRONTPAGE_TITLE = "Knowledge Commons Works"
# Header logo
# TODO: change
THEME_LOGO = "images/Network_Black-edited-line-with-color.jpg"
THEME_LOGO = "images/kc_logo.png"


# Invenio-App-RDM
Expand All @@ -387,23 +387,55 @@ THEME_LOGO = "images/Network_Black-edited-line-with-color.jpg"

# Instance's theme entrypoint file. Path relative to the ``assets/`` folder.
INSTANCE_THEME_FILE = "./less/theme.less"
# APP_RDM_RECORD_LANDING_PAGE_TEMPLATE = (
# "knowledge_commons_repository/records/detail.html"
# )
APP_RDM_RECORD_LANDING_PAGE_TEMPLATE = (
"invenio_modular_detail_page/detail.html"
)
# APP_RDM_DETAIL_SIDE_BAR_TEMPLATES = [
# "invenio_app_rdm/records/details/side_bar/manage_menu.html",
# "invenio_app_rdm/records/details/side_bar/metrics.html",
# "invenio_app_rdm/records/details/side_bar/versions.html",
# "invenio_app_rdm/records/details/side_bar/external_resources.html",
# "invenio_app_rdm/records/details/side_bar/communities.html",
# "invenio_app_rdm/records/details/side_bar/keywords_subjects.html",
# "invenio_app_rdm/records/details/side_bar/details.html",
# "invenio_app_rdm/records/details/side_bar/licenses.html",
# "invenio_app_rdm/records/details/side_bar/export.html",
# ]
KC_WORDPRESS_DOMAIN = "hcommons.org"

KC_FOOTER_LINKS = [
(
"_blank",
[
[_("Help & FAQ"), f"https://support.{KC_WORDPRESS_DOMAIN}/"],
[
_("Terms of use"),
f"https://sustaining.{KC_WORDPRESS_DOMAIN}/policies/terms/",
],
[_("Privacy policy"), f"https://{KC_WORDPRESS_DOMAIN}/privacy/"],
[
_("Guidelines for use"),
f"https://{KC_WORDPRESS_DOMAIN}/guidelines/",
],
],
("eight", "four", "four"),
),
(
"_blank",
[
[_("Knowledge Commons"), f"https://{KC_WORDPRESS_DOMAIN}/"],
[_("Mesh Research"), "https://meshresearch.net"],
],
("seven", "six", "six"),
),
(
"Major Supporters",
[
[_("Michigan State University"), "https://msu.edu", "MSU.svg"],
[_("National Science Foundation"), "https://nsf.gov", "USNSF.png"],
[
_("National Endowment for the Humanities"),
"https://neh.gov",
"NEH.png",
],
["", ""],
[
_("and others listed on our Supporters page"),
f"https://sustaining.{KC_WORDPRESS_DOMAIN}/financial/supporters/",
],
],
("sixteen", "six", "six"),
),
]

# top-level objects are sections in sidebar
APP_RDM_DETAIL_SIDEBAR_SECTIONS_RIGHT = [
Expand Down

This file was deleted.

Empty file.
Loading

0 comments on commit 8bb8ef3

Please sign in to comment.