Various changes to get the new production host up and running #249
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…monitoring Faster when test system has too much other stuff.
…th-new-deployment
…ver to work Pull is more flexible than push. The production server is usually in a DMZ without access to all the test/staging servers behind firewall. Pull will allow all test/stagings servers to replicate data from the production server. Push was only working if another production server in the DMZ is replicating data from the primary production server. Push mode was not working for the test/stagings servers. After rsync, there are permissions to fix up. We only have geoserver existing permission fix script so far. We will enable the other 2 once their permission fix script is created.
Required for RockyLinux8 but not Ubuntu, weird. On Ubuntu, if can not shift, we have an error.
For `sync-data` to keep staging server up-to-date with production data.
…th-new-deployment
…th-new-deployment
…th-new-deployment
Put assumption comment closer to code for better readability.
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/944/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/772/NOTEBOOK TEST RESULTS |
Need to match the user inside the Jupyter env.
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/945/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/773/NOTEBOOK TEST RESULTS |
…h port_bindings This makes sense since "host" network_mode will already bind all ports bindings to the host already so the specific port bindings is redundant. docker-20.10.17, docker-compose-1.29.2. ``` ERROR: for node-exporter "host" network_mode is incompatible with port_bindings Traceback (most recent call last): File "docker-compose", line 3, in <module> File "compose/cli/main.py", line 81, in main File "compose/cli/main.py", line 203, in perform_command File "compose/metrics/decorator.py", line 18, in wrapper File "compose/cli/main.py", line 1186, in up File "compose/cli/main.py", line 1182, in up File "compose/project.py", line 702, in up File "compose/parallel.py", line 108, in parallel_execute File "compose/parallel.py", line 206, in producer File "compose/project.py", line 688, in do File "compose/service.py", line 564, in execute_convergence_plan File "compose/service.py", line 480, in _execute_convergence_create File "compose/parallel.py", line 108, in parallel_execute File "compose/parallel.py", line 206, in producer File "compose/service.py", line 478, in <lambda> File "compose/service.py", line 457, in create_and_start File "compose/service.py", line 334, in create_container File "compose/service.py", line 941, in _get_container_create_options File "compose/service.py", line 1073, in _get_container_host_config File "docker/api/container.py", line 598, in create_host_config File "docker/types/containers.py", line 339, in __init__ docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings ```
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/946/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/774/NOTEBOOK TEST RESULTS |
tlvu
changed the title
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/947/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/775/NOTEBOOK TEST RESULTS |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/948/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/776/NOTEBOOK TEST RESULTS |
… jupyter user dir ownership
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/949/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/777/NOTEBOOK TEST RESULTS |
matprov
reviewed
Jul 19, 2022
CHANGES.md
Outdated
| - Various changes to get the new production host up and running | ||
|
|
||
| **Non-breaking changes** | ||
| - Boostrap testsuite: only crawl the subset enough to pass canarie-api monitoring: faster when system under test has too much other stuff. |
There was a problem hiding this comment.
Boostrap in here seems more to be a kind of music than a stack deployment concept ;)
There was a problem hiding this comment.
I meant "bootstrap" LOL, typo :D
matprov
reviewed
Jul 19, 2022
| @@ -97,5 +97,6 @@ LATEST_COMPOSE_VERSION="`git ls-remote https://github.com/docker/compose | grep | |||
| # LATEST_COMPOSE_VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r) # need jq :( | |||
| sudo curl -L "https://github.com/docker/compose/releases/download/$LATEST_COMPOSE_VERSION/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose | |||
| sudo chmod +x /usr/local/bin/docker-compose | |||
| sudo curl -L "https://raw.githubusercontent.com/docker/compose/${LATEST_COMPOSE_VERSION}/contrib/completion/bash/docker-compose" -o /etc/bash_completion.d/docker-compose | |||
| # docker-compose v2 removed the completion file, see https://github.com/docker/compose/issues/8550 | |||
| #sudo curl -L "https://raw.githubusercontent.com/docker/compose/${LATEST_COMPOSE_VERSION}/contrib/completion/bash/docker-compose" -o /etc/bash_completion.d/docker-compose | |||
There was a problem hiding this comment.
Is there a value in keeping this comment here?
There was a problem hiding this comment.
I was half-half on this one. That line will work for docker-compose v1. Given issue #246, we could potentially use v1 for a bit longer.
matprov
approved these changes
Jul 19, 2022
github-actions
bot
added
ci/operations
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/950/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-137.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/778/NOTEBOOK TEST RESULTS |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/951/Result : failure BIRDHOUSE_DEPLOY_BRANCH : fix-issues-found-with-new-deployment DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https:// PAVICS-e2e-workflow-tests Pipeline ResultsTests URL :NOTEBOOK TEST RESULTS
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Non-breaking changes
check-autodeploy-repos: to ensure autodeploy will trigger normally.sync-data: to pull data from existing production host to a new production host or to a staging host to emulate the production host.network_mode: hostandportsbinding since host network_mode will already automatically perform port bindingsBreaking changes
Related Issue / Discussion
Fixes #212
@tlogan2000 @moulab88 FYI