Syncing external repository on 2024_08_27

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,6 @@
+## Description
+<!-- Describe your issue or user story in detail -->
+
+## Describe How to Reproduce
+<!-- If this is an issue, provide sufficient context and steps to reproduce the issue,
+     including: what you did, what you expected to happen, and what happened instead. -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,23 @@
+<!--- Provide a general summary of your changes in the Title above -->
+
+## Description
+<!--- Describe your changes in detail. -->
+
+## Motivation and Context
+<!--- Why is this change required? What problem does it solve? -->
+<!--- If it fixes an open issue, please link to the issue here. -->
+Fixes #
+
+## How Has This Been Tested?
+<!--- Please describe in detail how you tested your changes. -->
+<!--- If this PR does not contain a new test case, explain why. -->
+
+## Checklist:
+<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
+<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
+- [] I have added a [Signed-off-by](https://github.com/IBMResilient/resilient-community-apps/blob/master/CONTRIBUTING.md)
+- [] Either no new documentation is required by this change, OR I added new documentation
+- [] Either no new tests are required by this change, OR I added new tests
+- [] I have run pep8 and pylint. I have cleaned up all valid errors and warnings in code I have added or modified. These tools may generate false positives. Don't be worried about ignoring some errors or warnings. The goal is clean, consistent, and readable code.
+
+Signed-off-by:

.gitignore

@@ -0,0 +1,31 @@
+build/
+__pycache__
+.DS_Store
+.gradle
+.settings
+.idea
+.cache
+.pytest_cache
+.coverage
+.tox
+.swp
+.eggs
+.virtualenv*
+.vscode/
+resilient_circuits_lockfile
+*.iml
+*.pem
+*.pyc
+*.log
+*.log.*
+*~
+*.egg-info
+*.tar.gz
+*.pytest_cache
+*.bak
+.python-version
+*/dist
+.env
+app.config
+_build
+!.drivers/**

.helper-scripts/mirror-containers/PrivateContainerRegistry.md

@@ -0,0 +1,136 @@
+# Private Registry setup for IBM Resilient App Host
+## Table of Contents
+  - [About this Documentation](#about-this-documentation)
+  - [Private Registries](#private-registries)
+  - [Mirroring IBM Resilient Containers to Private Registries](#mirroring-ibm-resilient-containers-to-private-registries)
+  - [Configuring App Host for Private Registries](#configuring-app-host-for-private-registries)
+  - [Building Containers for Private Registries](#building-containers-for-private-registries)
+---
+
+## History
+7/20 -  Initial Release
+
+## About this Documentation:
+Many customers will require the use of their own private container repository for App Host.
+One key use includes the building and referencing to their own integrations. Another is to maintain
+an air-gapped environment with IBM Resilient. This document provides guidance on how to setup a private respository.
+This documentation covers how to setup your private registry for use with App Host.
+
+## Private Registries
+Many different solutions are available for both on-premise and cloud-based container repositories.
+Below is a short list of some of these registries:
+
+Cloud-Based Solutions
+* IBM Cloud
+* Quay.io
+* Github
+* Azure
+* Google
+* JFrog
+
+Amazon AWS ECR doesn't appear to work App Host at this time.
+
+On-premise Solutions
+* Quay.io
+* Docker
+* Github
+* JFrog
+* Harbor
+
+## Mirroring IBM Resilient Containers to Private Registries
+Unfortunately, only one registry can be used at a time with App Host. In order to use
+both the IBM AppExchange published Apps and your own custom integrations, it will be 
+Refer to [README](README.pdf) file in this directory for information on
+how to mirror IBM Resilient containers to a private container registry. Unfortunately, 
+only one registry can be used with APP Host at a time, requiring this mirroring process.
+
+## Configuring App Host for Private Registries
+Once your containers are available in your private container registry,
+follow these steps to reconfigure App Host to use this registry: 
+
+* Login to your App Host
+* Run the `configureAppHostRegistry` tool. You may need to run the command as root.
+* Follow the prompts. The registry URL needs to be start with either `http://` or `https://`. 
+* If your registry is public, no additional authentication is required. Private
+registries require authentication user and password credentials. Some registries
+use API tokens and access policies for registry access. Enable `read` permissions for
+registry access. 
+
+### IBM Cloud Container Registry (CR) Setup
+To illustrate the setup of a private registry, the following steps were used to 
+configure IBM Cloud Registry for App Host. Each step could be performed using either the `ibmcloud cr` CLI tool
+or the web-based console. Documentation on all these steps is found [here](https://cloud.ibm.com/docs/Registry).
+* Configure a namespace for your container images. App Host requires `ibmresilient`.
+* Create a service-id associated with your registry
+* Add policies to your service-id for `Reader` permissions
+* Create an API key. The API key will always be named 'iamapikey' but will use the API key secret as its password. 
+
+The following example shows the App Host configuration steps when using IBM Cloud Container Registry. 
+```
+$ sudo configureAppHostRegistry 
+[sudo] password for appadmin: 
+IBM Resilient: Configuration setup for new AppHost registry
+13:13:36.128 [main] DEBUG io.fabric8.kubernetes.client.Config - Trying to configure client from Kubernetes config...
+13:13:36.132 [main] DEBUG io.fabric8.kubernetes.client.Config - Found for Kubernetes config at: [/root/.kube/config].
+Using controllerId: 78ce8a6b-67ff-44b9-b0a7-9200e9d7b2f3 (Ubuntu)
+Enter registry URL: https://us.icr.io
+Is the registry public? (y/n) n
+Enter registry username: iamapikey
+Enter registry password: 
+Reconfiguring AppHost registry...
+13:15:44.586 [main] INFO com.ibm.security.apps.controller.configuration.steps.impl.CreateRegistrySecretStep - Replacing secret controller-registry-secret in namespace 78ce8a6b-67ff-44b9-b0a7-9200e9d7b2f3
+13:15:44.636 [main] INFO com.ibm.security.apps.controller.configuration.steps.impl.CreateRegistryCredentialsStep - Creating registry credentials
+13:15:44.678 [main] INFO com.ibm.security.apps.controller.configuration.steps.impl.RestartSynchronizerStep - Restarting deployment 'deployment-synchronizer' in namespace 78ce8a6b-67ff-44b9-b0a7-9200e9d7b2f3
+Successfully configured
+``` 
+
+## Building Apps for Private Registries
+In order to build apps for a private registry, your development environment will need the following tools:
+* IBM Resilient `resilient-sdk` for App development.
+* Docker or Podman for container creation
+* Optionally the CLI tool provided by your registry provider (ex. `ibmcloud` for IBM Cloud Container Registry).
+
+### App Development
+Refer to the App development guides provided [here]( https://www-03preprod.ibm.com/support/knowledgecenter/SSBRUQ_37.0.0/doc/apps/Introduction.html). There are two steps needed once your App is complete:
+* Running `resilient-sdk package` to build the .zip file used to import your App into Resilient. 
+Information on these procedures are found in the Resilient App Host development guides.
+
+  `Ex. resilient-sdk package -p .`
+* Running `docker build` or `podman build` to tag and push your App container to your private registry.
+
+#### Private Registry Access
+When using the CLI tool provided by your registry provider, first sign in to the registry. Refer to your registry 
+provider's documentation on how to install the CLI tools.
+
+IBM Cloud Container Registry has the following syntax:
+```
+ibmcloud login -a https://cloud.ibm.com -u passcode -p xxxx
+ibmcloud cr login
+```
+
+Other registries require the developer to edit and use the docker or podman tool for authentication such as: 
+```
+docker login -u iamapikey -p <apikey> us.icr.io
+```
+
+Other login mechanism's requires one to edit the tool's configuration file located on the developer's file system such as:
+`~/.docker/config.json`.
+
+Once authenticated, container's can built and pushed to the private registry as following 
+(substitute podman for docker as appropriate):
+
+```
+docker build . -t us.icr.io/ibmresilient/fn_integration:1.0.0
+docker push us.icr.io/ibmresilient/fn_integration:1.0.0
+```
+
+Existing local containers can also be retagged and pushed to your private registry:
+
+```
+$ docker images
+REPOSITORY                     TAG        IMAGE ID            CREATED             SIZE
+ibmresilient/fn_integration    1.0.0      59ccf4795c0b        3 weeks ago         818MB
+
+$ docker tag 59ccf4795c0b us.icr.io/ibmresilient/fn_integration:1.0.0
+$ docker push us.icr.io/ibmresilient/fn_integration:1.0.0
+```

.helper-scripts/mirror-containers/README.md

@@ -0,0 +1,81 @@
+# Repository Mirror Scripts
+## Table of Contents
+  - [About this Package](#about-this-package)
+  - [mirror-all-images.sh](#mirror-all-imagessh)
+  - [mirror-images.sh](#mirror-imagessh)
+  - [App Host Configuration](#app-host-configuration)
+---
+
+## History
+7/20 -  Initial Documentation
+
+## About this Package:
+This branch contains scripts which can be used to mirror App Host container images
+for local registry use. This is necessary when customers need to maintain their own
+apps for use in App Host along with those officially published on the App Exchange
+or to replicate containers for air-gapped environments.
+
+Two scripts are provided:
+* [mirror-all-images.sh](#mirror-all-images.sh)
+* [mirror-images.sh](#mirror-images.sh)
+
+### Initialization
+* Ensure that the local container tool, docker or podman, is accessible. If root permissions
+are needed to run these commands, use `sudo` or `su` to access that login account in advance.
+* Login to each registry using the `docker login` orr `podman login` commands.
+* Add insecure_registry as an argument if working with HTTP registry with podman, docker may require you edit /.docker/config.json or an environment variable
+* Add latest_tag to only retrieve the most the most recent version of each app, instead of all the unique versions, that exist on quay.io
+
+## mirror-all-images.sh
+This script is used to copy all container images from the IBM official registry, quay.io,
+to a customer's private registry. It uses local container tools such as `docker` or `podman` to
+pull containers down from quay.io and then push them to the private registry.
+
+### Usage
+/bin/bash mirror-all-images fqdn.registry.io [docker | podman]
+
+* fqdn.registry.io - name or ip address of the replication registry
+* [docker | podman] - optional reference to the tool to perform the container transfer.
+If missing, the script will attempt to determine which tool exists.
+
+### Examples
+/bin/bash mirror-all-images 834299573936.dkr.ecr.us-east-2.amazonaws.com podman
+
+/bin/bash mirror-all-images localhost:5000 podman insecure_registry latest_tag
+
+## mirror-images.sh
+This script is used to copy a select number of container images from the IBM official registry, quay.io,
+to a customer's private registry. It uses local container tools such as `docker` or `podman` to
+pull containers down from quay.io and then push them to the private registry.
+Two additional files are used to control the behavior of the script:
+* repo_quay.conf - a list of container names and versions, one per line, to replicate. The format of a line is:
+   `container_name:x.x.x` where `x.x.x` is the tagged version.
+* preserved_images.conf - a list of container names and versions, one per line, to retain in the local docker or podman image
+container environment. Format of a line is the same as used in the `repo_quay.conf` file: `container_name:x.x.x`.
+
+### Usage
+/bin/bash mirror-all-images fqdn.registry.io [docker | podman]
+
+* fqdn.registry.io - name or ip address of the replication registry
+* [docker | podman] - optional reference to the tool to perform the container transfer.
+If missing, the script will attempt to determine which tool exists.
+
+### Example
+/bin/bash mirror-images 192.168.12.186:5000
+
+repo_quay.conf
+```
+fn_utilities:1.14.0
+fn_xforce:1.0.0
+```
+
+preserved_images.conf
+```
+fn_utilities:1.14.0
+```
+
+## App Host Configuration
+Refer to the [Configuring a private repository](https://www-03preprod.ibm.com/support/knowledgecenter/SSBRUQ_37.0.0/doc/apps/private_repo_config.html) document on how to
+develop Apps and configure App Host for your private registry.
+Also, it will be necessary to secure your local registry currently for it to be paired correctly with App Host e.g.
+https://www.redhat.com/sysadmin/simple-container-registry