Skip to content

Latest commit

 

History

History
236 lines (208 loc) · 8.97 KB

INSTALL.md

File metadata and controls

236 lines (208 loc) · 8.97 KB
Raw

Installation notes for PatrowlHears

Hardware Pre-requisites

PatrowlHears uses PosgreSQL to store data and RabbitMQ to process queues and asynchronous tasks. We recommend using a virtual machine with at least 2vCPU, 8 GB of RAM and 100 GB of storage disk. You can also use a physical machine with similar specifications.

Installation steps

The very easy way: Use Docker

  • Install Docker and Docker-compose
  • Build the docker stack:
git clone https://github.com/Patrowl/PatrowlHears
cd PatrowlHears
docker-compose build
  • [Optional] Build and store Docker images as tar file (.img)
docker save -o patrowlhears-docker-images.img $(docker-compose config | awk '{if ($1 == "image:") print $2;}' ORS=" ")
  • [Optional] Load
docker load < patrowlhears-docker-images.img
  • Run the docker stack:
docker-compose up
  • Open your browser on http://localhost:8383
    • default username: admin
    • default password: Bonjour1!
  • Load initial DB records and latest updates:
docker-compose exec patrowlhears bash -c 'cd backend_app && ./load_init_data.sh'
docker-compose exec patrowlhears bash -c 'cd backend_app && ./import_data_updates.sh'
  • Regularly update DB from public repository with command:
docker-compose exec patrowlhears bash -c 'cd backend_app && ./import_data_updates.sh'
  • [PRO] Regularly update DB from private repository with command:

Ensure env variables PRO_FEEDS_ACCESS_KEY and PRO_FEEDS_SECRET_KEY are set

docker-compose exec patrowlhears bash -c 'cd backend_app && env/bin/python manage.py downloadfeeds -o /tmp'
docker-compose exec patrowlhears bash -c 'cd backend_app && env/bin/python manage.py importfeeds_vulns -d /tmp/PatrowlHearsFeeds/feeds/'
docker-compose exec patrowlhears bash -c 'cd backend_app && env/bin/python manage.py importfeeds_exploits -d /tmp/PatrowlHearsFeeds/feeds/'

The easy way: Use installation script

  • Install Python3 on your server
  • Run the commands:
git clone https://github.com/Patrowl/PatrowlHears
cd PatrowlHears
./install.sh
  • Open your browser on http://localhost:8383
    • default username: admin
    • default password: Bonjour1!
  • Load initial data:
cd backend_app && ./load_init_data.sh
  • Regularly update DB with command:
cd backend_app && ./import_data_updates.sh

The DevOps way: Install and deploy from Ansible playbook

  • Go to the playbook location cd deploy/ansible/playbooks
  • Copy and update the sample file ansible/vars.yml.sample to ansible/vars.yml
  • Run the Ansible playbook:
ansible-playbook patrowlhears.yml -t patrowlhears-install -i myhost,

Note 1: Do not forget to update Ansible vars and default ansible.cfg options. Note 2: Do not forget the comma ',' after the hostname/ip (well-known Ansible inventory trick).

The Nerd way: Install and deploy from sources

The following section contains a step-by-step guide to build PatrowlHears from its sources.

1. Install system pre-requisites

The following software are required to download and run PatrowlHears:

We strongly recommend to use the system packages. To install the requirements and run PatrowlHears from sources, please follow the instructions below depending on your operating system.

1.1. MacOS/X with Brew
brew update
brew install postgres python3 rabbitmq npm memcached
python -m ensurepip
pip install virtualenv
1.2. Ubuntu 16.04/18.04/20.04 LTS with APT
sudo apt update
sudo apt upgrade -y
sudo apt install -y build-essential python3 python3-dev git curl rabbitmq-server postgresql postgresql-client nodejs libpq-dev nginx
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
rm get-pip.py
sudo pip3 install virtualenv
1.3. CentOS/RHEL with YUM
sudo yum install -y git python3 python3-pip python3-virtualenv rabbitmq-server postgresql postgresql-client npm postgresql-devel
2. Download PatrowlHears from GitHub
git clone https://github.com/Patrowl/PatrowlHears
3. Build the frontend application (VueJS) - Optional
cd PatrowlHears/frontend
npm install
npm run build
4. Install python dependencies within the virtualenv
cd ../backend_app
python3 -m virtualenv env
source env/bin/activate
pip install -r requirements.txt

Note 1: If python3 -m virtualenv env does not work, please consider the command virtualenv env but ensure that Python3 is selected.
Note 2: Be careful, next commands MUST be launched within the python3 virtual environment. The prefix (env3) should appear in the command prompt. Ex:

(env) GreenLock@GL02:PatrowlHears$ ls

If you opened another terminal, please ensure you use the virtualenv with the command source env3/bin/activate. If you want to exit the virtual environment, use the command deactivate. If not, do nothing and stop asking.

Note 3: for MacOs users, install pythons modules from requirements.macos.txt file.

5. Create the PostgreSQL database
  • Edit file the var/db/create_user_and_db.sql and update the user and password values (default values are: 'patrowlhears' and 'patrowlhears').

Note: You should consider to set a strong password in Production.

5.1. MacOS
  • Execute the SQL script:
psql < var/db/create_user_and_db.sql
5.2. Ubuntu 16.04/18.04/20.04 LTS and CentOS/RHEL
  • Execute the SQL script:
sudo -u postgres psql < var/db/create_user_and_db.sql

Note: By default, the script create the database 'patrowlhears_db' with the user/role 'patrowlhears'. The default password is 'patrowlhears'. If you change these settings, do not forget to update the backend_app/settings.py configuration file with your updates, or pass it it using environment variables.

6. Configure PatrowlHears (Django backend) application
  • Copy backend_app/settings.py.sample to backend_app/settings.py and update at least following options:
    • Application settings ALLOWED_HOSTS, LOGGING_LEVEL, PROXIES, SECRET_KEY
    • DB settings (service location and credentials): DATABASES,
    • RabbitMQ settings (service location and credentials): BROKER_URL (default values are guest/guest),
    • Email settings (alerting): EMAIL_HOST, EMAIL_HOST_USER, EMAIL_HOST_PASSWORD, EMAIL_HOST_PORT
  • Create the db schema using the Django commands:
python manage.py makemigrations
python manage.py migrate
  • Collect static files (production mode - all static files copied to /staticfiles/ and will be served by NGinx):
python manage.py collectstatic --noinput
  • Create the Django superuser with all privileges, more than Batman but without a cape:
python manage.py shell < var/bin/create_default_admin.py

Note 1: Default login is admin and password is Bonjour1!.
Note 2: You are in charge to renew the password once the application started. Please keep these credentials in a safe place. This account will be used for the first login on the PatrowlHears application.

  • Create the default organization:
python manage.py shell < var/bin/create_default_organization.py
7. Start the Django backend server
7.1 Testing environment
  • Start Supervisord (Celery workers consuming the tasks enqueued RabbitMQ - Yes, that's how asynchronus tasks work here):
supervisord -c var/etc/supervisord.conf

Note: The Supervisor daemon will be listening on port TCP/9002. Update this in the configuration file if you are not agree with that arbitrary choice. Who really cares ?

  • Check every celery workers are in state RUNNING:
supervisorctl -s http://127.0.0.1:9002 status all
  • Then, the Django application (development only):
python manage.py runserver 127.0.0.1:8000
  • or, using Gunicorn (recommended in production):
gunicorn backend_app.wsgi:application -b 127.0.0.1:8000 --access-logfile -
7.2 Production environment (Nginx serving static files)
  • Open the backend_app/settings.py file and set the variable DEBUG=False.
  • Follow the same steps for starting the development environment (see #7.1)
  • Customize the nginx.conf file provided. Then start it:
[sudo] nginx -p .

Note: By default the WEB pages is exposed from port TCP/8383

8. Load initial DB fixtures and latest updates
  • Run following commands (in path backend_app):
./load_init_data.sh
./import_data_updates.sh
Need help ? Stuck somewhere ?

Don't panic! The community could help you as soon as you double-checked your issue and its undoubtedly related to PatrowlHears installation:

Follow us @patrowl_io