diff --git a/HowToUse.md b/HowToUse.md new file mode 100644 index 0000000..a63e206 --- /dev/null +++ b/HowToUse.md @@ -0,0 +1,17 @@ +## How to use Lets Map Your Network + +> Honestly, there is no _'correct'_ way of using LMYN. It solely depends on what you want to acheive using it. Having said that, I am jotting down some _'good'_ practices, which will help you to start with and later feel free to explore it as per your requirement. Do provide your inputs with use-cases in comments and I shall update this document. + +1. Create a new project using _Project Management_ module, this will help you to segregate the information about the network. +2. Upload the CMBD (configuration management database) file, if applicable, to build the initial network. This will quickly give you idea about what it is in paper and what is in network. + > Tip: Use _'Stealth'_ mode while uploading the CMDB file, this will discover the surprisingly _**hidden assets**_ in your network +3. Now you can perform multiple network such as traceroute, scanning to build your network and/or validate the 'actual' network against the desired state of network +4. For cloud (AWS) network, you just have to provide the access_key & access_id and select a region. LMYN will build the network automatically +5. Once you build the network, now you can inspect the interesting part of entire network. Few examples: + - All Windows systems with specific IP range + - All Linux systems for a certain VPC + - All intermediate hops to reach google.com + - All local subnet IPs with no definite operating system details + - All internet gateways for a particular region + 6. Now you can use LMYN to monitor the changes in network. Use the refresh button on top right corner. + > All GREY nodes represent the systems which are not live at this point, but were detected live previously diff --git a/InstallationGuidelines.md b/InstallationGuidelines.md new file mode 100644 index 0000000..2fa9b78 --- /dev/null +++ b/InstallationGuidelines.md @@ -0,0 +1,66 @@ +## Installation Guidelines ## + +### Short version ### + +#### Using DockerHub - Linux only #### +1. Install [docker & docker-compose](https://docs.docker.com/install/linux/docker-ce/debian/#install-using-the-repository) + - sudo apt-get update + - sudo apt-get install docker-ce docker-compose +2. wget https://raw.githubusercontent.com/varchashva/LetsMapYourNetwork/master/docker-compose.yml +3. docker-compose up +4. Browse to http://localhost:9999/core and you are set to explore the tool :) + +### Long version ### +#### For Linux User (Note - Below commands have been provided for Debian-based linux. For other versions of Linux change all commands accordingly) #### + +1. Download LMYN from GitHub and extract all to a directory like /opt/LMYN ($LMYN_HOME) +2. Install python: sudo apt-get install python2.7 +3. Install nmap: sudo apt-get install nmap +4. Install rabbitmq-server: sudo apt-get install rabbitmq-server +5. Install Java Development Kit + - Install Oracle JDK 8 (it’s a prerequisite for Neo4j database) + - Run command java -version on terminal + - If output with version details then jump to Neo4j installation or else continue with Java installation + - Run below commands to install Java + - sudo add-apt-repository ppa:webupd8team/java -y + - sudo apt-get update + - sudo apt-get install oracle-java8-installer +6. Go to Neo4j [download](https://neo4j.com/download-center/#releases) section + - Select ‘Community Server’ section and [download Linux version of Neo4j](https://go.neo4j.com/download-thanks.html?edition=community&release=3.3.6&flavour=unix&_ga=2.217214878.946316120.1534600164-1297405808.1534400604) + - Extract the downloaded file with command sudo tar -xzf neo4j-community-3.3.6-unix.tar.gz -C /opt/neo4j + - Change to neo4j directory cd /opt/neo4j + - Run command ./bin/neo4j console to start the neo4j server + - Browse to Neo4j web console (http://localhost:7474) to change the default password from neo4j to Neo4j + - Please refer [Neo4j Installation Guide](https://neo4j.com/docs/operations-manual/current/installation) for any troubleshooting, if required +7. Run command sudo pip install --trusted-host pypi.python.org -r $LMYN/LetsMapYourNetwork/requirements.txt +8. Run command sudo python $LMYN_HOME/LetsMapYourNetwork/manage.py runserver 0.0.0.0:9999 --insecure +9. Open http://localhost:9999/core in browser and explore the tool + +#### For Windows User #### + +1. Download LMYN from GitHub and extract all. It is recommended to extract within Python home directory for e.g. C:\python\LMYN ($LMYN_HOME) +2. Install python + - Go to python [download](https://www.python.org/downloads/release/python-2715) section and click on [Windows x86 MSI installer for 32-bit](https://www.python.org/ftp/python/2.7.15/python-2.7.15.msi) user OR [Windows x86-64 MSI installer](https://www.python.org/ftp/python/2.7.15/python-2.7.15.amd64.msi) for 64-bit user + - Install the downloaded python file with all default settings +3. Download [nmap](https://nmap.org/dist/nmap-7.70-setup.exe) from here and install with all default settings +4. Download [RabbitMQ-Server](https://www.rabbitmq.com/install-windows.html) and install with all default settings +5. Install Microsoft Visual Studio C++ + - 32-bit user install [VC setup](http://download.microsoft.com/download/A/5/4/A54BADB6-9C3F-478D-8657-93B3FC9FE62D/vcsetup.exe) only with all optional product UNCHECKED + - 64-bit users install [Windows SDK and .NET Framework](https://www.microsoft.com/en-us/Download/confirmation.aspx?id=8442) with default settings +6. Install Oracle JDK 8 (it’s a prerequisite for Neo4j database) + - Run command java -version on command prompt + - If output with version details then jump to Neo4j installation or else continue with Java installation + - Go to Oracle [download](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) section and install JDK 8 with all default settings + - Go to JRE installation directory like C:/Program files/java/jre7/bin and create a folder ‘server’ and copy all content of folder ‘client’ to folder ‘server’ +7. Go to Neo4j [download](https://neo4j.com/download-center/#releases) section + - Select ‘Community Server’ section and [download Windows version](https://go.neo4j.com/download-thanks.html?edition=community&release=3.4.6&flavour=winzip&_ga=2.141706682.946316120.1534600164-1297405808.1534400604) + - Right click on downloaded file and click extract all to a directory like C:\neo4j + - Open the command prompt with administrative privileges and change to extracted directory like cd C:\neo4j + - Run command bin\neo4j console to start the neo4j server + - Browse to Neo4j web console (http://localhost:7474) to change the default password from neo4j to Neo4j + - Please refer [Neo4j Installation Guide](https://neo4j.com/docs/operations-manual/current/installation/) for any troubleshooting, if required +8. Open command prompt with Administrative privilegs and browse to Python home directory for e.g. cd C:\python + - Run command python -m pip install --trusted-host pypi.python.org -r $LMYN/LetsMapYourNetwork/requirements.txt +9. Use same command prompt with Administrative privileges and from Python home directory + - Run command python $LMYN_HOME\LetsMapYourNetwork\manage.py runserver 0.0.0.0:9999 --insecure +10. Open http://localhost:9999/core in browser and explore the tool diff --git a/KeyFeatures.md b/KeyFeatures.md new file mode 100644 index 0000000..da4264e --- /dev/null +++ b/KeyFeatures.md @@ -0,0 +1,44 @@ +## Key Features + +1. Project management + - User can create and delete multiple projects to view same network from different perspective and/or to analyze mulitple networks at same time + - Within a single project, result of all learning activities performed will be collated into a single view and thus provides a holistic perspective of network + +2. Bulk load of existing CMDB + - User can upload their existing CMDB file into the LMYN and it will provide the 'delta' with the actual network + - So LMYN will provide the segregation between what's in their CMDB and not in network and vice-versa for user to take actions on rogue system + - It uses different color-code scheme for different type of systems for e.g. systems which are live in network and not presented in CMDB will be shown as RED node + +3. Ability to perform on-demand network activities + - Other than uploading the CMDB file, user can perform below network activities to any project: + - Traceroute to any destination host + - Network scan to any IP and/or range (all well-known format of IP is accepted) + - LMYN will incorporate the result of above actions into same project to build the network + +4. Cloud (AWS) support + - LMYN fetches the topology information such as VPC, Subnets, Peering, Internet Gateway etc. from AWS APIs and represent it in form of graph + - LMYN makes logical segregation of AWS network as "Regions > VPCs > Subnets > Instances" and groups them accordingly + +5. Enumeration + - LMYN performs multiple enumeration probes to identify the operating system and type of device, as and when network is built + - For AWS, LMYN queries the AWS API to fetch the information of instances such as Platform, State, VPC, Subnet etc. + - If enumeration is successful, then LMYN assigns a relevant icon for each node + +6. Ability to analyse 'interesting' network only + - Now, once user builds the network using multiple activities (CMDB upload, ad-hoc network activities, cloud scan, enumeration), then user can filter only 'interesting' network out of the entire database on UI section + - This filtering process can be performed on the basis of actions (for e.g. IP range, destination host) or enumeration details (Linux, Windows, Router, VPC, Subnet, State etc.) + - Filtering process allows to perform 'AND' and 'OR' kind of operation for e.g. 'all IP in range 192.168.1.1/24 and Windows' + - Filtering process gives ability to users to feed all the information in database but at the same time not overwhelming with the information in UI and make a run-time decision on what user wants to see + +7. Continuous monitoring + - Also, LMYN gives ability to monitor any existing network over the period of time + - User can identify, in graph-form, that how their network is changing (which systems are disconnecting and connecting to network) + - LMYN again utilises color-code scheme to segregate the different type of systems in network for e.g. all nodes which are not live will be shown as GREY + +8. Segregation of backend activities and UI + - LMYN segregates functionally of backend activities with UI + - LMYN have implementation of Celery and RabbitMQ; thus, user have a seamless UI irrespective of background activities + - LMYN keeps track of status of all background activities and updates UI periodically + +9. Docker support + - All of these you can have in docker :) diff --git a/README.md b/README.md index 8967483..b38e29b 100644 --- a/README.md +++ b/README.md @@ -1,125 +1,61 @@ -## LET’S MAP [YOUR] NETWORK ## +## LET’S MAP [YOUR] NETWORK (v2.0) [![Black Hat Arsenal](https://github.com/jopohl/badges/blob/master/arsenal/europe/2018.svg)](http://www.toolswatch.org/2018/09/black-hat-arsenal-europe-2018-lineup-announced/) -#### WHAT IT IS #### +### Table of Contents +1. [WHAT IT IS](#what-it-is) +2. [WHY IT IS](#why-it-is) +3. [WHERE TO USE IT](#where-to-use-it) +4. [Presentations](#presentations) +5. [Key Features](KeyFeatures.md) +6. [Installation Guidelines](InstallationGuidelines.md) +7. [HOW TO USE IT](HowToUse.md) +8. [Understand Node](UnderstandNode.md) +9. [Contributer](#contributer) +10. [LMYN In Action](#lmyn-in-action) -It is utmost important for any security engineer to understand their network first before securing it and it becomes a daunting task to have a ‘true’ understanding of a widespread network. In a mid to large level organisation’s network having a network architecture diagram doesn’t provide the complete understanding and manual verification is a nightmare. Hence in order to secure entire network it is important to have a complete picture of all the systems which are connected to your network. +### WHAT IT IS ### -##### BOTTOM LINE - YOU CAN'T SECURE WHAT YOU ARE NOT AWARE OF. ##### +It is utmost important for any security engineer to understand their network first before securing it and it becomes a daunting task to have a ‘true’ understanding of a widespread network. In a mid to large level organisation’s network having a network architecture diagram doesn’t provide the complete understanding and manual verification is a nightmare. Hence in order to secure entire network it is important to have a complete picture of all the systems which are connected to your network, irrespective of their type, function, techology etc. -Let’s Map Your Network (LMYN) aims to provide an easy to use interface to security engineer and network administrator to have their network in graphical form with zero manual error, where a node represents a system and relationship between nodes represent the connection. To achieve this it uses the basic network commands such as traceroute, ping scan etc. +##### BOTTOM LINE - YOU CAN'T SECURE WHAT YOU ARE NOT AWARE OF. ##### -Below are the modules in LMYN to perform the task of mapping +Let’s Map Your Network (LMYN) aims to provide an easy to use interface to security engineer and network administrator to have their network in graphical form with zero manual error, where a node represents a system and relationship between nodes represent the connection. -1. Project Management – Two modules (New Project and Working Project) will allow you to create and switch between the different projects -2. Load CMDB - This module allows an administrator to upload the CMDB file (new line seperated) and map it entirely. Additionaly, it have a Be Intrusive feature which, if selected, will perform enumeration to identify the 'rogue' host that are not present in CMDB. RED color node will depicts the rogue nodes -3. Find Me – This module enumerates the IP of system in which it is running and further network mapping will be done considering the current system as a 'seed' -4. Roam Around – This module identifies all ‘live’ hosts in the same LAN in which seed system is connected -5. Go To – This module displays graph of all hops that packet is traversing through to reach destination (provided by user) from seed system -6. Scan – This module will perform ping scan of any arbitrary IP range provided by user +LMYN does it in two phases: +1. Learning: In this phase LMYN 'learns' the network by performing the network commands and quering the APIs and then builds graph database leveraging the responses. User can perform any of the learning activities at any point of time and LMYN will incorporate the results in existing database. +2. Monitoring: This is a continuos process, where LMYN monitors the 'in-scope' network for any changes, compare it with existing information and update the graph database accordingly. Below technologies have been used in the tool: 1. Django Python 2. Neo4j DB 3. Sigma JS +4. Celery and RabbitMQ -![LMYN IN ACTION](images/lmyn.png) - -#### WHY IT IS #### +### WHY IT IS ### - Visualizing infrastructure network in form of graph makes it more ‘visible’ and it becomes significantly easy to perform the analysis and identify the key areas of concern for a security engineer and network administrator - - Also, Let’s Map Your Network formulates the graph entirely based-on the network actions performed from ‘seed’ system which will be part of the actual network. Hence there is no chance of manual-error in the mapping of network + - Also, Let’s Map Your Network formulates the graph entirely based-on either network actions performed from ‘seed’ system which will be part of the actual network or quering the APIs. Hence there is no chance of manual-error in the mapping of network -#### WHERE TO USE IT #### +### WHERE TO USE IT ### 1. Network Architecture 'Validation' 2. Troubleshooting for network administrator -3. Internal Network VAPT +3. Internal Network vulnerability assessment and penetration testing -### Documentation ### +### Presentations ### - [BlackHatEurope2018 presentation slide](docs/LetsMapYourNetwork_BlackHat.pdf) - [Demo Videos](https://www.youtube.com/channel/UC77eNGlIzjGL0fgx3i6Hcyw) -## INSTALLATION GUIDELINES ## - -### Short version ### - -#### Using DockerHub - Linux only #### -1. Install [docker & docker-compose](https://docs.docker.com/install/linux/docker-ce/debian/#install-using-the-repository) - - sudo apt-get update - - sudo apt-get install docker-ce docker-compose -2. wget https://raw.githubusercontent.com/varchashva/LetsMapYourNetwork/master/docker-compose.yml -3. docker-compose up -4. Browse to http://localhost:9999/core and you are set to explore the tool :) - -### Long version ### -#### For Linux User (Note - Below commands have been provided for Debian-based linux. For other versions of Linux change all commands accordingly) #### - -1. Download LMYN from GitHub and extract all to a directory like /opt/LMYN ($LMYN_HOME) -2. Install python: sudo apt-get install python2.7 -3. Install nmap: sudo apt-get install nmap -4. Install Java Development Kit - - Install Oracle JDK 8 (it’s a prerequisite for Neo4j database) - - Run command java -version on terminal - - If output with version details then jump to Neo4j installation or else continue with Java installation - - Run below commands to install Java - - sudo add-apt-repository ppa:webupd8team/java -y - - sudo apt-get update - - sudo apt-get install oracle-java8-installer -5. Go to Neo4j [download](https://neo4j.com/download-center/#releases) section - - Select ‘Community Server’ section and [download Linux version of Neo4j](https://go.neo4j.com/download-thanks.html?edition=community&release=3.3.6&flavour=unix&_ga=2.217214878.946316120.1534600164-1297405808.1534400604) - - Extract the downloaded file with command sudo tar -xzf neo4j-community-3.3.6-unix.tar.gz -C /opt/neo4j - - Change to neo4j directory cd /opt/neo4j - - Run command ./bin/neo4j console to start the neo4j server - - Browse to Neo4j web console (http://localhost:7474) to change the default password from neo4j to Neo4j - - Please refer [Neo4j Installation Guide](https://neo4j.com/docs/operations-manual/current/installation) for any troubleshooting, if required -6. Run command sudo pip install --trusted-host pypi.python.org -r $LMYN/LetsMapYourNetwork/requirements.txt -7. Run command sudo python $LMYN_HOME/LetsMapYourNetwork/manage.py runserver 0.0.0.0:9999 --insecure -8. Open http://localhost:9999/core in browser and explore the tool - -#### For Windows User #### - -1. Download LMYN from GitHub and extract all. It is recommended to extract within Python home directory for e.g. C:\python\LMYN ($LMYN_HOME) -2. Install python - - Go to python [download](https://www.python.org/downloads/release/python-2715) section and click on [Windows x86 MSI installer for 32-bit](https://www.python.org/ftp/python/2.7.15/python-2.7.15.msi) user OR [Windows x86-64 MSI installer](https://www.python.org/ftp/python/2.7.15/python-2.7.15.amd64.msi) for 64-bit user - - Install the downloaded python file with all default settings -3. Download [nmap](https://nmap.org/dist/nmap-7.70-setup.exe) from here and install with all default settings -4. Install Microsoft Visual Studio C++ - - 32-bit user install [VC setup](http://download.microsoft.com/download/A/5/4/A54BADB6-9C3F-478D-8657-93B3FC9FE62D/vcsetup.exe) only with all optional product UNCHECKED - - 64-bit users install [Windows SDK and .NET Framework](https://www.microsoft.com/en-us/Download/confirmation.aspx?id=8442) with default settings -5. Install Oracle JDK 8 (it’s a prerequisite for Neo4j database) - - Run command java -version on command prompt - - If output with version details then jump to Neo4j installation or else continue with Java installation - - Go to Oracle [download](http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) section and install JDK 8 with all default settings - - Go to JRE installation directory like C:/Program files/java/jre7/bin and create a folder ‘server’ and copy all content of folder ‘client’ to folder ‘server’ -6. Go to Neo4j [download](https://neo4j.com/download-center/#releases) section - - Select ‘Community Server’ section and [download Windows version](https://go.neo4j.com/download-thanks.html?edition=community&release=3.4.6&flavour=winzip&_ga=2.141706682.946316120.1534600164-1297405808.1534400604) - - Right click on downloaded file and click extract all to a directory like C:\neo4j - - Open the command prompt with administrative privileges and change to extracted directory like cd C:\neo4j - - Run command bin\neo4j console to start the neo4j server - - Browse to Neo4j web console (http://localhost:7474) to change the default password from neo4j to Neo4j - - Please refer [Neo4j Installation Guide](https://neo4j.com/docs/operations-manual/current/installation/) for any troubleshooting, if required -7. Open command prompt with Administrative privilegs and browse to Python home directory for e.g. cd C:\python - - Run command python -m pip install --trusted-host pypi.python.org -r $LMYN/LetsMapYourNetwork/requirements.txt -8. Use same command prompt with Administrative privileges and from Python home directory - - Run command python $LMYN_HOME\LetsMapYourNetwork\manage.py runserver 0.0.0.0:9999 --insecure -9. Open http://localhost:9999/core in browser and explore the tool - -### HOW TO USE IT ### -1. First you need to connect your system into the target network (this will provide us the seed system) -2. Now you can create a dedicated project using New Project module or else you may choose to continue with default project -3. You can perform different network action such as - - Load the CMDB file containing IP one per line and you will have a comparative view of your network - - traceroute to different location system to identify the connection path - - Discover all hosts in seed system’s Local LAN - - Discover all hosts in a particular subnet range - - More action you perform, more visibility you get about the network - -#### FUTURE TO-DO #### -1. Continuos Monitoring -2. More interactive actions for node -3. Grouping of systems – like same subnet IPs as one group -4. System fingerprinting -5. Geographical location mapping - -Shoot any suggestion/complain/bug to varchashva \[at] gmail \[dot] com ### Contributer ### [Jyoti Raval](https://www.linkedin.com/in/jyoti-raval-61565157/): (Brutal!) QA + +### LMYN In Action + +#### Local subnet network +![LMYN IN ACTION](images/LMYN_1.gif) + +#### Network with traceroute to mulitple destinations +![LMYN IN ACTION](images/LMYN_2.gif) + +#### Cloud network +![LMYN IN ACTION](images/LMYN_3.gif) + +Shoot any suggestion/complain/bug to varchashva \[at] gmail \[dot] com diff --git a/UnderstandNode.md b/UnderstandNode.md new file mode 100644 index 0000000..eb668f7 --- /dev/null +++ b/UnderstandNode.md @@ -0,0 +1,18 @@ +## Understanding meaning of color and symbol of node + +### Color meaning + +1. ![##3333ff](https://placehold.it/15/3333ff/000000?text=+) `#3333ff` + - This is SEED node +2. ![##3971ac](https://placehold.it/15/3971ac/000000?text=+) `#3971ac` + - Node has external (public) IP +3. ![##20A8D8](https://placehold.it/15/20A8D8/000000?text=+) `#20A8D8` + - Node has internal (private) IP +4. ![##009900](https://placehold.it/15/009900/000000?text=+) `#009900` + - Node present in CMDB file and is live +5. ![##e60000](https://placehold.it/15/e60000/000000?text=+) `#e60000` + - Node present in CMDB file but not live +6. ![##a6a6a6](https://placehold.it/15/a6a6a6/000000?text=+) `#a6a6a6` + - Node is down at this moment +7. ![##e6e6e6](https://placehold.it/15/e6e6e6/000000?text=+) `#e6e6e6` + - Node is either Inline Router or VPC Peer diff --git a/docker-compose.yml b/docker-compose.yml index 555c0ec..9e3c642 100755 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -13,7 +13,7 @@ services: - "7473:7473" network_mode: host lmyn: - image: "pramodrana/letsmapyournetwork:1.0" + image: "pramodrana/letsmapyournetwork:2.0" restart: always ports: - "9999:9999" diff --git a/images/LMYN_1.gif b/images/LMYN_1.gif new file mode 100644 index 0000000..88f31bf Binary files /dev/null and b/images/LMYN_1.gif differ diff --git a/images/LMYN_2.gif b/images/LMYN_2.gif new file mode 100644 index 0000000..64b432c Binary files /dev/null and b/images/LMYN_2.gif differ diff --git a/images/LMYN_3.gif b/images/LMYN_3.gif new file mode 100644 index 0000000..4e4d2d5 Binary files /dev/null and b/images/LMYN_3.gif differ diff --git a/images/lmyn.png b/images/lmyn.png deleted file mode 100644 index 63c123d..0000000 Binary files a/images/lmyn.png and /dev/null differ