Remove heavy ipynb notebooks from main branch (#147)

* chore: Update publish_benchmarks.sh script to create benchmark directory before copying files

Remove heavy notebooks

chore: Remove trailing-whitespace hook and update check-yaml hook configuration

* chore: Update notebook_benchmark parameters for different environments

Adjust the parameters for the notebook_benchmark script to run benchmarks on different environments. This includes changing the gymid, training_episode_count, eval_episode_count, maximum_total_credentials, and plots_dir values for each environment.

Co-authored-by: William Blum <[email protected]>

---------

Co-authored-by: William Blum <[email protected]>

.github/workflows/ci.yml

@@ -73,13 +73,13 @@ jobs:
           pip install pytest-azurepipelines
           python -m pytest -v cyberbattle
 
-      - name: Papermill notebooks
+      - name: Papermill all notebooks in quick mode
         shell: bash -el {0}
         run: |
           conda activate cybersim
           jupyter kernelspec list
           cd notebooks
-          bash run_all.sh python3
+          bash run_quick.sh python3
         if: github.ref_name == 'main' || startsWith(github.ref, 'refs/heads/wiblum')
 
       - name: Archive milled notebooks

.gitignore

@@ -55,8 +55,6 @@ notebooks/untracked/**
 cyberbattle.code-workspace
 typings/**
 
-cyberbattle/agents/baseline/notebooks/images/*.png
-cyberbattle/agents/baseline/notebooks/images/*.gif
 log.txt
 
 notebooks/output/**

.pre-commit-config.yaml

@@ -24,7 +24,5 @@ repos:
       rev: v2.4.0
       hooks:
           - id: trailing-whitespace
-          - id: end-of-file-fixer
-            exclude: '.*\.ipynb$'
           - id: check-yaml
           - id: check-added-large-files

README.md

@@ -206,44 +206,41 @@ Cumulative rewards -- DQN=Red, Random=Green
 
 ## Jupyter notebooks
 
-To quickly get familiar with the project, you can open one of the provided Jupyter notebooks to play interactively with
-the gym environments. At the root of the repository run the following command and then open the notebook in the `notebooks` folder
-from the Jupyter interface:
+To quickly get familiar with the project, you can open one of the provided Jupyter notebooks to play interactively with the gymnasium environments.
+
+> Notes on the `.py` notebooks:
+> - Our notebooks are checked-in in Git as `.py` files. Those can be opened and run directly  in VSCode or in Jupyter using the [Jupytext extension](https://jupytext.readthedocs.io/en/latest/install.html).
+> - The output `.ipynb` files can also be automatically regenerated using [papermill](https://pypi.org/project/papermill/) by running the bash script [run_benchmark.sh](/notebooks/run_benchmark.sh).
+> - We also publish a snapshot of the corresponding `.ipynb`-notebooks with the entire output and plots in a separate git tag.
+The latest snapshot of the Jupyter notebooks output, including the benchmarks, are
+accessible from the following git tag: [/notebooks/benchmarks (latest_benchmark)](https://github.com/microsoft/CyberBattleSim/tree/latest_benchmark/notebooks/benchmarks).
 
-```python
-export PYTHONPATH=$(pwd)
-jupyter lab
-```
 
 Some notebooks to get started:
 
 - 'Capture The Flag' toy environment notebooks:
-  - [Random agent](notebooks/toyctf-random.ipynb)
-  - [Interactive session for a human player](notebooks/toyctf-blank.ipynb)
-  - [Interactive session - fully solved](notebooks/toyctf-solved.ipynb)
+  - [Random agent](notebooks/toyctf-random.py)
+  - [Interactive session for a human player](notebooks/toyctf-blank.py)
+  - [Interactive session - fully solved](notebooks/toyctf-solved.py)
 
 - Chain environment notebooks:
-  - [Random agent](notebooks/chainnetwork-random.ipynb)
+  - [Random agent](notebooks/chainnetwork-random.py)
 
 - Other environments:
-  - [Interactive session with a randomly generated environment](notebooks/randomnetwork.ipynb)
-  - [Random agent playing on randomly generated networks](notebooks/c2_interactive_interface.ipynb)
+  - [Interactive session with a randomly generated environment](notebooks/randomnetwork.py)
+  - [Random agent playing on randomly generated networks](notebooks/c2_interactive_interface.py)
 
-- Benchmarks:
+- Benchmarks:   The following notebooks show benchmark evaluation of the baseline agents on various environments.
 
-  The following notebooks show benchmark evaluation of the baseline agents on various environments.
+    - [Benchmarking on a given environment](notebooks/notebook_benchmark.py)
+    - [Benchmark on chain environments with a basic defender](notebooks/notebook_withdefender.py)
+    - [DQL transfer learning evaluation](notebooks/notebook_dql_transfer.py)
+    - [Epsilon greedy with credential lookups](notebooks/notebook_randlookups.py)
+    - [Tabular Q Learning](notebooks/notebook_tabularq.py)
 
-  > The source `.py`-versions of the notebooks are best viewed in VSCode or in Jupyter with the [Jupytext extension](https://jupytext.readthedocs.io/en/latest/install.html).
-  The `notebooks` folder contains the corresponding `.ipynb`-notebooks
-  with the entire output and plots. These can be regenerated via [papermill](https://pypi.org/project/papermill/) using this [bash script](cyberbattle/agents/baseline/notebooks/runall.sh)
-  .
+The latest snapshot of the Jupyter notebooks output, including the benchmarks, are
+accessible from the following git tag: [/notebooks/benchmarks (latest_benchmark)](https://github.com/microsoft/CyberBattleSim/tree/latest_benchmark/notebooks/benchmarks).
 
-    - Benchmarking on a given environment: [source](cyberbattle/agents/baseline/notebooks/notebook_benchmark.py): [output (Chain)](notebooks/notebook_benchmark-chain.ipynb), [output (Capture the flag)](notebooks/notebook_benchmark-toyctf.ipynb)
-    - Benchmark on chain environments with a basic defender: [source](cyberbattle/agents/baseline/notebooks/notebook_withdefender.py),
-    [output](notebooks/notebook_withdefender.ipynb);
-    - DQL transfer learning evaluation: [source](cyberbattle/agents/baseline/notebooks/notebook_dql_transfer.py), [output](notebooks/notebook_dql_transfer.ipynb);
-    - Epsilon greedy with credential lookups: [source](cyberbattle/agents/baseline/notebooks/notebook_randlookups.py), [output](notebooks/notebook_randlookups.ipynb);
-    - Tabular Q Learning: [source](cyberbattle/agents/baseline/notebooks/notebook_tabularq.py); [output](notebooks/notebook_tabularq.ipynb)
 
 ## How to instantiate the Gym environments?
 

cyberbattle/agents/baseline/baseline_test.py

@@ -24,6 +24,7 @@
 
 ep = w.EnvironmentBounds.of_identifiers(maximum_total_credentials=10, maximum_node_count=10, identifiers=cyberbattlechain.identifiers)
 
+# %% {"tags": ["parameters"]}
 training_episode_count = 2
 iteration_count = 5
 

notebooks/c2_interactive_interface.py

@@ -0,0 +1,85 @@
+# ---
+# jupyter:
+#   jupytext:
+#     formats: ipynb,py:percent
+#     text_representation:
+#       extension: .py
+#       format_name: percent
+#       format_version: '1.3'
+#       jupytext_version: 1.16.4
+#   kernelspec:
+#     display_name: cybersim
+#     language: python
+#     name: cybersim
+# ---
+
+# %% [markdown]
+# pyright: reportUnusedExpression=false
+
+# %% [markdown]
+# Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License.
+#
+# # Command and Control interface
+# This notebooks shows how to interact with the command&control server to observe the environment and initiate actions on the nodes where the attacker client is installed.
+
+# %%
+import networkx as nx
+from tabulate import tabulate
+import cyberbattle.simulation.model as model
+import cyberbattle.simulation.actions as actions
+import cyberbattle.simulation.commandcontrol as commandcontrol
+import importlib
+
+importlib.reload(model)
+importlib.reload(actions)
+importlib.reload(commandcontrol)
+import plotly.offline as plo
+
+plo.init_notebook_mode(connected=True)  # type: ignore
+# %matplotlib inline
+
+# %% [markdown]
+# We first create a simulation environment from a randomly generated network graph.
+
+# %%
+g = nx.erdos_renyi_graph(35, 0.05, directed=True)
+g = model.assign_random_labels(g)
+env = model.Environment(network=g, vulnerability_library=dict([]), identifiers=model.SAMPLE_IDENTIFIERS)
+
+# %% [markdown]
+# We create the `CommandControl` object used to the environment and execute actions, and plot the graph explored so far.
+#
+
+# %%
+c = commandcontrol.CommandControl(env)
+
+# %%
+c.plot_nodes()
+print("Nodes disovered so far: " + str(c.list_nodes()))
+starting_node = c.list_nodes()[0]["id"]
+
+# %% [markdown]
+# For debugging purpose it's also convient to view the internals of the environment via the `EnvironmentDebugging` object. For instance we can use it to plot the entire graph, including nodes that were not discovered yet by the attacker.
+
+# %%
+dbg = commandcontrol.EnvironmentDebugging(c)
+
+# %%
+env.plot_environment_graph()
+print(nx.info(env.network))  # type: ignore
+
+# %%
+print(tabulate(c.list_all_attacks(), {}))
+
+# %%
+outcome = c.run_attack(starting_node, "RecentlyAccessedMachines")
+outcome
+
+# %%
+c.plot_nodes()
+
+# %%
+print(tabulate(c.list_nodes(), {}))
+
+# %%
+print(tabulate(c.list_all_attacks(), {}))

notebooks/chainnetwork-optionwrapper.py

@@ -0,0 +1,59 @@
+# ---
+# jupyter:
+#   jupytext:
+#     formats: py:percent,ipynb
+#     text_representation:
+#       extension: .py
+#       format_name: percent
+#       format_version: '1.3'
+#       jupytext_version: 1.16.4
+#   kernelspec:
+#     display_name: cybersim
+#     language: python
+#     name: cybersim
+# ---
+
+# %% [markdown] magic_args="[markdown]"
+# Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License.
+
+# %%
+# %%
+import plotly.io.orca
+import gymnasium as gym
+import numpy
+from typing import cast
+from cyberbattle._env.cyberbattle_env import CyberBattleEnv
+
+# %%
+# %matplotlib inline
+plotly.io.orca.config.executable = "~/.npm-packages/bin/orca"  # type: ignore
+
+# %%
+from cyberbattle._env.cyberbattle_env import AttackerGoal
+from cyberbattle._env.option_wrapper import ContextWrapper, random_options
+
+# %%
+rnd = numpy.random.RandomState(13)
+
+# %%
+env = cast(CyberBattleEnv, gym.make("CyberBattleChain-v0", size=10, attacker_goal=AttackerGoal(reward=4000)))
+env = ContextWrapper(env, options=random_options)
+
+# %%
+s, _ = env.reset()
+env.render()
+
+# %%
+n = 10
+
+# %%
+for t in range(100):
+    s, r, done, _, info = env.step()
+    if r > 0:
+        print(r, done, info["action"])
+    #         env.render()
+    if done:
+        if n == 0:
+            break
+        n -= r > 0
+        env.reset()