Merge pull request #81 from crytic/dev-remove-manticore

Remove manticore integration

.github/workflows/pip-audit.yml

@@ -0,0 +1,37 @@
+name: Scan dependencies for vulnerabilities with pip-audit
+
+on:
+  push:
+    branches: [ "master" ]
+  pull_request:
+    branches: [ "master" ]
+  schedule:
+    - cron: "0 12 * * *"
+
+jobs:
+  pip-audit:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Install Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.x"
+
+      - name: Install project
+        run: |
+          python -m venv /tmp/pip-audit-env
+          source /tmp/pip-audit-env/bin/activate
+
+          python -m pip install --upgrade pip
+          python -m pip install .
+
+
+      - name: Run pip-audit
+        uses: trailofbits/[email protected]
+        with:
+          virtual-environment: /tmp/pip-audit-env
+

Dockerfile

@@ -59,8 +59,9 @@ WORKDIR /home/etheno
 
 COPY --chown=etheno:etheno LICENSE setup.py etheno/
 COPY --chown=etheno:etheno etheno/*.py etheno/etheno/
+
 RUN cd etheno && \
-    pip3 install --no-cache-dir --user '.[manticore]' && \
+    pip3 install --no-cache-dir && \
     cd .. && \
     rm -rf etheno
 

README.md

@@ -7,7 +7,7 @@
 <br />
 
 
-Etheno is the Ethereum testing Swiss Army knife. It’s a JSON RPC multiplexer, analysis tool wrapper, and test integration tool. It eliminates the complexity of setting up analysis tools like [Manticore](https://github.com/trailofbits/manticore/) and [Echidna](https://github.com/trailofbits/echidna) on large, multi-contract projects. In particular, custom Manticore analysis scripts require less code, are simpler to write, and integrate with Truffle.
+Etheno is the Ethereum testing Swiss Army knife. It’s a JSON RPC multiplexer, analysis tool wrapper, and test integration tool. It eliminates the complexity of setting up analysis tools like [Echidna](https://github.com/trailofbits/echidna) on large, multi-contract projects.
 
 If you are a smart contract developer, you should use Etheno to test your contracts. If you are an Ethereum client developer, you should use Etheno to perform differential testing on your implementation. For example, Etheno is [capable of automatically reproducing](examples/ConstantinopleGasUsage) the Constantinople gas usage consensus bug that caused a fork on Ropsten.
 
@@ -19,14 +19,8 @@ Etheno is named after the Greek goddess [Stheno](https://en.wikipedia.org/wiki/S
   * API for filtering and modifying JSON RPC calls
   * Enables differential testing by sending JSON RPC sequences to multiple Ethereum clients
   * Deploy to and interact with multiple networks at the same time
-* **Analysis Tool Wrapper**: Etheno provides a JSON RPC client for advanced analysis tools like [Manticore](https://github.com/trailofbits/manticore/)
-  * Lowers barrier to entry for using advanced analysis tools
-  * No need for custom scripts to set up account and contract state
-  * Analyze arbitrary transactions without Solidity source code
 * **Integration with Test Frameworks** like Ganache and Truffle
   * Run a local test network with a single command
-  * Use Truffle migrations to bootstrap Manticore analyses
-  * Symbolic semantic annotations within unit tests
 
 ## Quickstart
 
@@ -35,10 +29,6 @@ Use our prebuilt Docker container to quickly install and try Etheno:
 ```
 docker pull trailofbits/etheno
 docker run -it trailofbits/etheno
-
-# Run one of the examples
-etheno@982abdc96791:~$ cd examples/BrokenMetaCoin/
-etheno@982abdc96791:~/examples/BrokenMetaCoin$ etheno --truffle --ganache --manticore --manticore-max-depth 2 --manticore-script ExploitMetaCoinManticoreScript.py
 ```
 
 Alternatively, natively install Etheno in a few shell commands:
@@ -52,7 +42,7 @@ pip3 install --user etheno
 
 # Use the Etheno CLI
 cd /path/to/a/truffle/project
-etheno --manticore --ganache --truffle
+etheno --ganache --truffle
 ```
 
 ## Usage
@@ -70,7 +60,7 @@ etheno https://client1.url.com:1234/ https://client2.url.com:8545/ http://client
 * `--port` or `-p` allows you to specify a port on which to run Etheno’s JSON RPC server (default is 8545)
 * `--run-publicly` allows incoming JSON RPC connections from external computers on the network
 * `--debug` will run a web-based interactive debugger in the event that an internal Etheno client throws an exception while processing a JSON RPC call; this should _never_ be used in conjunction with `--run-publicly`
-* `--master` or `-s` will set the “master” client, which will be used for synchronizing with Etheno clients like Manticore. If a master is not explicitly provided, it defaults to the first client listed.
+* `--master` or `-s` will set the “master” client, which will be used for synchronizing with Etheno clients. If a master is not explicitly provided, it defaults to the first client listed.
 * `--raw`, when prefixed before a client URL, will cause Etheno to auto-sign all transactions and submit then to the client as raw transactions
 
 ### Geth and Parity Integration
@@ -123,38 +113,13 @@ By default, Echidna deploys a generic fuzz testing contract to all clients, enum
 * `--fuzz-limit` limits the number of transactions that Echidna will emit
 * `--fuzz-contract` lets the user specify a custom contract for Echidna to deploy and fuzz
 
-### Manticore Client
-
-Manticore—which, by itself, does not implement a JSON RPC interface—can be run as an Etheno client, synchronizing its accounts with Etheno’s master client and symbolically executing all transactions sent to Etheno.
-```
-etheno --manticore
-```
-This alone will not run any Manticore analyses; they must either be run manually, or automated through [the `--truffle` command](#truffle-integration);
-
-* `--manticore-verbosity` sets Manticore’s logging verbosity (default is 3)
-* `--manticore-max-depth` sets the maximum state depth for Manticore to explore; if omitted, Manticore will have no depth limit
-
 ### Truffle Integration
 
 Truffle migrations can automatically be run within a Truffle project:
 ```
 etheno --truffle
 ```
 
-When combined with the `--manticore` option, this will automatically run Manticore’s default analyses on all contracts created once the Truffle migration completes:
-```
-etheno --truffle --manticore
-```
-
-This requires a master JSON RPC client, so will most often be used in conjunction with Ganache. If a local Ganache server is not running, you can simply add that to the command:
-```
-etheno --truffle --manticore --ganache
-```
-
-If you would like to run a custom Manticore script instead of the standard Manticore analysis and detectors, it can be specified using the `--manticore-script` or `-r` command.
-
-This script does not need to import Manticore or create a `ManticoreEVM` object; Etheno will run the script with a global variable called `manticore` that already contains all of the accounts and contracts automatically provisioned.  See the [`BrokenMetaCoin` Manticore script](examples/BrokenMetaCoin/ExploitMetaCoinManticoreScript.py) for an example.
-
 Additional arguments can be passed to Truffle using `--truffle-args`.
 
 ### Logging
@@ -174,7 +139,6 @@ saved:
 ## Requirements
 
 * Python 3.6 or newer
-* [Manticore](https://github.com/trailofbits/manticore/) release 0.2.2 or newer
 * [Flask](http://flask.pocoo.org/), which is used to run the JSON RPC server
 
 ### Optional Requirements

etheno/__main__.py

@@ -19,45 +19,19 @@
 from . import parity
 from . import truffle
 
-try:
-    from .manticoreclient import ManticoreClient
-    from . import manticoreutils
-    MANTICORE_INSTALLED = True
-except ModuleNotFoundError:
-    MANTICORE_INSTALLED = False
-
-
-def main(argv=None):
-    parser = argparse.ArgumentParser(description='An Ethereum JSON RPC multiplexer and Manticore wrapper')
-    parser.add_argument('--debug', action='store_true', default=False,
-                        help='Enable debugging from within the web server')
-    parser.add_argument('--run-publicly', action='store_true', default=False,
-                        help='Allow the web server to accept external connections')
-    parser.add_argument('-p', '--port', type=int, default=GETH_DEFAULT_RPC_PORT,
-                        help='Port on which to run the JSON RPC webserver (default=%d)' % GETH_DEFAULT_RPC_PORT)
-    parser.add_argument('-a', '--accounts', type=int, default=None,
-                        help='Number of accounts to create in the client (default=10)')
-    parser.add_argument('-b', '--balance', type=float, default=100.0,
-                        help='Default balance (in Ether) to seed to each account (default=100.0)')
-    parser.add_argument('-c', '--gas-price', type=int, default=None,
-                        help='Default gas price (default=20000000000)')
-    parser.add_argument('-i', '--network-id', type=int, default=None,
-                        help='Specify a network ID (default is the network ID of the master client)')
-    parser.add_argument('-m', '--manticore', action='store_true', default=False,
-                        help='Run all transactions through manticore')
-    parser.add_argument('-r', '--manticore-script', type=argparse.FileType('rb'), default=None,
-                        help='Instead of running automated detectors and analyses, run this Manticore script')
-    parser.add_argument('--manticore-max-depth', type=int, default=None,
-                        help='Maximum state depth for Manticore to explore')
-    parser.add_argument('-e', '--echidna', action='store_true', default=False,
-                        help='Fuzz the clients using transactions generated by Echidna')
-    parser.add_argument('--fuzz-limit', type=int, default=None,
-                        help='The maximum number of transactions for Echidna to generate (default=unlimited)')
-    parser.add_argument('--fuzz-contract', type=str, default=None,
-                        help='Path to a Solidity contract to have Echidna use for fuzzing (default is to use a builtin '
-                             'generic Echidna fuzzing contract)')
-    parser.add_argument('-t', '--truffle', action='store_true', default=False,
-                        help='Run the truffle migrations in the current directory and exit')
+def main(argv = None):
+    parser = argparse.ArgumentParser(description='An Ethereum JSON RPC multiplexer, differential fuzzer, and test framework integration tool.')
+    parser.add_argument('--debug', action='store_true', default=False, help='Enable debugging from within the web server')
+    parser.add_argument('--run-publicly', action='store_true', default=False, help='Allow the web server to accept external connections')
+    parser.add_argument('-p', '--port', type=int, default=GETH_DEFAULT_RPC_PORT, help='Port on which to run the JSON RPC webserver (default=%d)' % GETH_DEFAULT_RPC_PORT)
+    parser.add_argument('-a', '--accounts', type=int, default=None, help='Number of accounts to create in the client (default=10)')
+    parser.add_argument('-b', '--balance', type=float, default=100.0, help='Default balance (in Ether) to seed to each account (default=100.0)')
+    parser.add_argument('-c', '--gas-price', type=int, default=None, help='Default gas price (default=20000000000)')
+    parser.add_argument('-i', '--network-id', type=int, default=None, help='Specify a network ID (default is the network ID of the master client)')
+    parser.add_argument('-e', '--echidna', action='store_true', default=False, help='Fuzz the clients using transactions generated by Echidna')
+    parser.add_argument('--fuzz-limit', type=int, default=None, help='The maximum number of transactions for Echidna to generate (default=unlimited)')
+    parser.add_argument('--fuzz-contract', type=str, default=None, help='Path to a Solidity contract to have Echidna use for fuzzing (default is to use a builtin generic Echidna fuzzing contract)')
+    parser.add_argument('-t', '--truffle', action='store_true', default=False, help='Run the truffle migrations in the current directory and exit')
     parser.add_argument('--truffle-cmd', type=str, default='truffle', help='Command to run truffle (default=truffle)')
     parser.add_argument('--truffle-args', type=str, default='migrate',
                         help='Arguments to pass to truffle (default=migrate)')
@@ -318,33 +292,6 @@ def main(argv=None):
     for client in args.raw:
         ETHENO.add_client(RawTransactionClient(RpcProxyClient(client), accounts))
 
-    manticore_client = None
-    if args.manticore:
-        if not MANTICORE_INSTALLED:
-            ETHENO.logger.error('Manticore is not installed! Running Etheno with Manticore requires Manticore version '
-                                '0.2.2 or newer. Reinstall Etheno with Manticore support by running '
-                                '`pip3 install --user \'etheno[manticore]\'`, or install Manticore separately with '
-                                '`pip3 install --user \'manticore\'`')
-            sys.exit(1)
-        new_enough = manticoreutils.manticore_is_new_enough()
-        if new_enough is None:
-            ETHENO.logger.warning(f"Unknown Manticore version {manticoreutils.manticore_version()}; it may not be new "
-                                  "enough to have Etheno support!")
-        elif not new_enough:
-            ETHENO.logger.error(f"The version of Manticore installed is {manticoreutils.manticore_version()}, but the "
-                                f"minimum required version with Etheno support is 0.2.2. We will try to proceed, but "
-                                f"things might not work correctly! Please upgrade Manticore.")
-        manticore_client = ManticoreClient()
-        ETHENO.add_client(manticore_client)
-        if args.manticore_max_depth is not None:
-            manticore_client.manticore.register_detector(manticoreutils.StopAtDepth(args.manticore_max_depth))
-        if manticoreutils.manticore_is_new_enough(0, 2, 4):
-            # the verbosity static method was deprecated
-            from manticore.utils.log import set_verbosity
-            set_verbosity(getattr(logger, args.log_level))
-        else:
-            manticore_client.manticore.verbosity(getattr(logger, args.log_level))
-
     if args.truffle:
         truffle_controller = truffle.Truffle(truffle_cmd=args.truffle_cmd, parent_logger=ETHENO.logger)
 
@@ -361,30 +308,16 @@ def truffle_thread():
             for plugin in ETHENO.plugins:
                 plugin.finalize()
 
-            if manticore_client is not None:
-                if args.manticore_script is not None:
-                    f = args.manticore_script
-                    code = compile(f.read(), f.name, 'exec')
-                    exec(code, {
-                        'manticore': manticore_client.manticore,
-                        'manticoreutils': manticoreutils,
-                        'logger': logger.EthenoLogger(os.path.basename(args.manticore_script.name), parent=manticore_client.logger)
-                    })
-                else:
-                    manticoreutils.register_all_detectors(manticore_client.manticore)
-                    manticore_client.multi_tx_analysis()
-                    manticore_client.manticore.finalize()
-                manticore_client.logger.info("Results are in %s" % manticore_client.manticore.workspace)
-                ETHENO.shutdown()
-            elif not ETHENO.clients and not ETHENO.plugins:
+            if not ETHENO.clients and not ETHENO.plugins:
                 ETHENO.logger.info("No clients or plugins running; exiting...")
                 ETHENO.shutdown()
 
         thread = Thread(target=truffle_thread)
         thread.start()
 
-    if args.run_differential and (ETHENO.master_client is not None) and \
-            next(filter(lambda c: not isinstance(c, ManticoreClient), ETHENO.clients), False):
+    # Without Manticore integration the only client types are geth, parity, and command-line raw/regular clients.
+    # So checking len() >= 1 should be sufficient.
+    if args.run_differential and (ETHENO.master_client is not None) and len(ETHENO.clients) >= 1:
         # There are at least two non-Manticore clients running
         ETHENO.logger.info("Initializing differential tests to compare clients %s" % ', '.join(
             map(str, [ETHENO.master_client] + ETHENO.clients)

etheno/logger.py

@@ -108,13 +108,6 @@ def getLogger(name: str):
         ret = ETHENO_LOGGERS[name]
     else:
         ret = _LOGGING_GETLOGGER(name)
-    # ####BEGIN####
-    # Horrible hack to workaround Manticore's global logging system.
-    # This can be removed after https://github.com/trailofbits/manticore/issues/1369
-    # is resolved.
-    if name.startswith('manticore'):
-        ret.propagate = False
-    # ####END####
     return ret