multiswitch.p4app starts and stops just after

[bakkarol]

Hi!

I have problems while running multiswitch.p4app. App starts and and stops just after. Am I able to run it like simple_router.p4app (ping hosts for example) and make some changes in topology? Run on VM of Ubuntu 16.04 LTS.

$ sudo p4app run multiswitch.p4app
Entering build directory.
Extracting package.
Reading package manifest.
> p4c-bm2-ss --p4v 16 "simple_router.p4" -o "simple_router.p4.json"
> python2 "/scripts/mininet/multi_switch_mininet.py" --log-dir "/tmp/p4app_logs" --manifest "./p4app.json" --target "multiswitch" --auto-control-plane --behavioral-exe "simple_switch" --json "simple_router.p4.json"
*** Error setting resource limits. Mininet's performance may be affected.
*** Creating network
*** Adding hosts:
h1 h2 
*** Adding switches:
s1 s2 
*** Adding links:
(0ms delay) (0ms delay) (h1, s1) (50ms delay) (50ms delay) (h2, s2) (0ms delay) (0ms delay) (s1, s2) 
*** Configuring hosts
h1 h2 
*** Starting controller

*** Starting 2 switches
s1 Starting P4 switch s1.
simple_switch -i 1@s1-eth1 -i 2@s1-eth2 --thrift-port 9090 --nanolog ipc:///tmp/bm-0-log.ipc --device-id 0 simple_router.p4.json
P4 switch s1 has been started.
s2 Starting P4 switch s2.
simple_switch -i 1@s2-eth1 -i 2@s2-eth2 --thrift-port 9091 --nanolog ipc:///tmp/bm-1-log.ipc --device-id 1 simple_router.p4.json
P4 switch s2 has been started.

table_set_default send_frame _drop
table_set_default forward _drop
table_set_default ipv4_lpm _drop
table_add send_frame rewrite_mac 1 => 00:aa:00:02:00:02
table_add forward set_dmac 10.0.2.101 => 00:04:00:00:02:01
table_add ipv4_lpm set_nhop 10.0.2.101/32 => 10.0.2.101 1
table_add send_frame rewrite_mac 2 => 00:aa:00:02:01:00
table_add forward set_dmac 10.0.1.101 => 00:aa:00:01:02:00
table_add ipv4_lpm set_nhop 10.0.1.101/32 => 10.0.1.101 2
Obtaining JSON from switch...
Done
Control utility for runtime P4 table manipulation
RuntimeCmd: Setting default action of send_frame
action:              _drop
runtime data:        
RuntimeCmd: Setting default action of forward
action:              _drop
runtime data:        
RuntimeCmd: Setting default action of ipv4_lpm
action:              _drop
runtime data:        
RuntimeCmd: Adding entry to exact match table send_frame
match key:           EXACT-00:01
action:              rewrite_mac
runtime data:        00:aa:00:02:00:02
Entry has been added with handle 0
RuntimeCmd: Adding entry to exact match table forward
match key:           EXACT-0a:00:02:65
action:              set_dmac
runtime data:        00:04:00:00:02:01
Entry has been added with handle 0
RuntimeCmd: Adding entry to lpm match table ipv4_lpm
match key:           LPM-0a:00:02:65/32
action:              set_nhop
runtime data:        0a:00:02:65	00:01
Entry has been added with handle 0
RuntimeCmd: Adding entry to exact match table send_frame
match key:           EXACT-00:02
action:              rewrite_mac
runtime data:        00:aa:00:02:01:00
Entry has been added with handle 1
RuntimeCmd: Adding entry to exact match table forward
match key:           EXACT-0a:00:01:65
action:              set_dmac
runtime data:        00:aa:00:01:02:00
Entry has been added with handle 1
RuntimeCmd: Adding entry to lpm match table ipv4_lpm
match key:           LPM-0a:00:01:65/32
action:              set_nhop
runtime data:        0a:00:01:65	00:02
Entry has been added with handle 1
RuntimeCmd: 

table_set_default send_frame _drop
table_set_default forward _drop
table_set_default ipv4_lpm _drop
table_add send_frame rewrite_mac 1 => 00:aa:00:01:00:01
table_add forward set_dmac 10.0.1.101 => 00:04:00:00:01:01
table_add ipv4_lpm set_nhop 10.0.1.101/32 => 10.0.1.101 1
table_add send_frame rewrite_mac 2 => 00:aa:00:01:02:00
table_add forward set_dmac 10.0.2.101 => 00:aa:00:02:01:00
table_add ipv4_lpm set_nhop 10.0.2.101/32 => 10.0.2.101 2
Obtaining JSON from switch...
Done
Control utility for runtime P4 table manipulation
RuntimeCmd: Setting default action of send_frame
action:              _drop
runtime data:        
RuntimeCmd: Setting default action of forward
action:              _drop
runtime data:        
RuntimeCmd: Setting default action of ipv4_lpm
action:              _drop
runtime data:        
RuntimeCmd: Adding entry to exact match table send_frame
match key:           EXACT-00:01
action:              rewrite_mac
runtime data:        00:aa:00:01:00:01
Entry has been added with handle 0
RuntimeCmd: Adding entry to exact match table forward
match key:           EXACT-0a:00:01:65
action:              set_dmac
runtime data:        00:04:00:00:01:01
Entry has been added with handle 0
RuntimeCmd: Adding entry to lpm match table ipv4_lpm
match key:           LPM-0a:00:01:65/32
action:              set_nhop
runtime data:        0a:00:01:65	00:01
Entry has been added with handle 0
RuntimeCmd: Adding entry to exact match table send_frame
match key:           EXACT-00:02
action:              rewrite_mac
runtime data:        00:aa:00:01:02:00
Entry has been added with handle 1
RuntimeCmd: Adding entry to exact match table forward
match key:           EXACT-0a:00:02:65
action:              set_dmac
runtime data:        00:aa:00:02:01:00
Entry has been added with handle 1
RuntimeCmd: Adding entry to lpm match table ipv4_lpm
match key:           LPM-0a:00:02:65/32
action:              set_nhop
runtime data:        0a:00:02:65	00:02
Entry has been added with handle 1
RuntimeCmd: 

**********
Network configuration for: h1
Default interface: h1-eth0	10.0.0.1	00:04:00:00:01:01
**********
**********
Network configuration for: h2
Default interface: h2-eth0	10.0.0.2	00:04:00:00:02:01
**********
port: 8000
echo_msg: foobar

h1 python echo_server.py $port
h2 python echo_client.py 10.0.1.101 $port $echo_msg
(None, '')
(None, '')
*** Stopping 0 controllers

*** Stopping 3 links
...
*** Stopping 2 switches
s1 s2 
*** Stopping 2 hosts
h1 h2 
*** Done

[robertsoule-barefoot]

Hi,

This will execute the commands in the p4app.json file ("cmd") and then terminate. If you want your program to start the Mininet CLI, then you should add "cli": true to the target configuration.

If you want more control over the topology, you may want to consider using a custom topology class:

https://github.com/p4lang/p4app#custom-topology-class

cheers,
Robert

[bakkarol]

@robertsoule-barefoot could you tell sth wider about adding a CLI into config? I found a CLI section in p4app/docker/scripts/p4apprunner.py , which was off:

def run_multiswitch(manifest):
    output_file = run_compile_bmv2(manifest)

    script_args = []
    script_args.append('--log-dir "/tmp/p4app_logs"')
    script_args.append('--manifest "%s"' % args.manifest)
    script_args.append('--target "%s"' % manifest.target)
    if 'auto-control-plane' in manifest.target_config and manifest.target_config['auto-control-plane']:
        script_args.append('--auto-control-plane' )
    script_args.append('--behavioral-exe "%s"' % 'simple_switch')
    script_args.append('--json "%s"' % output_file)
    #script_args.append('--cli')     # <----------------------- HERE

    program = '"%s/mininet/multi_switch_mininet.py"' % sys.path[0]
    return run_command('python2 %s %s' % (program, ' '.join(script_args)))

after enabled CLI and add "cli": true into multiswitch p4app.json file,

{
  "program": "simple_router.p4",
  "language": "p4-16",
  "targets": {
      "multiswitch": {
      "cli": true,
      "auto-control-plane": true,
      "links": [["h1", "s1"], ["s1", "s2"], ["s2", "h2", 50]],
      "hosts": {
        "h1": {
            "cmd": "python echo_server.py $port",
            "startup_sleep": 0.2,
            "wait": false
        },
        "h2": {
            "cmd": "python echo_client.py h1 $port $echo_msg",
            "wait": true
        }
      },
      "parameters": {
        "port": 8000,
        "echo_msg": "foobar"
      }
    }
  }
}

CLI starts. Is it proper way to get it started, with no consequences in further customization? Or there is other, better way to make it work?

Another question is whether there is a possibility to connect more than one switch to the host?

[robertsoule]

Hi @bakkarol , yes, that should be fine. I'm not sure why we had that line commented out in p4apprunner.py.

[bakkarol]

While running some custom topo on multiswitch.p4app I noticed that there is only one virtual interface available with name veth####### and some random ending like veth1260009 or vethf4f5549. While running simple_router form bmv2, all interfaces of elements in network are listed in ifconfig. Is it possible to use some network analyzer (like Wireshark) while run my topo in p4app-way?

[robertsoule-barefoot]

Unfortunately, it is not easy to run GUI-based applications, like Wireshark, from within the docker container. Online, you may be able to find some solutions such as using SSH forwarding of X11, but I don't have much experience with them:

http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/

As an alternative, you can use a command line tool like tcpdump. You can run this from the Wireshark command prompt on one of the hosts (e.g., mininet> h1 tcpdump .... ), or from outside the docker container using a docker exec command.

[theojepsen]

Hi @bakkarol,

Since p4app runs inside a Docker container, you will not see the interfaces from your host. You must enter the container to see/capture the interfaces. Try the following steps.

First, enable CLI mode, by adding "cli": true to the p4app.json. Run the p4app. You will see the mininet CLI, but nothing will start running yet. In another terminal, find the ID of the Docker container that the p4app is running in:

docker ps

Every time you start a p4app, the container will have a different ID. Use the container ID (in my case, it's 640488330e03) to connect to the container and start capturing packets:

docker exec 640488330e03 tcpdump -i s1-eth1 -U -w - | wireshark -k -i -

Then, go back to the other terminal and exit the CLI (type exit or ender CTRL+D). When the p4app runs and starts sending packets, you should see them in the wireshark window.

I hope this works for you.

[bakkarol]

Hello @theojepsen ,

Thank you for your response. When doing it in your way, with sudo privilages, wireshark starts

tcpdump: listening on s1-eth1, link-type EN10MB (Ethernet), capture size 262144 bytes

(wireshark:21463): GConf-WARNING **: Client failed to connect to the D-BUS daemon:
Failed to connect to socket /tmp/dbus-mFhzDiEtB0: Połączenie odrzucone
13:04:27          Warn Could not compile "of" in colorfilters file "/home/mininet/.wireshark/colorfilters".
"of" is neither a field nor a protocol name.
13:04:27          Warn Could not compile "Checksum Errors" in colorfilters file "/home/mininet/.wireshark/colorfilters".
Neither "cdp.checksum_bad" nor "1" are field or protocol names.

but shows error window with info:

Couldn't run /usr/bin/dumpcap in child process: Access denied

and, there si no interfaces in mininet. When I run h1 ping h2 in first terminal, also nothing happen.

I'm not sure if I clearly understand your advice because when i stop CLI in first terminal, it stops docker container and then, there is also nothing to measure.

[theojepsen]

You are getting the Access denied error because your user is not in the wireshark group. Follow the following steps and logout/login:
https://askubuntu.com/a/461037

From the output you showed in the first post, it looks like the multiswitch.p4app ran correctly. What is the problem?

Also, why are you running p4app with sudo (i.e. sudo p4app run multiswitch.p4app). If your user is in the docker group, you should not need to run it as root.

[bakkarol]

@theojepsen after adding user into wireshark and group, still not able to see interfaces.

To avoid problems with containers, I'm trying to run topo outside the docker, in regular way but I'm doing sth wrong. I'm using code example from source_routing in SIGCOMM2015 tutorial.

from mininet.net import Mininet
from mininet.topo import Topo
from mininet.log import setLogLevel, info
from mininet.cli import CLI

from p4_mininet import P4Switch, P4Host

import argparse
from time import sleep

parser = argparse.ArgumentParser(description='Mininet demo')
parser.add_argument('--behavioral-exe', help='Path to behavioral executable',
                    type=str, action="store", required=True)
parser.add_argument('--thrift-port', help='Thrift server port for table updates',
                    type=int, action="store", default=9090)
parser.add_argument('--json', help='Path to JSON config file',
                    type=str, action="store", required=True)

args = parser.parse_args()

class MyTopo(Topo):
    def __init__(self, sw_path, json_path,thrift_port, nb_hosts, nb_switches, links, **opts):
        # Initialize topology and default options
        Topo.__init__(self, **opts)

        for i in xrange(nb_switches):
            switch = self.addSwitch('s%d' % (i + 1),
                                    sw_path = sw_path,
                                    json_path = json_path,
                                    thrift_port = thrift_port +i,
                                    pcap_dump = True,
                                    device_id = i)

        for h in xrange(nb_hosts):
            host = self.addHost('h%d' % (h + 1),
                                ip = "10.0.%d.10/24" % h,
                                mac = '00:04:00:00:00:%02x' %h)

        for a, b in links:
            self.addLink(a, b)

def read_topo():
    nb_hosts = 0
    nb_switches = 0
    links = []
    with open("topo.txt", "r") as f:
        line = f.readline()[:-1]
        w, nb_switches = line.split()
        assert(w == "switches")
        line = f.readline()[:-1]
        w, nb_hosts = line.split()
        assert(w == "hosts")
        for line in f:
            if not f: break
            a, b = line.split()
            links.append( (a, b) )
    return int(nb_hosts), int(nb_switches), links

def main():

    nb_hosts, nb_switches, links = read_topo()

    topo = MyTopo(args.behavioral_exe,
                            args.json,
                            args.thrift_port,
                            nb_hosts, nb_switches, links)

    net = Mininet(topo = topo,
                  host = P4Host,
                  switch = P4Switch,
                  controller = None)
    net.start()

    CLI( net )
    net.stop()

if __name__ == '__main__':
    setLogLevel( 'info' )
    main()

Script run correctly in the way shown in
https://github.com/p4lang/behavioral-model#integrating-with-mininet
with topo : h1 --- s1 --- s2 --- h2 , net command gives

containernet> net
h1 h1-eth0:s1-eth1
h2 h2-eth0:s2-eth2
s1 lo:  s1-eth1:h1-eth0 s1-eth2:s2-eth1
s2 lo:  s2-eth1:s1-eth2 s2-eth2:h2-eth0

but ping gives

containernet> h1 ping h2
connect: Network is unreachable

@robertsoule-barefoot @theojepsen , may I ask for some advice or tips?

[robertsoule]

Just to be clear on what you did: did you implement the logic in the P4 switch to handle ICMP packets? I think the SIGCOMM2015 tutorial sample switch code:

https://github.com/p4lang/tutorials/blob/master/SIGCOMM_2015/source_routing/p4src/source_routing.p4

Would not be able to forward ping packets.

[duanev]

It looks like the multiswitch mininet backend is using ARP - is that why ICMP needs to be supported? Would there be any way to pre-load the arp tables in the hosts to avoid this?

[theojepsen]

Hi @duanev,

The multiswitch backend does not use ARP messages for configuring the hosts.

By default, the multiswitch backend assigns a MAC address and IP address to each interface. Then, on each host, an ARP entry is added so that the host knows how to route packets to adjacent switches. The multiswitch backend then adds l1/l2 table entries to each switch so that they know how to route a packet from one host to another.

If you don't want the multiswitch backend to automatically configure the hosts, you can set the auto-control-plane option to false. By default, it is set to true. It should be set within the multiswitch target, like so:

p4app/examples/multiswitch.p4app/p4app.json

Line 6 in 1d9a575

"auto-control-plane": true,

[duanev]

Ok, yes, I've set auto-control-plane to false, so I imagine I now need to do my own port routing. In the following output, can I assume the eth number for each switch matches the egress/ingress port number?

mininet> net
h1 h1-eth0:s1-eth1
h2 h2-eth0:s2-eth1
h3 h3-eth0:s3-eth1
s1 lo:  s1-eth1:h1-eth0 s1-eth2:s2-eth2 s1-eth3:s3-eth3
s2 lo:  s2-eth1:h2-eth0 s2-eth2:s1-eth2 s2-eth3:s3-eth2
s3 lo:  s3-eth1:h3-eth0 s3-eth2:s2-eth3 s3-eth3:s1-eth3

Last, is there a way to programmatically know (via metadata maybe?) which ports are active on each switch, and would simulating link up/down events be somewhere in the dev pipe?

[theojepsen]

can I assume the eth number for each switch matches the egress/ingress port number?

Yes. Also, the port number corresponds to the order in which the links were added when creating the topology.

Last, is there a way to programmatically know (via metadata maybe?) which ports are active on each switch

As far as I know, there is no way to find the state of ports from within your P4 program. You should provide your program with this information from the control plane. This could be done with a table on the switch that is populated by the controller.

and would simulating link up/down events be somewhere in the dev pipe?

What do you mean by the "dev pipe"? Right now there is no way of bringing links up/down. You could specify your own controller in the p4app.json, and have the controller add/remove table entries from the switch, to signal ports being available/unavailable.

[duanev]

sorry: dev pipe -> developer's list of features to implement

Specifying my own controller has helped, but there appears to be some default iface config in P4Host (p4_mininet.py) that I can't override - even with ifconfig commented out in the controller, a 10. ip is still given to each host.