Skip to content

OOPTOS user manual (for v0.8.0 or earlier)

Jump to bottom
Wataru Ishida edited this page Jun 27, 2019 · 1 revision

OOPTOS

OOPTOS is a network OS for Open Optical Packet Transponder (OOPT). Current supported platform is Edge-core Cassini.

OOPTOS uses many OSS internally. It uses ONL for base OS, SONiC for Ethernet ASIC management, and Kubernetes for container orchestration.

The ultimate goal of OOPTOS is to get merged into SONiC. After this get achieved, SONiC can support not only datacenter switches but also optical transponder.

Getting started

Installation

OOPTOS can be installed via ONIE.

You can download the onie-installer of OOPTOS from here.

Refer to ONIE user guide how to use ONIE to install NOS.

login info

After successful installation, OOPTOS will be booted automatically and login prompt will be shown. Use the following account to log in.

user : root, password : x1

setup the system

You need to assign an IP address to ma1 interface and set the default gateway. By default, DHCP is enabled for ma1 interface. Modify /etc/network/interfaces as you like. The internet connection is not mandatory.

Also, you need to set the correct time to make TLS work properly. Setting up NTP is also recommended.

$ date -s ‘Aug 15 19:00 2018’

note: date command uses UTC by default.

After setting IP address and time, we will start minikube. minikube is a tool to run single node Kubernetes.

$ systemctl start minikube

note: if starting minikube fails, you can use systemctl and journalctl command to debug. Most of the case, the reason for the failure is IP address and time setting. Removing garbage files by executing minikube delete could also fix the issue.

Kubernetes(k8s) is container orchestration system. Most of the key components of OOPTOS runs in a container, and its lifecycle is managed by k8s.

k8s can be configured by kubectl command. You can check k8s is properly running by

$ kubectl get node
NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     master        16d       v1.11.0

note: refer to https://kubernetes.io/docs/reference/kubectl/overview/ for kubectl details

After you check k8s is running, start OOPTOS system by

$ systemctl start oopt

This will boot up key components of OOPTOS. You can check what is booted by

$ kubectl get pods
NAME        READY     STATUS    RESTARTS   AGE
ofdpa       1/1       Running   0          1m
redis       1/1       Running   0          1m
sonic       7/7       Running   0          1m
transyncd   1/1       Running   0          1m

note: ofdpa pod is running OF-DPA agent to manage Broadcom ASIC, sonic pod is running SONiC agents, redis pod is running Redis database, transyncd pod is running transyncd agent to manage transport devices

Now, you are ready to use OOPTOS.

You don’t need to explicitly shut down the system before powering off. You can just power off the switch after executing

$ shutdown -h now

If you want to shut down only OOPTOS system, you can do that by

$ systemctl stop oopt

Next, we will dive into how to configure OOPTOS. We use oopt command for that.

system configuration

oopt is a CLI tool to configure OOPTOS.

In OOPTOS, we have 3 components to configure. optical-module, port and interface. optical-module means line side interface. Cassini has 8 optical modules. port means client side physical port. Cassini has 16 client ports.

interface means client side interface. By default, it has 1 to 1 mapping to port but when we configure breakout setting for a port, multiple interfaces will be mapped to 1 port.

Let's start from optical-module configuration.

Initial configuration of optical-module looks like this.

$ oopt optical-module
{
   "config": {
      "name": "Opt3"
   },
   "name": "Opt3"
}
{
   "config": {
      "name": "Opt4"
   },
   "name": "Opt4"
}
{
   "config": {
      "name": "Opt5"
   },
   "name": "Opt5"
}
{
   "config": {
      "name": "Opt6"
   },
   "name": "Opt6"
}
{
   "config": {
      "name": "Opt7"
   },
   "name": "Opt7"
}
{
   "config": {
      "name": "Opt8"
   },
   "name": "Opt8"
}
{
   "config": {
      "name": "Opt1"
   },
   "name": "Opt1"
}
{
   "config": {
      "name": "Opt2"
   },
   "name": "Opt2"
}

The index starts from 1. Opt1 corresponds to the leftmost line slot.

Suppose you inserted PIU and CFP2 transceiver into Opt1. OOPTOS will automatically detect the existence of both PIU and CFP2 transceiver and start booting it by default setting.

Let’s check the status of Opt1.

$ oopt opt Opt1 state
{
   "config": {
      "name": "Opt1"
   },
   "name": "Opt1",
   "optical-module-rms": {
      "state": {
         "xi": 73,
         "xq": 70,
         "yi": 70,
         "yq": 64
      }
   },
   "state": {
      "channel-stats": {
         "A": {
            "hd-fec-ber": "4.310749e-07",
            "name": "A",
            "post-fec-ber": "0.000000e+00",
            "sd-fec-ber": "7.797305e-03"
         },
         "B": {
            "hd-fec-ber": "4.395004e-07",
            "name": "B",
            "post-fec-ber": "0.000000e+00",
            "sd-fec-ber": "7.803342e-03"
         }
      },
      "operation-status": "STATE_READY",
      "sync-error": false
   }
}

note : you do not need to type whole command name when the prefix is unique among all commands. The example above uses opt which is equivalent to typing optical-module.

Please wait until the operation-status goes to STATE_READY

Here is the sequence of operation-status.

  1. STATE_DOWN
    • PIU or CFP2 transceiver is not plugged in or the module is disabled by admin.
  2. STATE_BOOTING_TOP_HALF
    • initialization stage ( this stage would take around 1 minute )
  3. STATE_WAITING_RX_SIGNAL
    • waiting RX signal
  4. STATE_BOOTING_BOTTOM_HALF
    • final adjustment including optimization
  5. STATE_READY
    • the link is ready

After the state goes to STATE_READY, check sync_error field is false and post-fec-ber field is 0.000000e+00.

If this is not the case, the link is not properly working. There are many possible reasons for this including the physical optical link status. See here for details.

You can change the frequency by

$ oopt opt Opt1 frequency grid GRID_50GHZ
$ oopt opt Opt1 frequency channel 10
$ oopt commit

You need to execute oopt commit to reflect the changes to the hardware.

You can change the modulation type by

$ oopt opt Opt1 modulation-type DP_16QAM
$ oopt commit

When using DP_16QAM modulation ( which is default setting ), the link capacity is 200Gbps. You can see two channel A and B in the state output above. Both channels are 100Gbps (total 200Gbps) and connected to Broadcom ASIC's 100GbE port respectively (using 2 client ports inside the box).

When using DP_QPSK modulation, the link capacity is 100Gbps. Only channel A is available in this mode.

For the complete oopt command see oopt user manual

Next, we need to connect a client interface to an optical module to make traffic flow.

Suppose you have connected your switch or router and Port1 with 100GbE QPSK28 transceivers.

Let's check the Port1 configuration.

$ oopt port Port1
{
   "breakout-mode": {
      "config": {
         "channel-speed": "SPEED_100GB",
         "num-channels": 1
      }
   },
   "config": {
      "name": "Port1"
   },
   "name": "Port1"
}

As you can see, every port is configured as a 100GbE port by default. The corresponding interface for Port1 is Ethernet1.

Let's see Ethernet1 state.

$ oopt interface Ethernet1 state
{
   "config": {
      "mtu": 1500,
      "name": "Ethernet1"
   },
   "name": "Ethernet1",
   "state": {
      "admin-status": "DOWN",
      "oper-status": "UP"
   }
}

check oper-status is UP.

If it is DOWN, the client port is not correctly working. Please check the client link connection.

You don't need to care about admin-status for now.

Next, we connect this Ethernet1 interface to Opt1's channel A.

$ oopt interface Ethernet1 optical-module-connection optical-module name Opt1
$ oopt interface Ethernet1 optical-module-connection optical-module channel A
$ oopt interface Ethernet1 optical-module-connection id 1000
$ oopt commit

Adding to specifying the optical-module, we also need to assign id for the connection. Internally this id is used for VLAN ID to tag the traffic from client interfaces when going out from the optical module. In this way, OOPTOS can support multiplexing multiple client interfaces into one optical module.

Now you are ready to forward the traffic. Please make sure you do the same setting for Cassini at the opposite end of the fiber link.

TIPs

- link is not synced or not error-free ( post FEC BER is not 0 )

There are many possible reasons for this. Following may solve the issue.

  1. try loopback
  2. unplug and plug in the optical fiber
  3. toggle PRBS or LOSI setting

- useful commands for debugging

  • kubectl logs transyncd -f

    • show log of transyncd. adding -f will stream the log (blocking)
    • see here for more details

  • kubectl logs sonic syncd -f

    • show log of syncd
    • you can see Ethernet port link up and down message by seeing this log

  • client_port_table_dump, client_flowtable_dump, client_grouptable_dump

    • these are utility commands of OF-DPA
    • these commands dump internal table contents of Ethernet ASIC

  • docker ps, docker images

    • show what kind of containers are running in the system
    • see here for more details

- jq

The output of oopt is formatted in the sequence of json. You can use jq to format the output as you like.

$ apt update
$ apt install -qy jq
$ oopt  opt | jq -r .name
Opt8
Opt1
Opt2
Opt3
Opt4
Opt5
Opt6
Opt7
Clone this wiki locally