Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this tutorial, you will learn how to setup and This guide does NOT provide detailed instructions, but rather, pointers to existing guides that we hope should give you enough information to set up an ONOS environment capable of communicating with P4Runtime-enabled Tofino-Based devices. The following set of instructions have been tested with the EdgeCore Wedge-100BF.

Table of Contents

Requirements

  • 1 or more Tofino based switches with Barefoot SDE 6.0.0 or later installed
  • 1 server with the latest ONOS master downloaded. the version has to be 1.12-SNAPSHOT or later.

Prepare the switch

With the SDE and all necessary tools installed the switch process can be started.

Code Block
languagebash
titleStart switchd
bf_switchd --install-dir $SDE_INSTALL --conf-file $SDE_INSTALL/share/p4/targets/skip_p4.conf --skip-p4

It's worth noting the configuration file (skip_p4.conf). This configuration option that makes the switchd start with no P4 program deployed. The program will be later deployed through a specific call from ONOS.  Please also note that the skip_p4.conf file will only be installed if the SDE is build & installed with the p4_examples. Otherwise this file can be found in the p4_examples tarball in the SDE package with the name tofino_skip_p4.conf.in

Create a Pipeconf for your P4 program. 

To Build a pipeconf for your p4 program and make ONOS capable of installing it on the device and controlling it you need to write an application

The application must live under onos/pipelines/<p4 program name> and must contain contain at least two classes:

  • implementation of Interpreter.java
  • implementation of PiPipeconfFactory.java

An example of such application can be found under onos/pipeconfs/default this contains the default.p4 P4_16 program, the DefaultPipeconf, DefaultInterpreter and other classes related to the Default.p4 program.

Interpreter

The interpreter is what enables ONOS to understand the specific constructs of your P4 Program. For example the Interpreter enables the translation from Traffic treatments into P4 defined Actions.

You can see an example of an interpreter for the default.p4 program in the DefaultP4Interpreter.java class. 

PiPipeconfFactory

The PiPipeconfFactory is where you piece together all the needed elements for ONOS to understand your P4 program.

You need:

  • bmv2 generated json for your program. 
  • P4Info for your program. 
  • Tofino.bin binary file
  • Tofino context.json file

The bmv2 json and the P4Info can be generated through. 

Code Block
languagebash
titlebmv2 json and P4Info
p4c-bm2-ss default.p4 --p4-16 -o default.json --p4runtime-file default.p4info --p4runtime-format text

This command uses the frontend compiler only.

The Tofino.bin and the context.json can be generated only with a Barefoot SDE and the attached Tofino specific backend compiler. For more information write to onos-dev@onosproject.org 

An example of a Pipeconf builder:

Code Block
languagejava
titlePipeconf Builder
return DefaultPiPipeconf.builder()
        .withId(new PiPipeconfId(format(PIPECONF_ID_BASE, system)))
        .withPipelineModel(Bmv2PipelineModelParser.parse(jsonUrl))
        .addBehaviour(PiPipelineInterpreter.class, DefaultP4Interpreter.class)
        .addBehaviour(Pipeliner.class, DefaultSingleTablePipeline.class)
        .addBehaviour(PortStatisticsDiscovery.class, DefaultP4PortStatisticsDiscovery.class)
        .addExtension(P4_INFO_TEXT, p4InfoUrl)
        .addExtension(BMV2_JSON, jsonUrl)
        .addExtension(TOFINO_BIN, tofinoUrl)
        .addExtension(TOFINO_CONTEXT_JSON, contextUrl)
        .build();

As you can see we are giving the 4 needed files. We are also compiling the pipeline model from the BMv2 son and adding the Interpreter behaviour. In the Pipeconf Builder you can also add other behaviours, specific to the P4 program such as the Pipeliner and the PortStatisticsDiscovery needed to leverage existing ONOS functionality. 

We are also setting a PipeconfId, this is the unique Id ONOS will use to reference the Pipeconf and that we will use in the net-cfg.json later.

The Pipeconf factory can also be the component that reports the pipeconf to the PiPipeconfService through the registerPipeconf method.

Run ONOS and bring the Pipeconf

Moving to the ONOS controller on the server, assuming you downloaded it and placed your pipeconf in it.

Run ONOS

You need to run it:

Code Block
languagebash
titleStart ONOS
$ buck run onos-local -- clean debug

It's worth noting that this start ONOS in a single instance cluster. The command also build ONOS and purges any previous state. The debug option offers the possibility to attach the debugger on port 5005. 

Login into the ONOS CLI

Having Started ONOS we need to login in it's CLI.

Code Block
languagebash
onos localhost

Start the Pipeconf

Having Started ONOS we need to start the pipeconf.

Code Block
languagebash
titleStart ONOS
onos> app activate <pipeconf_app_name>

It's worth noting that this start ONOS in a single instance cluster. The command also build ONOS and purges any previous state. The debug option offers the possibility to attach the debugger on port 5005. 

Start the Barefoot driver

Having Started ONOS we need to start the toxin drivers.

Code Block
languagebash
titleActivate Drivers
onos> app activate org.onosproject.drivers.barefoot

This command brings in all the needed applications to interact with the switch.

Verify the active applications

please type

Code Block
languagebash
apps -s -a

and verify that these app at least are active in your ONOS environment:

  • org.onosproject.generaldeviceprovider (General Device Provider)

  • org.onosproject.drivers (Default Drivers)

  • org.onosproject.protocols.grpc (gRPC Protocol Subsystem)

  • org.onosproject.protocols.p4runtime (P4Runtime Protocol Subsystem)

  • org.onosproject.p4runtime (P4Runtime Provider)

  • org.onosproject.drivers.p4runtime (P4Runtime Drivers)

  • org.onosproject.drivers.barefoot (Barefoot Drivers)

  • your own pipeconf app.

Build and push a configuration json

Having all the needed components in ONOS in place we can now tell ONOS about the device(s) and let the interaction begin.

First we need to create a .json file containing all the needed information such as IP/Port of the device, it's ports and the pipeconf we want to deploy.

Code Block
{
  "devices": {
    "device:<name>": {
      "generalprovider": {
        "p4runtime": {
          "ip": "<ip>",
          "port": "<port>",
          "deviceId": <id_of_device>
        }
      },
      "piPipeconf": {
        "piPipeconfId": "<name of your pipeconf>"
      },
      "ports": {
        <ports of 
      },
      "basic": {
        "name": "<freindly_name>",
        "latitude": 41,
        "driver": "tofino",
        "longitude": -107
      }
    }

A working example is in $ONOS_ROOT/tools/test/topos/tofino-demo-simple.json. Change the IP both in the DeviceId at the top and in the devices array. The port number by default on gRPC is 50051, so unless you made any changes to that leave it as is. 
Please not that you need to add the driver as "tofino" and the the piPipeconfId as the name of you pipeconf.

 Upload the configuration you just wrote to the instance of ONOS you are running:

Code Block
$ curl -X POST -H "content-type:application/json" http://localhost:8181/onos/v1/network/configuration -d @<path_to_your_json_configuration_file> --user onos:rocks

or 

Code Block
<your_machine>~$ onos-netcfg localhost <path_to_your_json_configuration_file>

This configuration gets picked up by the GeneralDeviceProvider.java that 

Check if the device is present in ONOS:

Code Block
onos> devices

Configure ports on the device

The last step to perform is to configure the ports on the switch.

Please repeat the following command for all the ports that you need on your device. This needs to be done after having pushed the pipeline config. 

Code Block
bf-sde> pm
bf-sde.pm> show
bf-sde.pm> port-add 1/0 100G NONE
bf-sde.pm> port-enb 1/0

Use

controlling Intel/Barefoot Tofino-based switches running Stratum.

Stratum is a software agent that runs on the switch and exposes gRPC-based interfaces such as P4Runtime, gNMI, and gNOI. These interfaces are used by ONOS to control and configure the switch.

The steps to follow are:

  1. Learn the basics of the Stratum-ONOS stack by doing the Next-Gen SDN Tutorial (based on Mininet and the BMv2 software switch)
  2. Learn to install and run the Stratum agent on your Tofino-based switch
  3. Learn how to deploy an existing ONOS pipeconf to your Tofino switch
  4. Use the acquired knowledge and code to create a pipeconf and ONOS apps that work with your own P4 program

1 - Learn the basics with the Next-Gen SDN Tutorial

Before experimenting with real hardware, and facing all the complexities associated with making things work for the first time, we strongly encourage you to understand the basic concepts of the Stratum-ONOS stack by doing this hands-on tutorial based on Mininet and the BMv2 software switch. Most of the provided code and instructions can be applied to the case of a network comprising real Tofino-based switches.

The tutorial is organized around a sequence of hands-on exercises that show how to build an IPv6-based leaf-spine data center fabric using P4, Stratum, and ONOS. It provides an introduction to concepts such as:

  • Data plane programming and control via P4 and P4Runtime
  • Device/port configuration via YANG, OpenConfig, and gNMI
  • Running and operating ONOS
  • Using ONOS to deploy an arbitrary P4 program to the network switches
  • Implementing ONOS apps that provide the control plane logic for the data plane P4 programs 

NG-SDN Tutorial repository

The tutorial instructions, along with slides, are available at this URL:

https://github.com/opennetworkinglab/ngsdn-tutorial

2 - Install and run Stratum on your Tofino-based switch

Stratum currently supports different Tofino-based switches, from different vendors. To learn if your switch supports Stratum, check out the main README of the GitHub Stratum repository:

https://github.com/stratum/stratum/blob/master/README.md

To install and run Stratum on a Tofino-based switch, use the following instructions:

https://github.com/stratum/stratum/tree/master/stratum/hal/bin/barefoot

Hint: consider using the proposed Docker-based approach to install and Stratum with minimal effort

3 - Learn how to deploy a Tofino-enabled pipeconf

From the NG-SDN tutorial, you should have learned that ONOS uses "pipeconfs" to deploy and manage a given P4 program on a device. Pipeconfs are distributed as ONOS applications, hence using the .oar packaging.

Most of the pipeconfs already available in ONOS, as well as that provided in the NG-SDN Tutorial, work only with the BMv2 software switch. To learn how to create and deploy pieconfs that can work with real Tofino-enabled switches, you can check out the fabric-tofino GitHub repository. fabric.p4 is an open-source P4 program distributed as part of ONOS, designed to work with Trellis, a set of SDN applications running on top of ONOS to provide the control plane for an IP fabric based on MPLS segment-routing.

The fabric-tofino GitHub repository provides instructions on how to:

  • Build a pipeconf .oar package for fabric.p4 that includes the output of the P4 compiler for Tofino (tofino.bin and context.json)
  • Install the pipeconf in a running ONOS instance
  • Tell ONOS to connect and to deploy the pipeconf to the Stratum agent running on the switch

fabric-tofino repository

Detailed Instructions are available in the main README of the fabric-tofino repository:

https://github.com/opencord/fabric-tofino

4 - Use the acquired knowledge and code

You should now know everything you need to:

  • Create a pipeconf package for your own P4 program (check the PipeconfLoader.java class from fabric-tofino for an example of how to include the tofino.bin and context.json)
  • Load the pipeconf in ONOS and deploy it to your switch (using the example netcfg.json provided in fabric-tofino)
  • Implement an ONOS app to manage the tables and other entities of your P4 program (using the app provided in the ng-sdn tutorial as an example)

Get help

If you need help, you can ask questions on the following mailing lists:

For issues concerning Tofino and other Intel/Barefoot products (e.g., P4 compiler errors), please reach out to Barefoot support channels.At this point if everything went smoothly you should be able to push rules defined with PiCriterion and PiInstruction according to the P4 program deployed on the device.