P4 Tutorial VM (http://onlab.vicci.org/onos/onos-p4-dev.ova)
This VM includes:
Mininet 2.3.0b1
ONOS 1.13-SNAPSHOT
Bmv2 P4 simulator with P4Runtime support
Trellis mininet topology(https://github.com/opennetworkinglab/routing/)
Reference: https://wiki.opencord.org/pages/viewpage.action?pageId=3014916
In Trellis, we run every switch as router, each router needs a mac address and multiple interface configurations.
Segment routing device setting:
"Device-id" : { "segmentrouting" : { "name" : "Device name", "ipv4NodeSid" : 123, "ipv4Loopback" : "192.168.0.123", "ipv6NodeSid" : 223, "ipv6Loopback" : "2000::c0a8:0223", "routerMac" : "00:00:00:00:01:23", "isEdgeRouter" : true, "adjacencySids" : [] } } |
In segment routing config:
Name: name for segment routing device
ipv4(6)NodeSid: global unique id for IP loopback, which used for MPLS lable in forwarding
ipv4(6)Loopback: Router loopback IP address, should not be part of same same subnet defined on dataplace interface
routerMac: Globally unique mac address this mac address will be used to reply ARP request for router IP or interface IP
isEdgeRouter: true if this device is edge/leaf, otherwise false
adjacencySids: Reserved, put empty array for now
Interface config:
"Device-id/port-number" : { "interfaces" : [ { "name": "interface name", "ips" : [ "10.0.1.254/24" ], "vlan-untagged": 10 } ] } |
In interface config:
Name: interface name, can be empty or unset
ips: IP address for this interface, can be the default gateway of the host
VLAN config:
vlan-untagged (integer): this interface consumes untagged packet only, and this interface belongs to specific vlan
vlan-tagged (integer array): this interface can consumes specific tagged vlans
vlan-native (integer): if an untagged consumes by this port, and this is an interface with vlan-tagged setup, device will use this vlan for internal use
vlan config can be: vlan-untagged only, vlan-tagged only or vlan-tagged + vlan-native
In demo mininet topology, we set IP from interface as default gateway of hosts
In this tutorial, we need to create 3 terminal window for:
ONOS karaf process
ONOS shell
Mininet emulator
And we run everything inside the virtual machine
Start ONOS with following command
$ cd ~/onos $ ok clean |
Use another terminal, connect to ONOS shell and activate follow apps:
$ onos localhost onos> app activate bmv2 fabric hostprovider lldpprovider proxyarp segmentrouting onos> apps -a -s * 9 org.onosproject.drivers 1.13.0.SNAPSHOT Default Drivers * 35 org.onosproject.generaldeviceprovider 1.13.0.SNAPSHOT General Device Provider * 36 org.onosproject.protocols.grpc 1.13.0.SNAPSHOT gRPC Protocol Subsystem * 37 org.onosproject.protocols.p4runtime 1.13.0.SNAPSHOT P4Runtime Protocol Subsystem * 38 org.onosproject.p4runtime 1.13.0.SNAPSHOT P4Runtime Provider * 39 org.onosproject.drivers.p4runtime 1.13.0.SNAPSHOT P4Runtime Drivers * 42 org.onosproject.proxyarp 1.13.0.SNAPSHOT Proxy ARP/NDP * 44 org.onosproject.hostprovider 1.13.0.SNAPSHOT Host Location Provider * 45 org.onosproject.lldpprovider 1.13.0.SNAPSHOT LLDP Link Provider * 73 org.onosproject.pipelines.basic 1.13.0.SNAPSHOT Basic Pipelines * 84 org.onosproject.route-service 1.13.0.SNAPSHOT Route Service Server * 95 org.onosproject.protocols.gnmi 1.13.0.SNAPSHOT gNMI Protocol Subsystem * 96 org.onosproject.drivers.gnmi 1.13.0.SNAPSHOT gNMI Drivers * 108 org.onosproject.pipelines.fabric 1.13.0.SNAPSHOT Fabric Pipeline * 120 org.onosproject.segmentrouting 1.13.0.SNAPSHOT Segment Routing * 121 org.onosproject.drivers.bmv2 1.13.0.SNAPSHOT BMv2 Drivers |
Push network configuration from another new terminal window
$ cd ~/routing/trellis $ onos-netcfg localhost trellisp4.json |
(Optional) Decrease flow and group polling time
ONOS use polling mechanism to sync table entries from P4 devices
We can manually setup polling frequency in ONOS shell so ONOS can poll flow entries and groups from devices more frequently. (default: 30 seconds)
onos> cfg set org.onosproject.net.flow.impl.FlowRuleManager fallbackFlowPollFrequency 5 onos> cfg set org.onosproject.net.group.impl.GroupManager fallbackGroupPollFrequency 5 |
Strat mininet topology on new terminal using trellisp4.py python script
trellisp4.py creates:
2 by 2 leaf-spine topology using 4 bmv2 devices
4 bidirectional links between leaf and spine
4 hosts, 2 hosts per leaf device
$ sudo -E ./trellisp4.py --onos-ip 127.0.0.1 |
After started, ONOS should discover 4 devices and 8 links
onos> devices -s id=device:bmv2:204, available=true, role=MASTER, type=SWITCH, driver=bmv2:org.onosproject.pipelines.fabric id=device:bmv2:205, available=true, role=MASTER, type=SWITCH, driver=bmv2:org.onosproject.pipelines.fabric id=device:bmv2:226, available=true, role=MASTER, type=SWITCH, driver=bmv2:org.onosproject.pipelines.fabric id=device:bmv2:227, available=true, role=MASTER, type=SWITCH, driver=bmv2:org.onosproject.pipelines.fabric onos> links src=device:bmv2:204/1, dst=device:bmv2:226/1, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:204/2, dst=device:bmv2:227/1, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:205/1, dst=device:bmv2:226/2, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:205/2, dst=device:bmv2:227/2, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:226/1, dst=device:bmv2:204/1, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:226/2, dst=device:bmv2:205/1, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:227/1, dst=device:bmv2:204/2, type=DIRECT, state=ACTIVE, expected=false src=device:bmv2:227/2, dst=device:bmv2:205/2, type=DIRECT, state=ACTIVE, expected=false |
Hosts should be learnt by ONOS
onos> hosts id=00:AA:00:00:00:01/None, mac=00:AA:00:00:00:01, locations=[device:bmv2:204/3], vlan=None, ip(s)=[10.0.2.1], provider=of:org.onosproject.provider.host, configured=false id=00:AA:00:00:00:02/None, mac=00:AA:00:00:00:02, locations=[device:bmv2:204/4], vlan=None, ip(s)=[10.0.2.2], provider=of:org.onosproject.provider.host, configured=false id=00:AA:00:00:00:03/None, mac=00:AA:00:00:00:03, locations=[device:bmv2:205/3], vlan=None, ip(s)=[10.0.3.1], provider=of:org.onosproject.provider.host, configured=false id=00:AA:00:00:00:04/None, mac=00:AA:00:00:00:04, locations=[device:bmv2:205/4], vlan=None, ip(s)=[10.0.3.2], provider=of:org.onosproject.provider.host, configured=false |
If any host was not discovered by ONOS, try send arp request by using arping tool from mininet shell:
mininet> h1 arping 10.0.2.254 |
Routing and bridging rule should be installed by ONOS after host been detected and hosts should be able to ping each other:
mininet> pingallfull <...skip debug logs...> *** Results: h1->h2: 1/1, rtt min/avg/max/mdev 1.267/1.267/1.267/0.000 ms h1->h3: 1/1, rtt min/avg/max/mdev 3.384/3.384/3.384/0.000 ms h1->h4: 1/1, rtt min/avg/max/mdev 4.708/4.708/4.708/0.000 ms h2->h1: 1/1, rtt min/avg/max/mdev 1.381/1.381/1.381/0.000 ms h2->h3: 1/1, rtt min/avg/max/mdev 3.138/3.138/3.138/0.000 ms h2->h4: 1/1, rtt min/avg/max/mdev 3.983/3.983/3.983/0.000 ms h3->h1: 1/1, rtt min/avg/max/mdev 3.446/3.446/3.446/0.000 ms h3->h2: 1/1, rtt min/avg/max/mdev 3.315/3.315/3.315/0.000 ms h3->h4: 1/1, rtt min/avg/max/mdev 1.388/1.388/1.388/0.000 ms h4->h1: 1/1, rtt min/avg/max/mdev 3.345/3.345/3.345/0.000 ms h4->h2: 1/1, rtt min/avg/max/mdev 4.479/4.479/4.479/0.000 ms h4->h3: 1/1, rtt min/avg/max/mdev 1.302/1.302/1.302/0.000 ms mininet> |