Due to a ransomware attack, the wiki was reverted to a July 2022 version. . We apologize for the lack of a more recent valid backup.
Introduction
This document provides information on how to install, configure and run SDN-IP in an SDN network.
It's recommended that you go through the tutorial and you read the architecture guide to become familiar with the concepts before attempting to set up the application.
Network model
SDN-IP allows an SDN network to peer and exchange traffic with adjacent external networks using the BGP routing protocol.
The following figure shows the various elements in the SDN-IP network model.
- The most fundamental building block is, of course, the SDN network. This is the network that is controlled by ONOS and will use SDN-IP to communicate with the outside world.
- The network is controlled by a cluster of ONOS instances (one or more).
- SDN-IP runs as an application on a subset of ONOS instances.
- External BGP routers are connected at the edge of the SDN network. These are routers belonging to other administrative domains that will peer with the SDN network through BGP. The SDN network needs to have direct (not routed) IP connectivity to each external router it needs to peer with.
- Inside the SDN network there are one or more BGP speakers. There are no specific requirements on the implementation of the BGP speakers - as long as they support both external BGP (eBGP) and internal BGP (iBGP) peering sessions. For our testing and deployments we use open-source software routers such as BIRD or Quagga. The internal BGP speakers peer in two different ways:
- Each BGP speaker must have at least one connection to the SDN data plane network. The BGP speaker will peer with external BGP routers over this connection using eBGP.
- Each BGP speaker also needs to peer with each SDN-IP instance using iBGP so it can relay routes to the SDN-IP instances. The connectivity for this peering must be out-of-band of the data plane.
BGP Peering Topology
There are no strict requirements on how the BGP topology should be set up, as long as each SDN-IP instance is able to receive all routes advertised to the SDN network through iBGP. In saying that, there are recommended deployment scenarios that we use and have tested thoroughly.
The following figure shows a BGP configuration example.
The icons show various nodes in the network and the lines show BGP peering sessions. Black lines are eBGP sessions between an internal BGP speaker and an external BGP router, and red lines are iBGP sessions amongst BGP speakers and SDN-IP instances.
Each external BGP router peers with one or more internal BGP speakers. SDN-IP does not currently support multihop BGP for external peering sessions, so each peering sessions must take place in the same subnet. Different peering sessions can be in different subnets however. In order to peer with external routers, the SDN network needs an IP address for each peering session. The IP addresses on the SDN network side are assigned to the BGP speakers so they can peer. It is not required that each internal BGP speaker peer with each external router, because the BGP speakers can redistribute routes between themselves using iBGP. However, for fully redundant peering with external networks, an external peer should have peering sessions with multiple internal BGP speakers. This can be seen in the figure where BGP Router 1 has a peering session to each internal BGP speaker.
Inside the network the BGP speakers peer with SDN-IP instances (running on ONOS) using iBGP. SDN-IP is a passive iBGP peer: it listens to BGP updates but it never advertises updates of its own. iBGP peering sessions within the network can be set up in multiple ways. The easiest way is to have a full mesh of iBGP peering sessions between the BGP speakers and the SDN-IP instances. This can be seen with the red iBGP peering sessions in the figure. In this way, all BGP nodes within the network can learn all the routes. Note it is not necessary (or possible) to have iBGP sessions between two SDN-IP instances, because the SDN-IP instances never advertise routes of their own.
A variety of examples of different iBGP topologies are shown in the following figure.
ONOS-1.1 note: A subset of BGP Multiprotocol Extensions Capabilities (RFC 4760) is implemented: IPv4 and IPv6 unicast routes (AFI/SAFI). However, the receiving and processing of IPv6 routes has been tested only over IPv4 BGP peering.
Configuration
SDN-IP currently uses a simple JSON file format to import configuration data. There are two separate files: addresses.json for IP and MAC address data, and sdnip.json for BGP peering session data.
Network configuration files
These configuration files need to be placed in the config directory.
If you plan to deploy ONOS and SDN-IP manually you need to copy the configuration file in the config directory on each instance. This is located at KARAF_ROOT/../config, and the path is not currently configurable.
Otherwise, when using the ONOS cell mechansim to deploy an ONOS cluster, the config files can be placed in the ONOS_ROOT/tools/package/config directory, on the management node you use to deploy. Then, they will be automatically be copied to the correct location on the cell instances when the cluster is deployed.
There are two simple JSON files for configuring the network:
addresses.jsonis used to configure IP and MAC addresses.sdnip.jsonis used to configure BGP peering sessions.
Addresses.json: configuring IP and MAC addresses
The addresses.json file is used to configure the addresses that ONOS uses to talk with the outside world. This configuration is mainly used for proxy ARP, so that the proxy ARP module knows how to respond to ARP requests coming from outside the network.
{
"addresses" : [
{
"dpid" : "00:00:00:00:00:00:00:a1",
"port" : "1",
"ips" : ["10.0.1.101/24"],
"mac" : "00:00:00:00:00:01"
},
{
"dpid" : "00:00:00:00:00:00:00:a2",
"port" : "1",
"ips" : ["10.0.2.101/24"],
"mac" : "00:00:00:00:00:01"
},
{
"dpid" : "00:00:00:00:00:00:00:a5",
"port" : "1",
"ips" : ["10.0.3.101/24"],
"mac" : "00:00:00:00:00:01"
},
{
"dpid" : "00:00:00:00:00:00:00:a6",
"port" : "1",
"ips" : ["10.0.4.101/24"],
"mac" : "00:00:00:00:00:01"
}
]
}
The file contains an addresses array which has multiple address entries. Each address entry describes a set of addresses which are bound to a port on a switch. Each entry contains the switch port the address is bound to (specified by DPID and port number), a set of IP addresses that are bound to the port, and a single MAC address to be used for ARP responses for each of the IP addresses.
When running SDN-IP, each address used by the SDN network to peer with an external router needs to be configured in the addresses.json file.
ONOS-1.1 note: Multiple IP addresses (including IPv6) can be configured per interface:
{
"addresses" : [
{
...
"ips" : ["10.0.1.101/24", "2001:DB8::1/64"],
...
},
...
]
}
BGP configuration
The sdnip.json file contains details of the BGP peering sessions, as well as the connectivity between internal BGP speakers and external BGP peers.
{
"bgpPeers" : [
{
"attachmentDpid" : "00:00:00:00:00:00:00:a1",
"attachmentPort" : "1",
"ipAddress" : "10.0.1.1"
},
{
"attachmentDpid" : "00:00:00:00:00:00:00:a2",
"attachmentPort" : "1",
"ipAddress" : "10.0.2.1"
},
{
"attachmentDpid" : "00:00:00:00:00:00:00:a5",
"attachmentPort" : "1",
"ipAddress" : "10.0.3.1"
},
{
"attachmentDpid" : "00:00:00:00:00:00:00:a6",
"attachmentPort" : "1",
"ipAddress" : "10.0.4.1"
}
],
"bgpSpeakers" : [
{
"name" : "bgp",
"attachmentDpid" : "00:00:00:00:00:00:00:a3",
"attachmentPort" : "1",
"macAddress" : "00:00:00:00:00:01",
"interfaceAddresses" : [
{
"interfaceDpid" : "00:00:00:00:00:00:00:a1",
"interfacePort" : "1",
"ipAddress" : "10.0.1.101"
},
{
"interfaceDpid" : "00:00:00:00:00:00:00:a2",
"interfacePort" : "1",
"ipAddress" : "10.0.2.101"
},
{
"interfaceDpid" : "00:00:00:00:00:00:00:a5",
"interfacePort" : "1",
"ipAddress" : "10.0.3.101"
},
{
"interfaceDpid" : "00:00:00:00:00:00:00:a6",
"interfacePort" : "1",
"ipAddress" : "10.0.4.101"
}
]
}
]
}
At the top level of the sdnip.json file are two lists: bgpPeers and bgpSpeakers.
The bgpPeers list contains an entry for every external peer that the SDN network peers with. Each peer has a peering address, and the attachment point where it is connected to the SDN network specified by DPID and port number.
The bgpSpeakers list contains an entry for each BGP speaker in the SDN network. The header for each BGP speaker includes an arbitrary name for the speaker, the attachment point where it is connected to the SDN network (specified by DPID and port number) and the MAC address of the BGP speaker. Then, each BGP speaker has a list of "interfaceAddresses", which are the external-facing interfaces this speaker has peering sessions on, and the IP address it uses for that peering session.
SDN-IP component configuration
In addition to the network configuration described above, there is a separate configuration file for the SDN-IP software component. Currently there is one configurable parameter, the port that SDN-IP listens for incoming BGP connections on. If you wish to configure this parameter, place this file at KARAF_ROOT/etc/org.onosproject.sdnip.SdnIp.cfg. Note that by default ONOS listens on TCP port number 2000 for incoming BGP connections, which is not the default BGP port number 179.
bgpPort=2000
ONOS-1.1 note: In ONOS-1.1 the above configuration should be placed in file KARAF_ROOT/etc/org.onosproject.routing.bgp.BgpSessionManager.cfg.
BGP speakers
Each BGP speaker potentially has multiple peering sessions to different external networks. It is possible for these peering sessions to be on different IP subnets, which means each BGP speaker can potentially have multiple IP addresses it is peering on. These addresses must be configured in the sdnip.json file, but they also need to be configured on the interfaces of the BGP speaker, so that it can receive packets sent to it on those addresses.
Running SDN-IP on ONOS
The first step is to get an ONOS system up and running, and you can read more about how to do this here: Installing and Running ONOS
Once ONOS is running, SDN-IP has a couple of additional dependencies that it relies on to ensure ARP requests are resolved properly.
onos-app-configis used to read in theaddresses.jsonconfiguration fileonos-app-proxyarpis the proxy ARP module that responds to ARP requests on behalf of hosts and external routers.
It's usually best to ensure these are loaded before starting SDN-IP, either by adding them to the ONOS_FEATURES, or by loading them manually:
onos> feature:install onos-app-config onos> feature:install onos-app-proxyarp
Once these dependencies are satisfied, the SDN-IP application can be installed:
onos> feature:install onos-app-sdnip
At this point, SDN-IP will start up, read its configuration files and install intents in the network to establish connectivity for the BGP peering sessions. Then it will begin to receive routes, which are installed into the network using MultiPointToSinglePoint intents.
SDN-IP CLI commands
SDN-IP includes CLI commands that allow a user to monitor the state of the system. Each command is detailed below.
bgp-neighbors [-j|--json] [-n|--neighbor <neighbor>]
Shows the iBGP neighbors that have connected to SDN-IP. Each neighbor is an internal BGP speaker. Can be limited to show a specific neighbor.
bgp-routes [-j|--json] [s|--summary]
Shows best routes received from BGP peers, including BGP specific information.
bgp-routes [-j|--json] [n|–neighbor <neighbor>]
Shows all routes received from a particular BGP neighbor (even if those are not the best routes).
routes [-j|--json] [s|--summary]
Shows the routing table of SDN-IP. This should be mostly the same as the BGP routes, but may include routes from other sources.
Troubleshooting
A full running SDN-IP system has a lot of pieces that need to be correct before anything will work. When setting up a system for the first time or when things go wrong, it's best to go step by step to check each piece is in place.
Check SDN-IP is installed:
onos> feature:list | grep sdnip onos-app-sdnip | 1.0.0 | x | onos-1.0.0 | SDN-IP peering application
The little x in the 3rd column means SDN-IP is installed. If it's not there, try install it again.
Check whether the external routers can resolve ARP properly. If they can't resolve ARP, then look in the addresses.json file and check that the address configuration is correct.
Check the BGP peering sessions are established with the internal BGP speakers. If the peering sessions are not established, it may be that the sdnip.json configuration is wrong. If it seems to be correct, check the configuration of the BGP peers.
Check the configuration files are in the correct directory.
Next, check that the internal BGP speakers are peering with ONOS correctly:
onos> bgp-neighbors
You should see a list of internal BGP speakers. If not, check that the connectivity is OK and the BGP configuration in the BGP speaker is correct.
Then, check if ONOS receives the expected routes:
onos> routes
If the routes are not received, check if the internal BGP speaker has received the routes and advertised them to ONOS.
In addition, you could use the following command to see BGP-specific information about the received routes:
onos> bgp-routes
Once the routes are received, ONOS will start to install MultiPointToSinglePoint intents for each route. You can check this is correct by looking at the number of MultiPointToSinglePoint intents in the intent summary. It should be the same as the number of routes, and all intents should move to the INSTALLED state before long.
onos> intents -s
Once intents are installed, there should be transit connectivity through the SDN network.
Questions? Comments? Problems?
Please let us know via the onos-deploy mailing list. Details about ONOS mailing lists can be found here: ONOS Mailing Lists
We really welcome your help to ensure the code and documentation are clear and informative, so let us know if you have and issues.
3 Comments
Larkland Morley
This would be a very useful use case if Host within the SDN network can communicate to the outside through these routers. Is there any plans to have this soon?
Luca Prete
Hi Larlkland,
Sure! It's in the plan for the February release. We understood it's an important limitation. You can see more details on future release at https://wiki.onosproject.org/display/ONOS/Upcoming+Release+Contents
Larkland Morley
I will be playing with the software soon. Does it currently support virtual router capabilities within the SDN network? Meaning L2/L3 Forwarding.