The workflow is recommended for developers who wish to use cells and the stc / onos-* utility scripts to manage an ONOS development or testing cluster using multiple VMs or hardware servers.
- Make sure you have read the Abstraction and Definitions section in the Administrator Guide on this wiki
- ONOS, being an SDN controller authored in Java, can run on a variety of platforms. However, in the interest of focus, the ONOS team engages primarily in testing on Ubuntu server distributions, specifically Ubuntu Server LTS 64-bit. Commands above are related
This page describes a mechanism for setting up a virtual test environment across multiple VMs or hardware servers.
If you simply want to do ONOS development with a virtual multi-node ONOS cluster, check out Mininet and onos.py workflow.
This page assumes that the reader has been able to work through Installing and running ONOS.
Testing and managing a distributed platform like ONOS can become a cumbersome task.
In order to simplify testing and to make it more repeatable, a number of assets, including the aforementioned ONOS scripts, have been developed to make the developer’s and tester’s lives easier. These are located under $ONOS_ROOT/tools/test/. Appendix A of this guide provides a listing of available utilities. From your management (in this case development) machine, with few commands you'll be able to:
- Deploy ONOS as a cluster on one or multiple remote target machines
- Deploy an arbitrary Mininet topology on a remote machine and let its the switches connect to the ONOS machines mentioned above
- Test that the ONOS cluster and the Mininet topology came up correctly (i.e. check automatically ONOS logs, number of switches, links, hosts, ...)
Test Environment Components
Following the model reported in the Abstractions and Definitions section, an ONOS developer's environment may include the following:
- Management/Development/build machine: for code development - i.e. running the IDE and building/packaging/deploying ONOS.
- One or more ONOS target machines: for running ONOS instances
- A Mininet target machine: for emulating networks to test with ONOS
In the above case, the ONOS scripts are run from the management machine, and directed towards the target machines. The target machines (both ONOS and Mininet target machines) can be either local or remote, physical, virtual, ...
Target machines requirements
The following requirements should be satisfied in order to be able to push software configurations on the remote target machines.
All target machines:
- Be all-to-all reachable at IP layer (basically all machines -including the management machine- need to be able to ping one each other)
- Have a user sdn that can be sudoer on the machine, without any need to insert the password for it
- Accessing without password from the management machine to the target machines. To do this the public key of the user on the management machine must be present in the .ssh/authorized_key file on all your target machines.
This can be done in two ways: manually or (almost) automatically using an ONOS script
- As a proof that everything works, from your management machine you should be able to login into any target machine doing ssh sdn@TARGET_MACHINE_IP and execute sudo echo hello (for example) with no passwords required
ONOS target machines
Additionally, ONOS target machines should have
- Oracle Java (at least JRE) 1.8 installed. The ONOS installation section, under the Administration Guide, provides examples for Ubuntu about how to do that.
- curl installed
- make sure the following ports are open:
8181 for REST API and GUI
8101 to access the ONOS CLI
9876 for intra-cluster communication (communication between target machines)
6653 optional, for OpenFlow
6640 optional, for OVSDB
Mininet target machine
Additionally, the Mininet target machine should have
- Mininet install. This can be simple as typing sudo apt-get install mininet. Anyway, for more, and more detailed instructions, please refer directly to the Mininet website
Some test scenario, such as smoke, require external libraries. (e.g. requests) If you see errors like "ImportError: No module named requests" install them on the management machine.
Since a developer may want to test different scenarios against different sets of VMs or servers, ONOS provides the notion of test cells.
A test cell refers to a specific target machines environment designated for testing. Cell definition files are bash snippets that simply export few environment variables into the developer’s (or tester’s) shell. These variables are then used by a number of the ONOS test and utility scripts. This allows the scripts and test scenarios to be independent to a specific test bench setup.
The cell and cells commands are available on the management machine, where the ONOS repository has been originally cloned.
Listing existing cells
Cells configured can be listed using the command cells.
Cell definitions are loaded into the shell environment with the
cell utility. This utility takes the name of a cell definition file as the argument. If the command is invoked without parameters, it will just print out the cell in use.
Cell files attributes
Below it's reported a list of the basic attributes you can express and find in a cell file.
- OCI : the default target machine node IP. This is an alias for OC1.
- OC[1-N] : the IP addresses of the ONOS target machines. You can set as many OC instances as you want (depending on the number of ONOS target machines you want to run)
- OCN : the IP address of the Mininet target machine
- ONOS_FEATURES : a comma-separated list of bundle names, loaded at startup by an ONOS instances within this cell
- ONOS_NIC : The address block used amongst ONOS instances for inter-controller (clustering) and OpenFlow communication. In the example above, it's assumed that all ONOS instances have an IP address in the subnet 192.168.56.0/24
Once a cell definition is exported, utilities such as
onos will fall back to using the value set in
OCI (192.168.56.101 for above) if not given any parameters.
The utility script
cells can be used to view all available cell definitions. Without parameters,
cell will list the values associated with the current cell in us
Defining cell files
Cell files can be found on your management machine, under $ONOS_ROOT/tools/test/cells, where $ONOS_ROOT is the onos folder containing the ONOS repository files cloned.
When a file is added to this directory, it automatically enables it to be loaded and liste
d using the cell and cells commands
A cell definition file defines values for the aforementioned environment variables. A cell definition file may look like the following:
Using environment variables instead of IPs
Once your cell file is loaded you can use the OCX name instead, instead of the IP of a certain machine. So, for example you may want to do ssh sdn@$OC1 instead of ssh email@example.com
ONOS 1.1.0 and later comes with vicell command to ease editing and applying cell definition files. See vicell -h for usage details.
Optionally, ONOS test scripts can automatically deploy also a custom Mininet topology for you on a remote Mininet target machine. Custom topologies should be defined in $ONOS_ROOT/tools/test/topos.
Listing existing topologies
To view the list of the existing topologies with their descriptions, you can use the command topos. An asterisk may indicate what topology is currently in use. An example is reported below.
Topologies can be load with the command topo $TOPO_NAME. The same command without arguments shows the details about the topology loaded.
Creating custom topologies
The folder already contains lots of examples that may help you to start to write your first own topology. A topology may include up to 3 files, by convention called the same way, but using different extensions:
- The Topology recipe file (.recipe). The topology recipe file lets you define a new topology. This is mandatory, in order to create a new topology. Optionally, it lets you also automatically check (after the virtual network has been deployed) if the the expected number of devices, links and hosts are set. The structure of this file is very simple. An commented example is reported below.
- The Mininet Python topology file, (.py) that expresses how the Mininet topology should look like. The file is mandatory if you want to deploy a Mininet topology as well (to create new topologies look at the existing examples shipped with ONOS and/or go on the Mininet website)
- The ONOS network configuration file (.json), that lets you specify an ONOS. You can refer to this guide to learn more about the ONOS network configuration.
Deploy ONOS and the topology STC
The STC command lets you deploy multiple ONOS instances, the Mininet topology and test the environment.
Once your cell file is loaded:
More STC scripts
STC scripts can do really much more than this. If you're interested to know more check out this page!
Putting everything together: using the test environment
Let's wrap up! The beauty of the cell system is that it will give you the flexibility to test different ONOS configurations on different Mininet topologies, on different remote machines (regardless they are physical servers, virtual machines, containers, ...).
A typical workflow may look like the following:
- Make changes to the code
- Build ONOS with onos-buck buld onos (more details here)
- Load the cell settings with
- Load your topology with topo $YOUR_TOPO_NAME
- Deploy ONOS target machines with stc setup
- Deploy the Mininet topology with stc net-setup
The correspondent one line command and the related output is reported below:
Other useful commands
Attach to the remote ONOS machine OCN
Start/Stop/Restart/Check the status of the ONOS service
Kill the ONOS process