Due to a ransomware attack, the wiki was reverted to a July 2022 version. . We apologize for the lack of a more recent valid backup.
Objective:
This test is designed to characterize ONOS' flow subsystem, particularly flow installation burst rate performance in various "East-West", "North-South" communication patterns. For a distributed system to realize scale-out benefit, there would exist some NS throughput limits that can be overcome by adding nodes which can utilize a high performance EW cluster communication.
Method:
In this test, a python utility "flow-tester.py" is used to start a flow generator internal of ONOS. This test script will cause ONOS to install a set of flows and return a response time when all flows are successfully installed. Burst rates with different run parameters are calculated as total flows installed over the response time. The "flow-tester.py" utility has a few "nobs" to adjust different test scenarios, see the script help command for details. The basic idea is to characterize ONOS under various of "North-South" and "East-West" communication patterns in the flow subsystem.
Refer to the following diagram. The key "nobs" for exercising those two dimensions of communications are: 1)N, the number of neighboring nodes, over which flow rules generated from a server are sent - due to flows belonging to switches whose masterships are not local to the generating server; 2)S, the number of servers installing flows, i.e. servers generating flow rules. In one extreme case, all flows installed from the script target to devices (switches) whose mastership is the ONOS node generating flows. In this case, all flows are "NS" bounded, and installed locally through the serving ONOS. There is no need for flows to be sent from one ONOS node to another - eliminating EW communication overhead. In another extreme case, flows installed from one ONOS node are spread to all devices on all nodes in the cluster, highly utilizing the EW communication before the flow can be installed locally on each ONOS node.
In our current test methodology, consider ONOS cluster as a system in whole with NB and SB interfacing the loading system. In order to make performance comparisons, we fix the loading to this clustered system, i.e. total number of flows installed into the cluster to be a constant; the number of null devices connected to this cluster is also a constant - with masterships "evenly" distributed to all active ONOS nodes.
The test is to iterate on each N while the response time is measured with scale of the cluster changes from 1 to 3, 5, 7 nodes. Multiple measurements are taken to obtain statistical mean and standard deviation. In this way we will see the scale-out performance with a specific neighboring scenario. Starting the experiment with N=0, this scenario should give us characteristics of the cluster with mostly NS-intensive operations. When N=6, the experiment will exercise the "worst" case scenario with EW operations. With N in between 0 and 6, the result will be a mix of the two extreme scenarios. In a "combinational" scenario, where (N+1) = S, or the cluster size - all active nodes, are generating flow rules and receiving flows from all other nodes.
Diagram showing how "flow-tester.py" script causes flow rules being generated, routed for local and remote neighbor nodes to install.
Procedure:
The basic test is to get the returned response time from the "flow-tester.py" test command. The following steps can be used as a reference to run the test:
1) Based on the desired (S, N, SW) combination, configure the appropriate Null Link Provider and Null Device Provider config files accordingly;
2) Package the config file with ONOS before installing to all relevant ONOS nodes;
3) Run iterations of the flow-tester.py commands in order to obtain statistical results.