Due to a ransomware attack, the wiki was reverted to a July 2022 version. . We apologize for the lack of a more recent valid backup.
This section describes the subsystems responsible for ONOS's distributed functionalities. This includes information distribution and synchronization, cluster management, and device mastership management.
Overview
A multi-instance ONOS deployment is a cluster of one or more ONOS instances, or nodes, each with an unique NodeId. Each node in a cluster is capable of using local information gathered from its own services to generate events. These events are shared with all of the nodes in a cluster via distributed mechanisms implemented in various services' Stores.
In addition to data distribution, an ONOS cluster must:
- Detect and handle nodes joining and leaving the cluster
- Delegate control over devices, such that any given device has one primary controller
The first item is managed by the Cluster subsystem, and the second, by the Mastership subsystem. The remaining sections elaborate on the distributed store, and describe the function of these managers.
Distributed Stores
Depending on the requirements of a service, the contents of a store may be either strongly or eventually consistent; this is made possible by having each service's store implement the appropriate distribution mechanism. Currently, Hazelcast's distributed structures are used for strong consistency, and a gossip protocol implementation on Netty is used for eventual consistency. With the exception of cluster, mastership, and flow rule management, all services employ eventually consistent stores.
Cluster Management
We start by noting that ONOS also uses the term 'cluster' to refer to connected subgraphs of the network topology, which has no association to the cluster in the multi-instance sense. When clusters are mentioned in this section, they are strictly in terms of the latter.
Device Mastership Management
The mastership subsystem is responsible for assigning per-device roles to each node in a cluster. The three roles that a node can take with respect to a device are:
- NONE : The node may or may not have knowledge of the device, and cannot interact with it.
- STANDBY : The node has knowledge of the device, and can read, but not write to it.
- MASTER : the node has knowledge of the device, and have full read-write access to it.
A node can have different roles for different devices. The three roles map to the NONE, SLAVE, and MASTER roles specified by OpenFlow v1.2+, respectively.
A device is free to connect to one or more nodes in a cluster. For a device connected to multiple nodes, only one node is MASTER at any given time, and the rest are either STANDBY or NONE.