Design Decisions:

• Group Identifier will be represented as a simple Integer. Will be revisited at a later point of time to see if it needs to be aligned with PortNumber representation
  
• Group objects for a device will only be created/updated/removed by current Master controller instance.
• So as, the Group Identifier allocation/management also will only be done by current Master controller instance.
• Replication:
  • Alternative1: Groups for a device are only replicated in the Master controller instance and another potential candidate instance. 
  • Alternative2: Since typically the number Groups per device will not be huge, Groups are replicated in all the instances of the cluster
  • Resolution: Though the Groups are created only by Master controller instance, they may typically be required/used by all controller instances in the cluster. So it is recommended to replicate the Group database in all the instances of the cluster. 
• Consistency & Durability:
  • Eventual consistency or Strong consistency for Group Objects?
  • Do we need durability?
  • Resolution: Eventual consistency seems to be sufficient for Group store. Durability is not needed.
• Group creation/update/removal/query request can be submitted in any controller instance and that instance will take care of routing the request to current Master controller instance.
• Group creation API does not return any Group object as in some scenarios the request needs to be routed to Master controller instance where the Group object gets created. Application should use Group Query API to retrieve the Group object after creation.
• Group query API blocks the caller thread until it retrieves from the remote Master controller instance if the instance where request is submitted is not having the control over that device 
• Group query API returns the Group object even if the Group object is in PENDING_ADD state
• Group subsystem generates GROUP_ADDED event only after it receives confirmation from Data plane through Barrier Reply or Group Description Reply (In the first phase, it will be based on Group Description Reply)
• Authoritativeness:
  • Group store has full authority of groups in the device.
  • Group store will wipe off any extraneous groups in a device if found during AUDIT. That means creation of groups bypassing the Group store or any default groups existing in a device can not be supported
• Group Key
  • As part of the Group operations, Group subsystem accepts a cookie from the applications in the form of a GroupKey. This is to facilitate applications to query the Group sub system based on cookie instead of Group Identifier.

Group Subsystem Design:

High level layout of Group subsystem components is depicted below.

Class Diagram

Group Service

• Provides the following API to applications

  • void CreateGroup(Device, GroupType, Collection<GroupBuckets>, GroupKey, appid)

  • Group GetGroup(Device, GroupKey)

  • void AddBucketsToGroup(Device, OldGroupKey, Collection<GroupBuckets>, NewGroupKey, appid)

  • void RemoveBucketsFromGroup(Device, OldGroupKey, Collection<GroupBuckets>, NewGroupKey, appid)

  • void RemoveGroup(Device, GroupKey, appid)
  • GetAllGroups(Device, appid)

  • GetAllGroupStatistics(Device, appid)
    
    

• DataModel

  

  • Group Types to be supported

    • Select

    • Indirect

    • All

    • Failover

  • Key

    • An application specific cookie that supports Hash and Equals method

  • Id

    • Integer

  • Bucket

    • One or more collection of Traffic Instructions

{
    Collection<TrafficTreatment> groupBuckets = new ArrayList<TrafficTreatment>();
	List<PortNumber> outPorts = getPortsToDevice(srcDevice, neighborDevice);

    for (PortNumber portNumber: outPorts) {
        TrafficTreatment.Builder tBuilder = DefaultTrafficTreatment.builder();
        tBuilder.setOutput(portNumber)
                .setEthDst(getRouterMacAddress(neighborDevice))
                .setEthSrc(getRouterMacAddress(srcDevice))
                .pushMpls()
                .setMpls(getMplsId(finalDstDevice));
        groupBuckets.add(tBuilder.build());
    }

	GroupKey gorupKey = SegmentRoutingGroupKey.build(srcDevice, Collection<NeighborDevice>, dstDevice);
    Future<Group> future = groupService.createGroup(srcDevice, Group.Type.SELECT,
            groupBuckets, groupKey, getAppId());
    Group group = future.get();
}
 
public void SetIpRule() {
....
	GroupKey gorupKey = SegmentRoutingGroupKey.build(srcDevice, Collection<NeighborDevice>, dstDevice);
	Group group = groupService.getGroup(srcDevice, groupKey);
	tbuilder.setGroup(group.id());
....
 
}

Group Manager

This component implements the Group Service interface by defining Group create/modify/remove/query operations on devices.

If the Master for the device is local instance, then the operations are performed locally. If the device is not under the current controller instance, this component send GROUP_ADD_REQUEST to the master controller instance of that device.

Maintains monotonically increasing Group ID number space for each device that will be incremented every time a new group is created.

Creates the groups only if the device doesn’t have those groups populated. If group already exists, the create APIs doesn't perform any operation.

Group AUDIT

As shown in the below figure, Group Manager perform a AUDIT whenever a device connects to current controller instance as Master. During the Group AUDIT, no group operations can be supported.

Group Store

Group Store handles the distribution functionality of Group subsystem. Based on the design decision, the Group Store replicates the Groups to either all instances or to a subset of instances. Similarly based on the consistency and durability needs, the type of the data store to be used for Group store would be determined.

Group Store will have authoritative role. i.e. When a Device is connected to the controller as a Master, Group Store will wipe off all extraneous groups in the device and inserts if any groups are missing.

High level flow of events in the Group subsystem is depicted below when an operation like CreateGroup is submitted by the application.

High level flow of events in the Group subsystem is depicted below when an operation like CreateGroup is submitted by the application to non-Master controller instance in a multi-instance environment.

Multi-instance Support

TODO

Group Provider

• Provides SB APIs towards core

  • CreateGroup(GroupId, Buckets)

  • ModifyGroup(GroupId, Buckets)

  • RemoveGroup(GroupId)

  • GetGroups()

OF Group Provider

• Implements Group Providers interface

• Builds OF GROUP_MOD (ADD/MOD/DELETE) Message depending on the API invocation and writes the message to switch driver

OF Switch Driver

• No changes