Work in progress...
Prerequsite
You will need:
- An ONOS cluster installed and running
- An OpenStack service installed and running ("stable/mitaka" version is used here)
OpenStack Setup
How to deploy OpenStack is out of scope of this documentation. Here, it only describes some configurations related to use SONA. All other settings are completely up to your environment.
Controller Node
Let's install networking-onos (Neutron ML2 plugin for ONOS) first.
Code Block | ||
---|---|---|
| ||
$ git clone https://github.com/openstack/networking-onos.git $ cd networking-onos $ sudo python setup.py install |
Specify ONOS access information. You may want to copy the config file to /etc/neutron/plugins/ml2/ where the other Neutron configuration files are.
Code Block | ||||
---|---|---|---|---|
| ||||
# Configuration options for ONOS ML2 Mechanism driver [onos] # (StrOpt) ONOS ReST interface URL. This is a mandatory field. url_path = http://onos.instance.ip.addr:8181/onos/openstackswitching # (StrOpt) Username for authentication. This is a mandatory field. username = onos # (StrOpt) Password for authentication. This is a mandatory field. password = rocks |
Next step is to install and run OpenStack services.
For DevStack users, use this sample DevStack local.conf to build OpenStack controller node. Make sure your DevStack branch is consistent with the OpenStack branches, "stable/mitaka" in this example.
Code Block | ||
---|---|---|
| ||
[[local|localrc]] HOST_IP=10.134.231.28 SERVICE_HOST=10.134.231.28 RABBIT_HOST=10.134.231.28 DATABASE_HOST=10.134.231.28 Q_HOST=10.134.231.28 ADMIN_PASSWORD=nova DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=$ADMIN_PASSWORD DATABASE_TYPE=mysql # Log SCREEN_LOGDIR=/opt/stack/logs/screen # Images IMAGE_URLS="http://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64.tar.gz,http://www.planet-lab.org/cord/trusty-server-multi-nic.img" FORCE_CONFIG_DRIVE=True # Networks Q_ML2_TENANT_NETWORK_TYPE=vxlan Q_ML2_PLUGIN_MECHANISM_DRIVERS=onos_ml2 Q_PLUGIN_EXTRA_CONF_PATH=/opt/stack/networking-onos/etc Q_PLUGIN_EXTRA_CONF_FILES=(conf_onos.ini) NEUTRON_CREATE_INITIAL_NETWORKS=False # Services enable_service q-svc disable_service n-net disable_service n-cpu disable_service tempest disable_service c-sch disable_service c-api disable_service c-vol # Branches GLANCE_BRANCH=stable/mitaka HORIZON_BRANCH=stable/mitaka KEYSTONE_BRANCH=stable/mitaka NEUTRON_BRANCH=stable/mitaka NOVA_BRANCH=stable/mitaka |
If you use other deploy tools or build the controller node manually, please set the following configurations to Nova and Neutron configuration files.
1. Set Neutron to use ONOS ML2 plugin.
Code Block | ||||
---|---|---|---|---|
| ||||
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin dhcp_agent_notification = False |
Code Block | ||
---|---|---|
| ||
[ml2] tenant_network_types = vxlan type_drivers = vxlan mechanism_drivers = onos_ml2 [securitygroup] enable_security_group = True |
2. Set Nova to use config drive for metadata service, so that we don't need to launch Neutron metadata-agent.
And of course, set to use Neutron for network service.
Code Block | ||
---|---|---|
| ||
[DEFAULT] force_config_drive = True network_api_class = nova.network.neutronv2.api.API security_group_api = neutron [neutron] url = http://[controller_ip]:9696 auth_strategy = keystone admin_auth_url = http://[controller_ip]:35357/v2.0 admin_tenant_name = service admin_username = neutron admin_password = [admin passwd] |
3. Don't forget to specify conf_onos.ini when you start Neutron service.
Code Block |
---|
/usr/bin/python /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /opt/stack/networking-onos/etc/conf_onos.ini |
Compute node
No special configurations are required for compute node other than setting network api to Neutron.
For DevStack users, here's sample DevStack local.conf.
Code Block | ||
---|---|---|
| ||
[[local|localrc]] HOST_IP=10.134.231.30 <-- local IP SERVICE_HOST=162.243.x.x <-- controller IP, must be reachable from your test browser for console access from Horizon RABBIT_HOST=10.134.231.28 DATABASE_HOST=10.134.231.28 ADMIN_PASSWORD=nova DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=$ADMIN_PASSWORD DATABASE_TYPE=mysql NOVA_VNC_ENABLED=True VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP VNCSERVER_LISTEN=$HOST_IP LIBVIRT_TYPE=kvm # Log SCREEN_LOGDIR=/opt/stack/logs/screen # Services ENABLED_SERVICES=n-cpu,neutron # Branches NOVA_BRANCH=stable/mitaka KEYSTONE_BRANCH=stable/mitaka NEUTRON_BRANCH=stable/mitaka |
Note |
---|
If your compute node is a VM, try http://docs.openstack.org/developer/devstack/guides/devstack-with-nested-kvm.html this first or set |
For manual set ups, set Nova to use Neutron as a network API.
Code Block | ||
---|---|---|
| ||
[DEFAULT] force_config_drive = always network_api_class = nova.network.neutronv2.api.API security_group_api = neutron [neutron] url = http://[controller_ip]:9696 auth_strategy = keystone admin_auth_url = http://[controller_ip]:35357/v2.0 admin_tenant_name = service admin_username = neutron admin_password = [admin passwd] |
Gateway node
No OpenStack service needs to be running on gateway nodes.
Additional compute and gateway node setup
1. Make sure your OVS version is 2.3.0 or later.
2. Set OVSDB listening mode in your compute nodes. There are two ways.
Code Block |
---|
$ ovs-appctl -t ovsdb-server ovsdb-server/add-remote ptcp:6640:host_ip |
Or you can make the setting permanent by adding the following line to /usr/share/openvswitch/scripts/ovs-ctl, right after "set ovsdb-server "$DB_FILE" line. You need to restart the openvswitch-switch service.
Code Block |
---|
set "$@" --remote=ptcp:6640 |
In either way, you should be able to see port "6640" is in listening state.
Code Block | ||
---|---|---|
| ||
$ netstat -ntl Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:6640 0.0.0.0:* LISTEN tcp6 0 0 :::22 |
3. Check your OVSDB. It is okay If there's a bridge with name "br-int", but note that SONA will add or update its controller, DPID, and fail mode.
Code Block |
---|
$ sudo ovs-vsctl show cedbbc0a-f9a4-4d30-a3ff-ef9afa813efb ovs_version: "2.3.0" |
TODO: Add additional gateway node setups if required
ONOS Setup
How to deploy ONOS is out of scope of this documentation.
1. If your ONOS is ready, create network-cfg.json file to configure SONA apps.
Code Block | ||
---|---|---|
| ||
{ "userDefined" : { "openstacknetworking" : { "config" : { "physicalRouterMac" : "00:00:00:00:00:01", "gatewayBridgeId" : "of:00000000000000a3", "gatewayExternalInterfaceName" : "veth1", "gatewayExternalInterfaceMac" : "00:00:00:00:00:01", "nodes" : [ { "dataPlaneIp" : "10.134.34.222", "bridgeId" : "of:00000000000000a1" }, { "dataPlaneIp" : "10.134.34.223", "bridgeId" : "of:00000000000000a2" }, { "dataPlaneIp" : "10.134.33.208", "bridgeId" : "of:00000000000000a3" } ] } } }, "apps" : { "org.onosproject.openstackinterface" : { "openstackinterface" : { "neutronServer" : "http://10.243.139.46:9696/v2.0/", "keystoneServer" : "http://10.243.139.46:5000/v2.0/", "userName" : "admin", "password" : "nova" } }, "org.onosproject.openstacknode" : { "openstacknode" : { "nodes" : [ { "hostname" : "compute-01", "ovsdbIp" : "192.168.56.112", "ovsdbPort" : "6640", "bridgeId" : "of:00000000000000a1", "openstackNodeType" : "COMPUTENODE" }, { "hostname" : "compute-02", "ovsdbIp" : "192.168.56.106", "ovsdbPort" : "6640", "bridgeId" : "of:00000000000000a2", "openstackNodeType" : "COMPUTENODE" }, { "hostname" : "network", "ovsdbIp" : "192.168.56.108", "ovsdbPort" : "6640", "bridgeId" : "of:00000000000000a3", "openstackNodeType" : "GATEWAYNODE", "gatewayExternalInterfaceName" : "eth1", "gatewayExternalInterfaceMac" : "00:00:00:00:00:10" } ] } } }, "devices" : { "of:00000000000000a1" : { "basic" : { "driver" : "sona" } }, "of:00000000000000a2" : { "basic" : { "driver" : "sona" } } } } |
NOT MERGED YET: will be updated to below
Code Block | ||
---|---|---|
| ||
{ "apps" : { "org.onosproject.openstackinterface" : { "openstackinterface" : { "neutronServer" : "http://10.243.139.46:9696/v2.0/", "keystoneServer" : "http://10.243.139.46:5000/v2.0/", "userName" : "admin", "password" : "nova" } }, "org.onosproject.openstacknode" : { "openstacknode" : { "nodes" : [ { "hostname" : "compute-01", "type" : "COMPUTE", "managementIp" : "10.203.25.244", "dataIp" : "10.134.34.222", "integrationBridge" : "of:00000000000000a1" }, { "hostname" : "compute-02", "type" : "COMPUTE", "managementIp" : "10.203.229.42", "dataIp" : "10.134.34.223", "integrationBridge" : "of:00000000000000a2", }, { "hostname" : "gateway-01", "type" : "GATEWAY", "managementIp" : "10.203.198.125", "dataIp" : "10.134.33.208", "integrationBridge" : "of:00000000000000a3", "routerBridge" : "of:00000000000000b3", }, { "hostname" : "gateway-02", "type" : "GATEWAY", "managementIp" : "10.203.198.131", "dataIp" : "10.134.33.209", "integrationBridge" : "of:00000000000000a4", "routerBridge" : "of:00000000000000b4", } ] } } }, "devices" : { "of:00000000000000a1" : { "basic" : { "driver" : "sona" } }, "of:00000000000000a2" : { "basic" : { "driver" : "sona" } } } } |
2. Make sure the following ONOS applications are started successfully.
Code Block |
---|
ONOS_APPS=drivers,drivers.ovsdb,openflow-base,ovsdb-base,dhcp,openstacknode,openstackswitching,openstackrouting,openstackinterface |
3. Push the network config file to ONOS.
Code Block |
---|
curl --user onos:rocks -X POST -H "Content-Type: application/json" http://onos-01:8181/onos/v1/network/configuration/ -d @network-cfg.json |
NOT MERGED YET: 4. Check all your compute and gateway nodes are in COMPLETE state in ONOS CLI.
Code Block |
---|
onos> openstack-nodes hostname=compute-01, type=COMPUTE, managementIp=10.203.25.244, dataIp=10.134.34.222, intBridge=of:00000000000000a1, routerBridge=Optional.empty init=INCOMPLETE hostname=compute-02, type=COMPUTE, managementIp=10.203.229.42, dataIp=10.134.34.223, intBridge=of:00000000000000a2, routerBridge=Optional.empty init=COMPLETE hostname=gateway-01, type=GATEWAY, managementIp=10.203.198.125, dataIp=10.134.33.208, intBridge=of:00000000000000a3, routerBridge=Optional[of:00000000000000b3] init=COMPLETE hostname=gateway-02, type=GATEWAY, managementIp=10.203.198.131, dataIp=10.134.33.209, intBridge=of:00000000000000a4, routerBridge=Optional[of:00000000000000b4] init=COMPLETE Total 4 nodes |
It's done. Enjoy.
NOT MERGED YET: CLI Commands
Command | Usage | Description |
---|---|---|
openstack-nodes | openstack-nodes | Shows the list of compute and gateway nodes that registered to openstack node service |
openstack-node-check | openstack-node-check [hosthame] | Shows the state of each node bootstrap steps |
openstack-node-init | openstack-node-init [hostname] | Try to re-initialize a given node. It's no harm to re-initialize already in COMPLETE state. |