Have questions? Stuck? Please check our FAQ for some common questions and answers.

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

This page provides guidelines for testing SONA in an automated way to help developers test their changes do not break anything. As SONA is an implementation of Neutron ML2 mechanism driver, we can make use of general OpenStack testing tools. Particularly, Tempest is OpenStack's official test suit for running a set of functional tests against an OpenStack cluster. Most OpenStack projects employ Tempest as a gate for automated testing of committed change. It is strongly recommended to run Tempest API and scenarios tests described here against your changes to SONA before submitting. Some test cases specific to SONA still need manual testing.

Setup Test Environment

Some Tempest scenario tests include accessing a VM with floating IP. That means you have to prepare a test machine reachable to the floating range. If you're using all-in-one gateway node, which has Quagga container for external router inside a gateway node, you can run the test at the gateway node. We use Rally for installing and configuring Tempest easily.

1. Install Docker

$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce


2. Create admin-openrc.sh with access information to target OpenStack deployment.

admin-openrc.sh
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=nova
export OS_AUTH_URL=http://10.1.1.161:35357/v2.0
export LC_ALL=C


3. Create Tempest configuration file for SONA. Note that, we only test the SONA against following API extensions.

  1. default-subnetpools: A subnet pool manages a pool of addresses from which subnets can be allocated. It ensures that there is no overlap between any two subnets allocated from the same pool. (link)
  2. network-ip-availability: Determining how much of a network’s or subnet’s IPs are being consumed with respect to the total number of IPs they support. (link)
  3. subnet_allocation: Network MTU update is available for all core plugins that implement the net-mtu-writable API extension. (link)
  4. external-net: The external network extension is used to specify whether the network is external or not. This information is used by Layer-3 network (router) extension. (link)
  5. router: Layer 3 network extension API. (link)
  6. security-group: Security group API. (link)
  7. ext-gw-mode: With this extension, the user will have the option of choosing whether SNAT should be enabled or not on a router basis. (link)
  8. net-mtu: After you adjust MTU configuration options in neutron.conf and ml2_conf.ini, you should update mtu attribute for all existing networks that need a new MTU. (link)


To test the floating IP features, you need to create an external network named "net-public" with a subnet associated with it in OpenStack. Note that, by far SONA does not support IPv6, so we need to disable the IPv6 related test cases.


sona-tempest.conf
[network-feature-enabled]
ipv6_subnet_attributes = False
ipv6 = False
api_extensions = default-subnetpools,network-ip-availability,subnet_allocation,external-net,router,security-group,ext-gw-mode,net-mtu
 
[compute-feature-enabled]
metadata_service = False
 
[network]
floating_network_name = net-public
project_networks_reachable = False


4. Run rally container through sona-setup. The sona-setup should be executed in SONA gateway node, and the role of sona-setup is to spawn a rally container. Note that, you should be able to connect to OpenStack spawned VM inside rally container by giving VM's floating IP.

$ sudo mkdir /var/lib/rally_container
$ sudo chown 65500 /var/lib/rally_container
$ sudo cp ~/admin-openrc.sh /var/lib/rally_container
$ sudo cp ~/sona-tempest.conf /var/lib/rally_container
$ git clone https://github.com/sonaproject/sona-setup.git
$ cd sona-setup && git checkout rally
$ ./createExternalRouter.sh


5. Enter rally container and register target OpenStack deployment.

$ sudo docker exec -it router bash
╔═════════════════════════════════════════════════════════════════════════════╗
║ Welcome to Rally Docker container!                                          ║
║  Rally certification tasks, samples and docs are located at /opt/rally/     ║
║  Rally at readthedocs - http://rally.readthedocs.org                        ║
║  How to contribute - http://rally.readthedocs.org/en/latest/contribute.html ║
║  If you have any questions, you can reach the Rally team by:                ║
║    * e-mail - openstack-dev@lists.openstack.org with tag [Rally] in subject ║
║    * irc - "#openstack-rally" channel at freenode.net                       ║
╚═════════════════════════════════════════════════════════════════════════════╝


root@router:/home/rally#
root@router:/home/rally# rally-manage db recreate
root@router:/home/rally# source admin-openrc.sh
root@router:/home/rally# rally deployment create --fromenv --name sona-test


6. Create Tempest verifier and configure it. It is required to use SONA specific version of Tempest until critical bugs are fixed. Also note that, you need to specify which OpenStack version that you would like to test against. In following example, we test the SONA against OpenStack pike.

root@router:/home/rally# rally verify create-verifier --type tempest --name tempest-verifier --source https://github.com/sonaproject/tempest.git --version pike
root@router:/home/rally# rally verify configure-verifier --extend sona-tempest.conf

Run Test

1. Create sona-skip-list.yaml as below to filter out not supported feature tests. The latest version of blacklist of test cases can be found in following link. https://github.com/sonaproject/tempest-sona-conf/blob/master/sona-skip-list.yaml

sona-skip-list.yaml
tempest.scenario.test_snapshot_pattern.TestSnapshotPattern.test_snapshot_pattern[compute,id-608e604b-1d63-4a82-8e3e-91bc665c90b4,image,network,slow]: "skip temporarily"
tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip[id-82068503-2cf2-4ed4-b3be-ecb89432e4bb]: "bug"
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic[compute,id-e79f879e-debb-440c-a7e4-efeda05b6848,network]: "bug"
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups[compute,id-d2f77418-fcc4-439d-b935-72eca704e293,network,slow]: "bug"
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port[compute,id-7c0bb1a2-d053-49a4-98f9-ca1a1d849f63,network,slow]: "not support"
tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_port_update_new_security_group[compute,id-f4d556d7-1526-42ad-bafb-6bebf48568f6,network,slow]: "not support"


2. Now, run Tempest network related API and scenario tests.

rally@1cc98e0b5941:~$ rally verify start --pattern network --skip-list sona-skip-list.yaml --detail
2017-05-16 06:03:35.215 1109 INFO rally.api [-] Starting verification (UUID=e9fcd543-a438-4845-82a9-62cc2ead2693) for deployment 'sona-test' (UUID=f51f26b4-0603-4f8a-a29a-21d045d898df) by verifier 'tempest-verifier' (UUID=aadced51-ddf6-4cb9-a422-df0d51119679).

======
Totals
======

Ran: 326 tests in 1515.553 sec.
 - Success: 147
 - Skipped: 179
 - Expected failures: 0
 - Unexpected success: 0
 - Failures: 0

Using verification (UUID=27174189-bbad-4292-b58a-dccf7dfab996) as the default verification for the future operations.
  • No labels