Due to a ransomware attack, the wiki was reverted to a July 2022 version. . We apologize for the lack of a more recent valid backup.
This page provides guidelines for testing SONA in an automated way to help developers test their changes do not break anything. As SONA is an implementation of Neutron ML2 mechanism driver, we can make use of general OpenStack testing tools. Particularly, Tempest is OpenStack's official test suit for running a set of functional tests against an OpenStack cluster. Most OpenStack projects employ Tempest as a gate for automated testing of committed change. It is strongly recommended to run Tempest API and scenarios tests described here against your changes to SONA before submitting. Some test cases specific to SONA still need manual testing.
Setup Test Environment
Some Tempest scenario tests include accessing a VM with floating IP. That means you have to prepare a test machine reachable to the floating range. If you're using all-in-one gateway node, which has Quagga container for external router inside a gateway node, you can run the test at the gateway node. We use Rally for installing and configuring Tempest easily.
1. Install Docker
$ sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - $ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" $ sudo apt-get update $ sudo apt-get install docker-ce
2. Create admin-openrc.sh with access information to target OpenStack deployment.
admin-openrc.sh
export OS_TENANT_NAME=admin export OS_USERNAME=admin export OS_PASSWORD=nova export OS_AUTH_URL=http://10.1.1.161:35357/v2.0 export LC_ALL=C
3. Create Tempest configuration file for SONA.
sona-tempest.conf
[network-feature-enabled] ipv6_subnet_attributes = False ipv6 = False api_extensions = default-subnetpools,network-ip-availability,subnet_allocation,external-net,router,security-group,ext-gw-mode,net-mtu [compute-feature-enabled] metadata_service = False [network] floating_network_name = net-public project_networks_reachable = False
4. Run Rally container.
$ sudo mkdir /var/lib/rally_container $ sudo chown 65500 /var/lib/rally_container $ sudo cp ~/admin-openrc.sh /var/lib/rally_container $ sudo cp ~/sona-tempest.conf /var/lib/rally_container $ sudo docker run -it -v /var/lib/rally_container:/home/rally rallyforge/rally:0.9.1
5. Register target OpenStack deployment.
rally@1cc98e0b5941:~$ rally-manage db recreate rally@1cc98e0b5941:~$ source admin-openrc.sh rally@1cc98e0b5941:~$ rally deployment create --fromenv --name sona-test
6. Create Tempest verifier and configure it. It is required to use SONA specific version of Tempest until critical bugs are fixed.
rally@1cc98e0b5941:~$ rally verify create-verifier --type tempest --name tempest-verifier --source https://github.com/sonaproject/tempest.git rally@1cc98e0b5941:~$ rally verify configure-verifier --extend sona-tempest.conf
Run Test
1. Create sona-skip-list.yaml as below to filter out not supported feature tests.
sona-skip-list.yaml
tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_delete_external_networks_with_floating_ip[id-82068503-2cf2-4ed4-b3be-ecb89432e4bb]: "bug" tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic[compute,id-e79f879e-debb-440c-a7e4-efeda05b6848,network]: "bug" tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups[compute,id-d2f77418-fcc4-439d-b935-72eca704e293,network,slow]: "bug" tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port[compute,id-7c0bb1a2-d053-49a4-98f9-ca1a1d849f63,network,slow]: "not support" tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details[compute,id-d8bb918e-e2df-48b2-97cd-b73c95450980,network,slow]: "not support" tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_instance_port_admin_state[compute,id-f5dfcc22-45fd-409f-954c-5bd500d7890b,network,slow]: "not support" tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_update_router_admin_state[compute,id-04b9fe4e-85e8-4aea-b937-ea93885ac59f,network,slow]: "not support"
2. Now, run Tempest network related API and scenario tests.
rally@1cc98e0b5941:~$ rally verify start --pattern network --skip-list sona-skip-list.yaml --detail 2017-05-16 06:03:35.215 1109 INFO rally.api [-] Starting verification (UUID=e9fcd543-a438-4845-82a9-62cc2ead2693) for deployment 'sona-test' (UUID=f51f26b4-0603-4f8a-a29a-21d045d898df) by verifier 'tempest-verifier' (UUID=aadced51-ddf6-4cb9-a422-df0d51119679). ====== Totals ====== Ran: 375 tests in 928.312 sec. - Success: 181 - Skipped: 194 - Expected failures: 0 - Unexpected success: 0 - Failures: 0 Using verification (UUID=27174189-bbad-4292-b58a-dccf7dfab996) as the default verification for the future operations.