View Source

SRDynamicConf at 06 Oct 2020 01:37:21

<img src="https://jenkins.onosproject.org/view/QA/job/postjob-Fabric4/lastSuccessfulBuild/artifact/SRDynamicConf-stratum_master_20-builds_graph.jpg", alt="SRDynamicConf-stratum", style="width:525px;height:350px;border:0">

commit 095ec1e20e0f30d69581637db9c4be8580b7b7cd (HEAD -] master, origin/master, origin/HEAD)
Author: pierventre [pier@opennetworking.org]
AuthorDate: Mon Sep 28 14:15:23 2020 +0200
Commit: Pier Luigi Ventre [pier@opennetworking.org]
CommitDate: Fri Oct 2 13:59:32 2020 +0000

Improve onos-diagnostics-k8s for a better integration with tost

Case 13: Changing port configuration from untagged 10 to tagged 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

13.1 Constructing test variables - PASS
13.2 Apply cell to environment - PASS
13.3 Uninstalling Atomix - PASS
13.4 Uninstalling ONOS package - PASS
13.5 Building ONOS Docker image - PASS
13.6 Create Cluster Config - PASS
13.7 Installing Atomix via docker containers - PASS
13.8 Installing ONOS via docker containers - PASS
13.9 Set up ONOS secure SSH - PASS
13.10 Starting ONOS CLI sessions - PASS
13.11 Checking ONOS nodes - PASS
13.12 Checking ONOS applications - PASS
13.13 Set logging levels - PASS
13.14 Set ONOS configurations - PASS
13.15 Starting Mininet Topology - PASS
13.16 Configure switches in ONOS - PASS
13.17 Check whether the flow count is bigger than 104 - PASS
13.18 Check whether all flow status are ADDED - PASS
13.19 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE13_Before - PASS
13.20 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
13.21 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
13.22 Check whether the flow count is bigger than 104 - PASS
13.23 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

13.24 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE13_After - FAIL

IP connectivity failed

13.25 Stopping Mininet - PASS
13.26 Copying karaf logs - PASS

Case 23: Changing port configuration from untagged 10 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

23.1 Apply cell to environment - PASS
23.2 Uninstalling Atomix - PASS
23.3 Uninstalling ONOS package - PASS
23.4 Create Cluster Config - PASS
23.5 Installing Atomix via docker containers - PASS
23.6 Installing ONOS via docker containers - PASS
23.7 Set up ONOS secure SSH - PASS
23.8 Starting ONOS CLI sessions - PASS
23.9 Checking ONOS nodes - PASS
23.10 Checking ONOS applications - PASS
23.11 Set logging levels - PASS
23.12 Set ONOS configurations - PASS
23.13 Starting Mininet Topology - PASS
23.14 Configure switches in ONOS - PASS
23.15 Check whether the flow count is bigger than 106 - PASS
23.16 Check whether all flow status are ADDED - PASS
23.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE23_Before - PASS
23.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE23_Before - PASS
23.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
23.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
23.21 Check whether the flow count is bigger than 106 - PASS
23.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

23.23 Verify full connectivity for [u'h1', u'h2'] with tag CASE23_After - PASS
23.24 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE23_After - FAIL

IP connectivity failed

23.25 Stopping Mininet - PASS
23.26 Copying karaf logs - PASS

Case 33: Changing port configuration from untagged 10 to untagged 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

33.1 Apply cell to environment - PASS
33.2 Uninstalling Atomix - PASS
33.3 Uninstalling ONOS package - PASS
33.4 Create Cluster Config - PASS
33.5 Installing Atomix via docker containers - PASS
33.6 Installing ONOS via docker containers - PASS
33.7 Set up ONOS secure SSH - PASS
33.8 Starting ONOS CLI sessions - PASS
33.9 Checking ONOS nodes - PASS
33.10 Checking ONOS applications - PASS
33.11 Set logging levels - PASS
33.12 Set ONOS configurations - PASS
33.13 Starting Mininet Topology - PASS
33.14 Configure switches in ONOS - PASS
33.15 Check whether the flow count is bigger than 104 - FAIL

Flow count looks wrong; found False, expecting at least 104

33.16 Check whether all flow status are ADDED - PASS
33.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE33_Before - FAIL

IP connectivity failed

33.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE33_Before - FAIL

IP connectivity failed

33.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - FAIL

Flow count looks wrong: 0

33.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - FAIL

Flow count looks wrong: 0

33.21 Check whether the flow count is bigger than 104 - FAIL

Flow count looks wrong; found False, expecting at least 104

33.22 Check whether all flow status are ADDED - PASS
33.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE33_After - FAIL

IP connectivity failed

33.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE33_After - FAIL

IP connectivity failed

33.25 Stopping Mininet - PASS
33.26 Copying karaf logs - PASS

Case 43: Changing port configuration from untagged 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

43.1 Apply cell to environment - PASS
43.2 Uninstalling Atomix - PASS
43.3 Uninstalling ONOS package - PASS
43.4 Create Cluster Config - PASS
43.5 Installing Atomix via docker containers - PASS
43.6 Installing ONOS via docker containers - PASS
43.7 Set up ONOS secure SSH - PASS
43.8 Starting ONOS CLI sessions - PASS
43.9 Checking ONOS nodes - PASS
43.10 Checking ONOS applications - PASS
43.11 Set logging levels - PASS
43.12 Set ONOS configurations - PASS
43.13 Starting Mininet Topology - PASS
43.14 Configure switches in ONOS - PASS
43.15 Check whether the flow count is bigger than 104 - PASS
43.16 Check whether all flow status are ADDED - PASS
43.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE43_Before - PASS
43.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE43_Before - PASS
43.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
43.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
43.21 Check whether the flow count is bigger than 104 - PASS
43.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

43.23 Verify full connectivity for [u'h1', u'h2'] with tag CASE43_After - PASS
43.24 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE43_After - PASS
43.25 Stopping Mininet - PASS
43.26 Copying karaf logs - PASS

Case 53: Changing port configuration from untagged 10 to tagged 20 with native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

53.1 Apply cell to environment - PASS
53.2 Uninstalling Atomix - PASS
53.3 Uninstalling ONOS package - PASS
53.4 Create Cluster Config - PASS
53.5 Installing Atomix via docker containers - PASS
53.6 Installing ONOS via docker containers - PASS
53.7 Set up ONOS secure SSH - PASS
53.8 Starting ONOS CLI sessions - PASS
53.9 Checking ONOS nodes - PASS
53.10 Checking ONOS applications - PASS
53.11 Set logging levels - PASS
53.12 Set ONOS configurations - PASS
53.13 Starting Mininet Topology - PASS
53.14 Configure switches in ONOS - PASS
53.15 Check whether the flow count is bigger than 104 - PASS
53.16 Check whether all flow status are ADDED - PASS
53.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE53_Before - PASS
53.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE53_Before - PASS
53.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
53.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
53.21 Check whether the flow count is bigger than 104 - PASS
53.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

53.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE53_After - PASS
53.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE53_After - PASS
53.25 Stopping Mininet - PASS
53.26 Copying karaf logs - PASS

Case 63: Changing port configuration from untagged 10 to tagged 120 with native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

63.1 Apply cell to environment - PASS
63.2 Uninstalling Atomix - PASS
63.3 Uninstalling ONOS package - PASS
63.4 Create Cluster Config - PASS
63.5 Installing Atomix via docker containers - PASS
63.6 Installing ONOS via docker containers - PASS
63.7 Set up ONOS secure SSH - PASS
63.8 Starting ONOS CLI sessions - PASS
63.9 Checking ONOS nodes - PASS
63.10 Checking ONOS applications - PASS
63.11 Set logging levels - PASS
63.12 Set ONOS configurations - PASS
63.13 Starting Mininet Topology - PASS
63.14 Configure switches in ONOS - PASS
63.15 Check whether the flow count is bigger than 104 - PASS
63.16 Check whether all flow status are ADDED - PASS
63.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE63_Before - PASS
63.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE63_Before - PASS
63.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
63.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
63.21 Check whether the flow count is bigger than 104 - PASS
63.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

63.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE63_After - PASS
63.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE63_After - PASS
63.25 Stopping Mininet - PASS
63.26 Copying karaf logs - PASS

Case 73: Changing port configuration from tagged 10 to untagged 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

73.1 Apply cell to environment - PASS
73.2 Uninstalling Atomix - PASS
73.3 Uninstalling ONOS package - PASS
73.4 Create Cluster Config - PASS
73.5 Installing Atomix via docker containers - PASS
73.6 Installing ONOS via docker containers - PASS
73.7 Set up ONOS secure SSH - PASS
73.8 Starting ONOS CLI sessions - PASS
73.9 Checking ONOS nodes - PASS
73.10 Checking ONOS applications - PASS
73.11 Set logging levels - PASS
73.12 Set ONOS configurations - PASS
73.13 Starting Mininet Topology - PASS
73.14 Configure switches in ONOS - PASS
73.15 Check whether the flow count is bigger than 106 - PASS
73.16 Check whether all flow status are ADDED - PASS
73.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE73_Before - PASS
73.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE73_Before - PASS
73.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
73.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
73.21 Check whether the flow count is bigger than 106 - PASS
73.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

73.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE73_After - PASS
73.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE73_After - FAIL

IP connectivity failed

73.25 Stopping Mininet - PASS
73.26 Copying karaf logs - PASS

Case 83: Changing port configuration from tagged 20 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

83.1 Apply cell to environment - PASS
83.2 Uninstalling Atomix - PASS
83.3 Uninstalling ONOS package - PASS
83.4 Create Cluster Config - PASS
83.5 Installing Atomix via docker containers - PASS
83.6 Installing ONOS via docker containers - PASS
83.7 Set up ONOS secure SSH - PASS
83.8 Starting ONOS CLI sessions - PASS
83.9 Checking ONOS nodes - PASS
83.10 Checking ONOS applications - PASS
83.11 Set logging levels - PASS
83.12 Set ONOS configurations - PASS
83.13 Starting Mininet Topology - PASS
83.14 Configure switches in ONOS - PASS
83.15 Check whether the flow count is bigger than 104 - PASS
83.16 Check whether all flow status are ADDED - PASS
83.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE83_Before - PASS
83.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE83_Before - PASS
83.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
83.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
83.21 Check whether the flow count is bigger than 104 - PASS
83.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

83.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE83_After - PASS
83.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE83_After - PASS
83.25 Stopping Mininet - PASS
83.26 Copying karaf logs - PASS

Case 93: Changing port configuration from tagged 10 to tagged 120 with native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

93.1 Apply cell to environment - PASS
93.2 Uninstalling Atomix - PASS
93.3 Uninstalling ONOS package - PASS
93.4 Create Cluster Config - PASS
93.5 Installing Atomix via docker containers - PASS
93.6 Installing ONOS via docker containers - PASS
93.7 Set up ONOS secure SSH - PASS
93.8 Starting ONOS CLI sessions - PASS
93.9 Checking ONOS nodes - PASS
93.10 Checking ONOS applications - PASS
93.11 Set logging levels - PASS
93.12 Set ONOS configurations - PASS
93.13 Starting Mininet Topology - PASS
93.14 Configure switches in ONOS - PASS
93.15 Check whether the flow count is bigger than 106 - PASS
93.16 Check whether all flow status are ADDED - PASS
93.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE93_Before - PASS
93.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE93_Before - PASS
93.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
93.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
93.21 Check whether the flow count is bigger than 106 - PASS
93.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

93.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE93_After - PASS
93.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE93_After - PASS
93.25 Stopping Mininet - PASS
93.26 Copying karaf logs - PASS

Case 103: Changing port configuration from tagged 10 and native 20 to untagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - PASS

103.1 Apply cell to environment - PASS
103.2 Uninstalling Atomix - PASS
103.3 Uninstalling ONOS package - PASS
103.4 Create Cluster Config - PASS
103.5 Installing Atomix via docker containers - PASS
103.6 Installing ONOS via docker containers - PASS
103.7 Set up ONOS secure SSH - PASS
103.8 Starting ONOS CLI sessions - PASS
103.9 Checking ONOS nodes - PASS
103.10 Checking ONOS applications - PASS
103.11 Set logging levels - PASS
103.12 Set ONOS configurations - PASS
103.13 Starting Mininet Topology - PASS
103.14 Configure switches in ONOS - PASS
103.15 Check whether the flow count is bigger than 104 - PASS
103.16 Check whether all flow status are ADDED - PASS
103.17 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE103_Before - PASS
103.18 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
103.19 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
103.20 Check whether the flow count is bigger than 104 - PASS
103.21 Check whether all flow status are ADDED - PASS
103.22 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE103_After - PASS
103.23 Stopping Mininet - PASS
103.24 Copying karaf logs - PASS

Case 113: Changing port configuration from tagged 20 and native 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

113.1 Apply cell to environment - PASS
113.2 Uninstalling Atomix - PASS
113.3 Uninstalling ONOS package - PASS
113.4 Create Cluster Config - PASS
113.5 Installing Atomix via docker containers - PASS
113.6 Installing ONOS via docker containers - PASS
113.7 Set up ONOS secure SSH - PASS
113.8 Starting ONOS CLI sessions - PASS
113.9 Checking ONOS nodes - PASS
113.10 Checking ONOS applications - PASS
113.11 Set logging levels - PASS
113.12 Set ONOS configurations - PASS
113.13 Starting Mininet Topology - PASS
113.14 Configure switches in ONOS - PASS
113.15 Check whether the flow count is bigger than 104 - PASS
113.16 Check whether all flow status are ADDED - PASS
113.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE113_Before - FAIL

IP connectivity failed

113.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE113_Before - FAIL

IP connectivity failed

113.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - FAIL

Flow count looks wrong: 39

113.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
113.21 Check whether the flow count is bigger than 104 - PASS
113.22 Check whether all flow status are ADDED - PASS
113.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE113_After - FAIL

IP connectivity failed

113.24 Verify full connectivity for [u'h1', u'h2', u'h3'] with tag CASE113_After - PASS
113.25 Stopping Mininet - PASS
113.26 Copying karaf logs - PASS

Case 123: Changing port configuration from tagged 20 and native 10 to tagged 20 and native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

123.1 Apply cell to environment - PASS
123.2 Uninstalling Atomix - PASS
123.3 Uninstalling ONOS package - PASS
123.4 Create Cluster Config - PASS
123.5 Installing Atomix via docker containers - PASS
123.6 Installing ONOS via docker containers - PASS
123.7 Set up ONOS secure SSH - PASS
123.8 Starting ONOS CLI sessions - PASS
123.9 Checking ONOS nodes - PASS
123.10 Checking ONOS applications - PASS
123.11 Set logging levels - PASS
123.12 Set ONOS configurations - PASS
123.13 Starting Mininet Topology - PASS
123.14 Configure switches in ONOS - PASS
123.15 Check whether the flow count is bigger than 104 - PASS
123.16 Check whether all flow status are ADDED - PASS
123.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE123_Before - PASS
123.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE123_Before - PASS
123.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
123.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
123.21 Check whether the flow count is bigger than 104 - PASS
123.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

123.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE123_After - PASS
123.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE123_After - PASS
123.25 Stopping Mininet - PASS
123.26 Copying karaf logs - PASS

Case 133: Changing port configuration from tagged 20 and native 10 to tagged 120 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

133.1 Apply cell to environment - PASS
133.2 Uninstalling Atomix - PASS
133.3 Uninstalling ONOS package - PASS
133.4 Create Cluster Config - PASS
133.5 Installing Atomix via docker containers - PASS
133.6 Installing ONOS via docker containers - PASS
133.7 Set up ONOS secure SSH - PASS
133.8 Starting ONOS CLI sessions - PASS
133.9 Checking ONOS nodes - PASS
133.10 Checking ONOS applications - PASS
133.11 Set logging levels - PASS
133.12 Set ONOS configurations - PASS
133.13 Starting Mininet Topology - PASS
133.14 Configure switches in ONOS - PASS
133.15 Check whether the flow count is bigger than 104 - PASS
133.16 Check whether all flow status are ADDED - PASS
133.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE133_Before - PASS
133.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE133_Before - PASS
133.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 52 - PASS
133.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 52 - PASS
133.21 Check whether the flow count is bigger than 104 - PASS
133.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

133.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE133_After - PASS
133.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE133_After - PASS
133.25 Stopping Mininet - PASS
133.26 Copying karaf logs - PASS

Case 143: Changing port configuration from untagged 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

143.1 Apply cell to environment - PASS
143.2 Uninstalling Atomix - PASS
143.3 Uninstalling ONOS package - PASS
143.4 Create Cluster Config - PASS
143.5 Installing Atomix via docker containers - PASS
143.6 Installing ONOS via docker containers - PASS
143.7 Set up ONOS secure SSH - PASS
143.8 Starting ONOS CLI sessions - PASS
143.9 Checking ONOS nodes - PASS
143.10 Checking ONOS applications - PASS
143.11 Set logging levels - PASS
143.12 Set ONOS configurations - PASS
143.13 Starting Mininet Topology - PASS
143.14 Configure switches in ONOS - PASS
143.15 Check whether the flow count is bigger than 106 - PASS
143.16 Check whether all flow status are ADDED - PASS
143.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE143_Before - PASS
143.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE143_Before - PASS
143.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
143.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
143.21 Check whether the flow count is bigger than 106 - PASS
143.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

143.23 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE143_After - PASS
143.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE143_After - PASS
143.25 Stopping Mininet - PASS
143.26 Copying karaf logs - PASS

Case 153: Changing port configuration from untagged 10 to tagged 120, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

153.1 Apply cell to environment - PASS
153.2 Uninstalling Atomix - PASS
153.3 Uninstalling ONOS package - PASS
153.4 Create Cluster Config - PASS
153.5 Installing Atomix via docker containers - PASS
153.6 Installing ONOS via docker containers - PASS
153.7 Set up ONOS secure SSH - PASS
153.8 Starting ONOS CLI sessions - PASS
153.9 Checking ONOS nodes - PASS
153.10 Checking ONOS applications - PASS
153.11 Set logging levels - PASS
153.12 Set ONOS configurations - PASS
153.13 Starting Mininet Topology - PASS
153.14 Configure switches in ONOS - PASS
153.15 Check whether the flow count is bigger than 106 - PASS
153.16 Check whether all flow status are ADDED - PASS
153.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE153_Before - PASS
153.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE153_Before - PASS
153.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
153.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
153.21 Check whether the flow count is bigger than 106 - PASS
153.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

153.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE153_After - PASS
153.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE153_After - PASS
153.25 Stopping Mininet - PASS
153.26 Copying karaf logs - PASS

Case 163: Changing port configuration from untagged 10 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

163.1 Apply cell to environment - PASS
163.2 Uninstalling Atomix - PASS
163.3 Uninstalling ONOS package - PASS
163.4 Create Cluster Config - PASS
163.5 Installing Atomix via docker containers - PASS
163.6 Installing ONOS via docker containers - PASS
163.7 Set up ONOS secure SSH - PASS
163.8 Starting ONOS CLI sessions - PASS
163.9 Checking ONOS nodes - PASS
163.10 Checking ONOS applications - PASS
163.11 Set logging levels - PASS
163.12 Set ONOS configurations - PASS
163.13 Starting Mininet Topology - PASS
163.14 Configure switches in ONOS - PASS
163.15 Check whether the flow count is bigger than 106 - PASS
163.16 Check whether all flow status are ADDED - PASS
163.17 Verify full connectivity for [u'h3', u'h4'] with tag CASE163_Before - PASS
163.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE163_Before - PASS
163.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
163.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
163.21 Check whether the flow count is bigger than 106 - PASS
163.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

163.23 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE163_After - PASS
163.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE163_After - PASS
163.25 Stopping Mininet - PASS
163.26 Copying karaf logs - PASS

Case 173: Changing port configuration from tagged 20 to tagged 120, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

173.1 Apply cell to environment - PASS
173.2 Uninstalling Atomix - PASS
173.3 Uninstalling ONOS package - PASS
173.4 Create Cluster Config - PASS
173.5 Installing Atomix via docker containers - PASS
173.6 Installing ONOS via docker containers - PASS
173.7 Set up ONOS secure SSH - PASS
173.8 Starting ONOS CLI sessions - PASS
173.9 Checking ONOS nodes - PASS
173.10 Checking ONOS applications - PASS
173.11 Set logging levels - PASS
173.12 Set ONOS configurations - PASS
173.13 Starting Mininet Topology - PASS
173.14 Configure switches in ONOS - PASS
173.15 Check whether the flow count is bigger than 106 - PASS
173.16 Check whether all flow status are ADDED - PASS
173.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE173_Before - PASS
173.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE173_Before - PASS
173.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
173.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
173.21 Check whether the flow count is bigger than 106 - PASS
173.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

173.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE173_After - PASS
173.24 Verify full connectivity for [u'h1', u'h2', u'h3'] with tag CASE173_After - PASS
173.25 Stopping Mininet - PASS
173.26 Copying karaf logs - PASS

Case 183: Changing port configuration from tagged 20 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

183.1 Apply cell to environment - PASS
183.2 Uninstalling Atomix - PASS
183.3 Uninstalling ONOS package - PASS
183.4 Create Cluster Config - PASS
183.5 Installing Atomix via docker containers - PASS
183.6 Installing ONOS via docker containers - PASS
183.7 Set up ONOS secure SSH - PASS
183.8 Starting ONOS CLI sessions - PASS
183.9 Checking ONOS nodes - PASS
183.10 Checking ONOS applications - PASS
183.11 Set logging levels - PASS
183.12 Set ONOS configurations - PASS
183.13 Starting Mininet Topology - PASS
183.14 Configure switches in ONOS - PASS
183.15 Check whether the flow count is bigger than 106 - PASS
183.16 Check whether all flow status are ADDED - PASS
183.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE183_Before - PASS
183.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE183_Before - PASS
183.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
183.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
183.21 Check whether the flow count is bigger than 106 - PASS
183.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

183.23 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE183_After - PASS
183.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE183_After - PASS
183.25 Stopping Mininet - PASS
183.26 Copying karaf logs - PASS

Case 193: Changing port configuration from tagged 20 and native 10 to untagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

193.1 Apply cell to environment - PASS
193.2 Uninstalling Atomix - PASS
193.3 Uninstalling ONOS package - PASS
193.4 Create Cluster Config - PASS
193.5 Installing Atomix via docker containers - PASS
193.6 Installing ONOS via docker containers - PASS
193.7 Set up ONOS secure SSH - PASS
193.8 Starting ONOS CLI sessions - PASS
193.9 Checking ONOS nodes - PASS
193.10 Checking ONOS applications - PASS
193.11 Set logging levels - PASS
193.12 Set ONOS configurations - PASS
193.13 Starting Mininet Topology - PASS
193.14 Configure switches in ONOS - PASS
193.15 Check whether the flow count is bigger than 106 - PASS
193.16 Check whether all flow status are ADDED - PASS
193.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE193_Before - PASS
193.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE193_Before - PASS
193.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
193.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
193.21 Check whether the flow count is bigger than 106 - PASS
193.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

193.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE193_After - PASS
193.24 Verify full connectivity for [u'h1', u'h2', u'h3'] with tag CASE193_After - PASS
193.25 Stopping Mininet - PASS
193.26 Copying karaf logs - PASS

Case 203: Changing port configuration from tagged 20 and native 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - PASS

203.1 Apply cell to environment - PASS
203.2 Uninstalling Atomix - PASS
203.3 Uninstalling ONOS package - PASS
203.4 Create Cluster Config - PASS
203.5 Installing Atomix via docker containers - PASS
203.6 Installing ONOS via docker containers - PASS
203.7 Set up ONOS secure SSH - PASS
203.8 Starting ONOS CLI sessions - PASS
203.9 Checking ONOS nodes - PASS
203.10 Checking ONOS applications - PASS
203.11 Set logging levels - PASS
203.12 Set ONOS configurations - PASS
203.13 Starting Mininet Topology - PASS
203.14 Configure switches in ONOS - PASS
203.15 Check whether the flow count is bigger than 106 - PASS
203.16 Check whether all flow status are ADDED - PASS
203.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE203_Before - PASS
203.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE203_Before - PASS
203.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
203.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
203.21 Check whether the flow count is bigger than 106 - PASS
203.22 Check whether all flow status are ADDED - PASS
203.23 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE203_After - PASS
203.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE203_After - PASS
203.25 Stopping Mininet - PASS
203.26 Copying karaf logs - PASS

Case 213: Changing port configuration from tagged 20 and native 10 to tagged 20 and native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

213.1 Apply cell to environment - PASS
213.2 Uninstalling Atomix - PASS
213.3 Uninstalling ONOS package - PASS
213.4 Create Cluster Config - PASS
213.5 Installing Atomix via docker containers - PASS
213.6 Installing ONOS via docker containers - PASS
213.7 Set up ONOS secure SSH - PASS
213.8 Starting ONOS CLI sessions - PASS
213.9 Checking ONOS nodes - PASS
213.10 Checking ONOS applications - PASS
213.11 Set logging levels - PASS
213.12 Set ONOS configurations - PASS
213.13 Starting Mininet Topology - PASS
213.14 Configure switches in ONOS - PASS
213.15 Check whether the flow count is bigger than 106 - PASS
213.16 Check whether all flow status are ADDED - PASS
213.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE213_Before - PASS
213.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE213_Before - PASS
213.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
213.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
213.21 Check whether the flow count is bigger than 106 - PASS
213.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

213.23 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE213_After - PASS
213.24 Verify full connectivity for [u'h1', u'h2'] with tag CASE213_After - PASS
213.25 Stopping Mininet - PASS
213.26 Copying karaf logs - PASS

Case 223: Changing port configuration from tagged 20 and native 10 to tagged 120 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

223.1 Apply cell to environment - PASS
223.2 Uninstalling Atomix - PASS
223.3 Uninstalling ONOS package - PASS
223.4 Create Cluster Config - PASS
223.5 Installing Atomix via docker containers - PASS
223.6 Installing ONOS via docker containers - PASS
223.7 Set up ONOS secure SSH - PASS
223.8 Starting ONOS CLI sessions - PASS
223.9 Checking ONOS nodes - PASS
223.10 Checking ONOS applications - PASS
223.11 Set logging levels - PASS
223.12 Set ONOS configurations - PASS
223.13 Starting Mininet Topology - PASS
223.14 Configure switches in ONOS - PASS
223.15 Check whether the flow count is bigger than 106 - PASS
223.16 Check whether all flow status are ADDED - PASS
223.17 Verify full connectivity for [u'h1', u'h3', u'h4'] with tag CASE223_Before - PASS
223.18 Verify full connectivity for [u'h1', u'h2'] with tag CASE223_Before - PASS
223.19 Check whether the flow count of device device:bmv2:leaf1 is bigger than 53 - PASS
223.20 Check whether the flow count of device device:bmv2:leaf2 is bigger than 53 - PASS
223.21 Check whether the flow count is bigger than 106 - PASS
223.22 Check whether all flow status are ADDED - FAIL

Flow status is wrong!

223.23 Verify full connectivity for [u'h3', u'h4'] with tag CASE223_After - PASS
223.24 Verify full connectivity for [u'h1', u'h2', u'h3'] with tag CASE223_After - PASS
223.25 Stopping Mininet - PASS
223.26 Copying karaf logs - PASS

Case 13: Changing port configuration from untagged 10 to tagged 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 23: Changing port configuration from untagged 10 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 33: Changing port configuration from untagged 10 to untagged 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 43: Changing port configuration from untagged 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 53: Changing port configuration from untagged 10 to tagged 20 with native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 63: Changing port configuration from untagged 10 to tagged 120 with native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 73: Changing port configuration from tagged 10 to untagged 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 83: Changing port configuration from tagged 20 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 93: Changing port configuration from tagged 10 to tagged 120 with native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 103: Changing port configuration from tagged 10 and native 20 to untagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - PASS

Case 113: Changing port configuration from tagged 20 and native 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 123: Changing port configuration from tagged 20 and native 10 to tagged 20 and native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 133: Changing port configuration from tagged 20 and native 10 to tagged 120 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 143: Changing port configuration from untagged 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 153: Changing port configuration from untagged 10 to tagged 120, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 163: Changing port configuration from untagged 10 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 173: Changing port configuration from tagged 20 to tagged 120, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 183: Changing port configuration from tagged 20 to tagged 20 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 193: Changing port configuration from tagged 20 and native 10 to untagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 203: Changing port configuration from tagged 20 and native 10 to tagged 20, with 2x2 leaf-spine topology and 3 ONOS instances - PASS

Case 213: Changing port configuration from tagged 20 and native 10 to tagged 20 and native 110, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 223: Changing port configuration from tagged 20 and native 10 to tagged 120 and native 10, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL

Case 233: Consecutive configuration changes, with 2x2 leaf-spine topology and 3 ONOS instances - FAIL