1.10-SR High Availability

Case 1: High Availability tests - ONOS failures with 2x2 Leaf-spine - FAIL

• 1.1 Constructing test variables - PASS
• 1.2 Create and Apply cell file - PASS
• 1.3 Create and Install ONOS package - PASS
• 1.4 Starting ONOS service - PASS
• 1.5 Checking if ONOS CLI is ready - PASS
• 1.6 Starting Mininet Topology - PASS
• 1.7 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 1.8 Check whether all flow status are ADDED - PASS
• 1.9 Connectivity for [u'h1', u'h2', u'h3', u'h4'] - FAIL
• IP connectivity failed
• 1.10 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 1.11 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 1.12 Killing ONOS instance - PASS
• 1.13 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Failure1 - FAIL
• IP connectivity failed
• 1.14 Connectivity for [u'olt10', u'vsg10'] CASE1_Failure1 - FAIL
• IP connectivity failed
• 1.15 Connectivity for [u'olt5', u'vsg5'] CASE1_Failure1 - FAIL
• IP connectivity failed
• 1.16 Recovering ONOS instance - PASS
• 1.17 Checking if ONOS CLI is ready - PASS
• 1.18 Checking ONOS nodes - PASS
• 1.19 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 1.20 Check whether all flow status are ADDED - PASS
• 1.21 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Recovery1 - FAIL
• IP connectivity failed
• 1.22 Connectivity for [u'olt10', u'vsg10'] CASE1_Recovery1 - FAIL
• IP connectivity failed
• 1.23 Connectivity for [u'olt5', u'vsg5'] CASE1_Recovery1 - FAIL
• IP connectivity failed
• 1.24 Killing ONOS instance - PASS
• 1.25 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Failure2 - FAIL
• IP connectivity failed
• 1.26 Connectivity for [u'olt10', u'vsg10'] CASE1_Failure2 - FAIL
• IP connectivity failed
• 1.27 Connectivity for [u'olt5', u'vsg5'] CASE1_Failure2 - FAIL
• IP connectivity failed
• 1.28 Recovering ONOS instance - PASS
• 1.29 Checking if ONOS CLI is ready - PASS
• 1.30 Checking ONOS nodes - PASS
• 1.31 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 1.32 Check whether all flow status are ADDED - PASS
• 1.33 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Recovery2 - FAIL
• IP connectivity failed
• 1.34 Connectivity for [u'olt10', u'vsg10'] CASE1_Recovery2 - FAIL
• IP connectivity failed
• 1.35 Connectivity for [u'olt5', u'vsg5'] CASE1_Recovery2 - FAIL
• IP connectivity failed
• 1.36 Killing ONOS instance - PASS
• 1.37 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Failure3 - FAIL
• IP connectivity failed
• 1.38 Connectivity for [u'olt10', u'vsg10'] CASE1_Failure3 - FAIL
• IP connectivity failed
• 1.39 Connectivity for [u'olt5', u'vsg5'] CASE1_Failure3 - FAIL
• IP connectivity failed
• 1.40 Recovering ONOS instance - PASS
• 1.41 Checking if ONOS CLI is ready - PASS
• 1.42 Checking ONOS nodes - PASS
• 1.43 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 1.44 Check whether all flow status are ADDED - PASS
• 1.45 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE1_Recovery3 - FAIL
• IP connectivity failed
• 1.46 Connectivity for [u'olt10', u'vsg10'] CASE1_Recovery3 - FAIL
• IP connectivity failed
• 1.47 Connectivity for [u'olt5', u'vsg5'] CASE1_Recovery3 - FAIL
• IP connectivity failed

Case 2: High Availability tests - ONOS random failures with 2x2 Leaf-spine - FAIL

• 2.1 Create and Apply cell file - PASS
• 2.2 Create and Install ONOS package - PASS
• 2.3 Starting ONOS service - PASS
• 2.4 Checking if ONOS CLI is ready - PASS
• 2.5 Starting Mininet Topology - PASS
• 2.6 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 2.7 Check whether all flow status are ADDED - PASS
• 2.8 Connectivity for [u'h1', u'h2', u'h3', u'h4'] - FAIL
• IP connectivity failed
• 2.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 2.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 2.11 Killing ONOS instance - PASS
• 2.12 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Failure1 - FAIL
• IP connectivity failed
• 2.13 Connectivity for [u'olt10', u'vsg10'] CASE2_Failure1 - FAIL
• IP connectivity failed
• 2.14 Connectivity for [u'olt5', u'vsg5'] CASE2_Failure1 - FAIL
• IP connectivity failed
• 2.15 Recovering ONOS instance - PASS
• 2.16 Checking if ONOS CLI is ready - PASS
• 2.17 Checking ONOS nodes - PASS
• 2.18 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 2.19 Check whether all flow status are ADDED - PASS
• 2.20 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Recovery1 - FAIL
• IP connectivity failed
• 2.21 Connectivity for [u'olt10', u'vsg10'] CASE2_Recovery1 - FAIL
• IP connectivity failed
• 2.22 Connectivity for [u'olt5', u'vsg5'] CASE2_Recovery1 - FAIL
• IP connectivity failed
• 2.23 Killing ONOS instance - PASS
• 2.24 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Failure2 - FAIL
• IP connectivity failed
• 2.25 Connectivity for [u'olt10', u'vsg10'] CASE2_Failure2 - FAIL
• IP connectivity failed
• 2.26 Connectivity for [u'olt5', u'vsg5'] CASE2_Failure2 - FAIL
• IP connectivity failed
• 2.27 Recovering ONOS instance - PASS
• 2.28 Checking if ONOS CLI is ready - PASS
• 2.29 Checking ONOS nodes - PASS
• 2.30 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 2.31 Check whether all flow status are ADDED - PASS
• 2.32 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Recovery2 - FAIL
• IP connectivity failed
• 2.33 Connectivity for [u'olt10', u'vsg10'] CASE2_Recovery2 - FAIL
• IP connectivity failed
• 2.34 Connectivity for [u'olt5', u'vsg5'] CASE2_Recovery2 - FAIL
• IP connectivity failed
• 2.35 Killing ONOS instance - PASS
• 2.36 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Failure3 - FAIL
• IP connectivity failed
• 2.37 Connectivity for [u'olt10', u'vsg10'] CASE2_Failure3 - FAIL
• IP connectivity failed
• 2.38 Connectivity for [u'olt5', u'vsg5'] CASE2_Failure3 - FAIL
• IP connectivity failed
• 2.39 Recovering ONOS instance - PASS
• 2.40 Checking if ONOS CLI is ready - PASS
• 2.41 Checking ONOS nodes - PASS
• 2.42 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 2.43 Check whether all flow status are ADDED - PASS
• 2.44 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE2_Recovery3 - FAIL
• IP connectivity failed
• 2.45 Connectivity for [u'olt10', u'vsg10'] CASE2_Recovery3 - FAIL
• IP connectivity failed
• 2.46 Connectivity for [u'olt5', u'vsg5'] CASE2_Recovery3 - FAIL
• IP connectivity failed

Case 3: High Availability tests - ONOS failures with 4x4 Leaf-spine - FAIL

• 3.1 Create and Apply cell file - PASS
• 3.2 Create and Install ONOS package - PASS
• 3.3 Starting ONOS service - PASS
• 3.4 Checking if ONOS CLI is ready - PASS
• 3.5 Starting Mininet Topology - PASS
• 3.6 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 3.7 Check whether all flow status are ADDED - PASS
• 3.8 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] - FAIL
• IP connectivity failed
• 3.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 3.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 3.11 Killing ONOS instance - PASS
• 3.12 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Failure1 - FAIL
• IP connectivity failed
• 3.13 Connectivity for [u'olt10', u'vsg10'] CASE3_Failure1 - FAIL
• IP connectivity failed
• 3.14 Connectivity for [u'olt5', u'vsg5'] CASE3_Failure1 - FAIL
• IP connectivity failed
• 3.15 Recovering ONOS instance - PASS
• 3.16 Checking if ONOS CLI is ready - PASS
• 3.17 Checking ONOS nodes - PASS
• 3.18 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 3.19 Check whether all flow status are ADDED - PASS
• 3.20 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Recovery1 - FAIL
• IP connectivity failed
• 3.21 Connectivity for [u'olt10', u'vsg10'] CASE3_Recovery1 - FAIL
• IP connectivity failed
• 3.22 Connectivity for [u'olt5', u'vsg5'] CASE3_Recovery1 - FAIL
• IP connectivity failed
• 3.23 Killing ONOS instance - PASS
• 3.24 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Failure2 - FAIL
• IP connectivity failed
• 3.25 Connectivity for [u'olt10', u'vsg10'] CASE3_Failure2 - FAIL
• IP connectivity failed
• 3.26 Connectivity for [u'olt5', u'vsg5'] CASE3_Failure2 - FAIL
• IP connectivity failed
• 3.27 Recovering ONOS instance - PASS
• 3.28 Checking if ONOS CLI is ready - PASS
• 3.29 Checking ONOS nodes - PASS
• 3.30 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 3.31 Check whether all flow status are ADDED - PASS
• 3.32 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Recovery2 - FAIL
• IP connectivity failed
• 3.33 Connectivity for [u'olt10', u'vsg10'] CASE3_Recovery2 - FAIL
• IP connectivity failed
• 3.34 Connectivity for [u'olt5', u'vsg5'] CASE3_Recovery2 - FAIL
• IP connectivity failed
• 3.35 Killing ONOS instance - PASS
• 3.36 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Failure3 - FAIL
• IP connectivity failed
• 3.37 Connectivity for [u'olt10', u'vsg10'] CASE3_Failure3 - FAIL
• IP connectivity failed
• 3.38 Connectivity for [u'olt5', u'vsg5'] CASE3_Failure3 - FAIL
• IP connectivity failed
• 3.39 Recovering ONOS instance - PASS
• 3.40 Checking if ONOS CLI is ready - PASS
• 3.41 Checking ONOS nodes - PASS
• 3.42 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 3.43 Check whether all flow status are ADDED - PASS
• 3.44 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE3_Recovery3 - FAIL
• IP connectivity failed
• 3.45 Connectivity for [u'olt10', u'vsg10'] CASE3_Recovery3 - FAIL
• IP connectivity failed
• 3.46 Connectivity for [u'olt5', u'vsg5'] CASE3_Recovery3 - FAIL
• IP connectivity failed

Case 4: High Availability tests - ONOS random failures with 4x4 Leaf-spine - FAIL

• 4.1 Create and Apply cell file - PASS
• 4.2 Create and Install ONOS package - PASS
• 4.3 Starting ONOS service - PASS
• 4.4 Checking if ONOS CLI is ready - PASS
• 4.5 Starting Mininet Topology - PASS
• 4.6 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 4.7 Check whether all flow status are ADDED - PASS
• 4.8 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] - FAIL
• IP connectivity failed
• 4.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 4.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 4.11 Killing ONOS instance - PASS
• 4.12 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Failure1 - FAIL
• IP connectivity failed
• 4.13 Connectivity for [u'olt10', u'vsg10'] CASE4_Failure1 - FAIL
• IP connectivity failed
• 4.14 Connectivity for [u'olt5', u'vsg5'] CASE4_Failure1 - FAIL
• IP connectivity failed
• 4.15 Recovering ONOS instance - PASS
• 4.16 Checking if ONOS CLI is ready - PASS
• 4.17 Checking ONOS nodes - PASS
• 4.18 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 4.19 Check whether all flow status are ADDED - PASS
• 4.20 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Recovery1 - FAIL
• IP connectivity failed
• 4.21 Connectivity for [u'olt10', u'vsg10'] CASE4_Recovery1 - FAIL
• IP connectivity failed
• 4.22 Connectivity for [u'olt5', u'vsg5'] CASE4_Recovery1 - FAIL
• IP connectivity failed
• 4.23 Killing ONOS instance - PASS
• 4.24 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Failure2 - FAIL
• IP connectivity failed
• 4.25 Connectivity for [u'olt10', u'vsg10'] CASE4_Failure2 - FAIL
• IP connectivity failed
• 4.26 Connectivity for [u'olt5', u'vsg5'] CASE4_Failure2 - FAIL
• IP connectivity failed
• 4.27 Recovering ONOS instance - PASS
• 4.28 Checking if ONOS CLI is ready - PASS
• 4.29 Checking ONOS nodes - PASS
• 4.30 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 4.31 Check whether all flow status are ADDED - PASS
• 4.32 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Recovery2 - FAIL
• IP connectivity failed
• 4.33 Connectivity for [u'olt10', u'vsg10'] CASE4_Recovery2 - FAIL
• IP connectivity failed
• 4.34 Connectivity for [u'olt5', u'vsg5'] CASE4_Recovery2 - FAIL
• IP connectivity failed
• 4.35 Killing ONOS instance - PASS
• 4.36 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Failure3 - FAIL
• IP connectivity failed
• 4.37 Connectivity for [u'olt10', u'vsg10'] CASE4_Failure3 - FAIL
• IP connectivity failed
• 4.38 Connectivity for [u'olt5', u'vsg5'] CASE4_Failure3 - FAIL
• IP connectivity failed
• 4.39 Recovering ONOS instance - PASS
• 4.40 Checking if ONOS CLI is ready - PASS
• 4.41 Checking ONOS nodes - PASS
• 4.42 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 4.43 Check whether all flow status are ADDED - PASS
• 4.44 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE4_Recovery3 - FAIL
• IP connectivity failed
• 4.45 Connectivity for [u'olt10', u'vsg10'] CASE4_Recovery3 - FAIL
• IP connectivity failed
• 4.46 Connectivity for [u'olt5', u'vsg5'] CASE4_Recovery3 - FAIL
• IP connectivity failed

Case 5: High Availability tests - ONOS failures and Switch failures with 2x2 Leaf-spine - FAIL

• 5.1 Create and Apply cell file - PASS
• 5.2 Create and Install ONOS package - PASS
• 5.3 Starting ONOS service - PASS
• 5.4 Checking if ONOS CLI is ready - PASS
• 5.5 Starting Mininet Topology - PASS
• 5.6 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.7 Check whether all flow status are ADDED - PASS
• 5.8 Connectivity for [u'h1', u'h2', u'h3', u'h4'] - FAIL
• IP connectivity failed
• 5.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 5.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 5.11 Killing ONOS instance - PASS
• 5.12 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Failure1 - FAIL
• IP connectivity failed
• 5.13 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Failure1 - FAIL
• IP connectivity failed
• 5.14 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Failure1 - FAIL
• IP connectivity failed
• 5.15 Kill spine101 - PASS
• 5.16 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 5.17 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 5.18 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 5.19 Recovering spine101 - PASS
• 5.20 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.21 Check whether all flow status are ADDED - PASS
• 5.22 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 5.23 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 5.24 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 5.25 Recovering ONOS instance - PASS
• 5.26 Checking if ONOS CLI is ready - PASS
• 5.27 Checking ONOS nodes - PASS
• 5.28 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.29 Check whether all flow status are ADDED - PASS
• 5.30 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 5.31 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 5.32 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 5.33 Killing ONOS instance - PASS
• 5.34 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Failure2 - FAIL
• IP connectivity failed
• 5.35 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Failure2 - FAIL
• IP connectivity failed
• 5.36 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Failure2 - FAIL
• IP connectivity failed
• 5.37 Kill spine102 - PASS
• 5.38 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 5.39 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 5.40 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 5.41 Recovering spine102 - PASS
• 5.42 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.43 Check whether all flow status are ADDED - PASS
• 5.44 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 5.45 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 5.46 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 5.47 Recovering ONOS instance - PASS
• 5.48 Checking if ONOS CLI is ready - PASS
• 5.49 Checking ONOS nodes - PASS
• 5.50 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.51 Check whether all flow status are ADDED - PASS
• 5.52 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 5.53 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 5.54 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 5.55 Killing ONOS instance - PASS
• 5.56 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Failure3 - FAIL
• IP connectivity failed
• 5.57 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Failure3 - FAIL
• IP connectivity failed
• 5.58 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Failure3 - FAIL
• IP connectivity failed
• 5.59 Kill spine101 - PASS
• 5.60 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 5.61 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 5.62 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 5.63 Recovering spine101 - PASS
• 5.64 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.65 Check whether all flow status are ADDED - PASS
• 5.66 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 5.67 Connectivity for [u'olt10', u'vsg10'] CASE5_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 5.68 Connectivity for [u'olt5', u'vsg5'] CASE5_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 5.69 Recovering ONOS instance - PASS
• 5.70 Checking if ONOS CLI is ready - PASS
• 5.71 Checking ONOS nodes - PASS
• 5.72 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 5.73 Check whether all flow status are ADDED - PASS
• 5.74 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE5_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 5.75 Connectivity for [u'olt10', u'vsg10'] CASE5_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 5.76 Connectivity for [u'olt5', u'vsg5'] CASE5_ONOS_Recovery3 - FAIL
• IP connectivity failed

Case 6: High Availability tests - ONOS random failures and Switch random failures with 2x2 Leaf-spine - FAIL

• 6.1 Create and Apply cell file - PASS
• 6.2 Create and Install ONOS package - PASS
• 6.3 Starting ONOS service - PASS
• 6.4 Checking if ONOS CLI is ready - PASS
• 6.5 Starting Mininet Topology - PASS
• 6.6 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.7 Check whether all flow status are ADDED - PASS
• 6.8 Connectivity for [u'h1', u'h2', u'h3', u'h4'] - FAIL
• IP connectivity failed
• 6.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 6.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 6.11 Killing ONOS instance - PASS
• 6.12 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Failure1 - FAIL
• IP connectivity failed
• 6.13 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Failure1 - FAIL
• IP connectivity failed
• 6.14 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Failure1 - FAIL
• IP connectivity failed
• 6.15 Kill spine102 - PASS
• 6.16 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 6.17 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 6.18 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 6.19 Recovering spine102 - PASS
• 6.20 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.21 Check whether all flow status are ADDED - PASS
• 6.22 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 6.23 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 6.24 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 6.25 Recovering ONOS instance - PASS
• 6.26 Checking if ONOS CLI is ready - PASS
• 6.27 Checking ONOS nodes - PASS
• 6.28 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.29 Check whether all flow status are ADDED - PASS
• 6.30 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 6.31 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 6.32 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 6.33 Killing ONOS instance - PASS
• 6.34 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Failure2 - FAIL
• IP connectivity failed
• 6.35 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Failure2 - FAIL
• IP connectivity failed
• 6.36 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Failure2 - FAIL
• IP connectivity failed
• 6.37 Kill spine101 - PASS
• 6.38 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 6.39 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 6.40 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 6.41 Recovering spine101 - PASS
• 6.42 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.43 Check whether all flow status are ADDED - PASS
• 6.44 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 6.45 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 6.46 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 6.47 Recovering ONOS instance - PASS
• 6.48 Checking if ONOS CLI is ready - PASS
• 6.49 Checking ONOS nodes - PASS
• 6.50 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.51 Check whether all flow status are ADDED - PASS
• 6.52 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 6.53 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 6.54 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 6.55 Killing ONOS instance - PASS
• 6.56 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Failure3 - FAIL
• IP connectivity failed
• 6.57 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Failure3 - FAIL
• IP connectivity failed
• 6.58 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Failure3 - FAIL
• IP connectivity failed
• 6.59 Kill spine101 - PASS
• 6.60 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 6.61 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 6.62 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 6.63 Recovering spine101 - PASS
• 6.64 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.65 Check whether all flow status are ADDED - PASS
• 6.66 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 6.67 Connectivity for [u'olt10', u'vsg10'] CASE6_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 6.68 Connectivity for [u'olt5', u'vsg5'] CASE6_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 6.69 Recovering ONOS instance - PASS
• 6.70 Checking if ONOS CLI is ready - PASS
• 6.71 Checking ONOS nodes - PASS
• 6.72 Check whether the flow count is bigger than 116 - FAIL
• Flow count looks wrong: False
• 6.73 Check whether all flow status are ADDED - PASS
• 6.74 Connectivity for [u'h1', u'h2', u'h3', u'h4'] CASE6_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 6.75 Connectivity for [u'olt10', u'vsg10'] CASE6_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 6.76 Connectivity for [u'olt5', u'vsg5'] CASE6_ONOS_Recovery3 - FAIL
• IP connectivity failed

Case 7: High Availability tests - ONOS failures and Switch failures with 4x4 Leaf-spine - FAIL

• 7.1 Create and Apply cell file - PASS
• 7.2 Create and Install ONOS package - PASS
• 7.3 Starting ONOS service - PASS
• 7.4 Checking if ONOS CLI is ready - PASS
• 7.5 Starting Mininet Topology - PASS
• 7.6 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.7 Check whether all flow status are ADDED - PASS
• 7.8 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] - FAIL
• IP connectivity failed
• 7.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 7.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 7.11 Killing ONOS instance - PASS
• 7.12 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Failure1 - FAIL
• IP connectivity failed
• 7.13 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Failure1 - FAIL
• IP connectivity failed
• 7.14 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Failure1 - FAIL
• IP connectivity failed
• 7.15 Kill spine101 - PASS
• 7.16 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 7.17 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 7.18 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 7.19 Recovering spine101 - PASS
• 7.20 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.21 Check whether all flow status are ADDED - PASS
• 7.22 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 7.23 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 7.24 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 7.25 Recovering ONOS instance - PASS
• 7.26 Checking if ONOS CLI is ready - PASS
• 7.27 Checking ONOS nodes - PASS
• 7.28 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.29 Check whether all flow status are ADDED - PASS
• 7.30 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 7.31 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 7.32 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 7.33 Killing ONOS instance - PASS
• 7.34 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Failure2 - FAIL
• IP connectivity failed
• 7.35 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Failure2 - FAIL
• IP connectivity failed
• 7.36 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Failure2 - FAIL
• IP connectivity failed
• 7.37 Kill spine102 - PASS
• 7.38 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 7.39 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 7.40 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 7.41 Recovering spine102 - PASS
• 7.42 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.43 Check whether all flow status are ADDED - PASS
• 7.44 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 7.45 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 7.46 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 7.47 Recovering ONOS instance - PASS
• 7.48 Checking if ONOS CLI is ready - PASS
• 7.49 Checking ONOS nodes - PASS
• 7.50 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.51 Check whether all flow status are ADDED - PASS
• 7.52 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 7.53 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 7.54 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 7.55 Killing ONOS instance - PASS
• 7.56 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Failure3 - FAIL
• IP connectivity failed
• 7.57 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Failure3 - FAIL
• IP connectivity failed
• 7.58 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Failure3 - FAIL
• IP connectivity failed
• 7.59 Kill spine103 - PASS
• 7.60 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 7.61 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 7.62 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 7.63 Recovering spine103 - PASS
• 7.64 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.65 Check whether all flow status are ADDED - PASS
• 7.66 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 7.67 Connectivity for [u'olt10', u'vsg10'] CASE7_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 7.68 Connectivity for [u'olt5', u'vsg5'] CASE7_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 7.69 Recovering ONOS instance - PASS
• 7.70 Checking if ONOS CLI is ready - PASS
• 7.71 Checking ONOS nodes - PASS
• 7.72 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 7.73 Check whether all flow status are ADDED - PASS
• 7.74 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE7_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 7.75 Connectivity for [u'olt10', u'vsg10'] CASE7_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 7.76 Connectivity for [u'olt5', u'vsg5'] CASE7_ONOS_Recovery3 - FAIL
• IP connectivity failed

Case 8: High Availability tests - ONOS random failures and Switch random failures with 4x4 Leaf-spine - FAIL

• 8.1 Create and Apply cell file - PASS
• 8.2 Create and Install ONOS package - PASS
• 8.3 Starting ONOS service - PASS
• 8.4 Checking if ONOS CLI is ready - PASS
• 8.5 Starting Mininet Topology - PASS
• 8.6 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.7 Check whether all flow status are ADDED - PASS
• 8.8 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] - FAIL
• IP connectivity failed
• 8.9 Connectivity for [u'olt10', u'vsg10'] - FAIL
• IP connectivity failed
• 8.10 Connectivity for [u'olt5', u'vsg5'] - FAIL
• IP connectivity failed
• 8.11 Killing ONOS instance - PASS
• 8.12 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Failure1 - FAIL
• IP connectivity failed
• 8.13 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Failure1 - FAIL
• IP connectivity failed
• 8.14 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Failure1 - FAIL
• IP connectivity failed
• 8.15 Kill spine102 - PASS
• 8.16 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 8.17 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 8.18 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Failure1 - FAIL
• IP connectivity failed
• 8.19 Recovering spine102 - PASS
• 8.20 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.21 Check whether all flow status are ADDED - PASS
• 8.22 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 8.23 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 8.24 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Recovery1 - FAIL
• IP connectivity failed
• 8.25 Recovering ONOS instance - PASS
• 8.26 Checking if ONOS CLI is ready - PASS
• 8.27 Checking ONOS nodes - PASS
• 8.28 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.29 Check whether all flow status are ADDED - PASS
• 8.30 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 8.31 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 8.32 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 8.33 Killing ONOS instance - PASS
• 8.34 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Failure2 - FAIL
• IP connectivity failed
• 8.35 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Failure2 - FAIL
• IP connectivity failed
• 8.36 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Failure2 - FAIL
• IP connectivity failed
• 8.37 Kill spine104 - PASS
• 8.38 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 8.39 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 8.40 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Failure2 - FAIL
• IP connectivity failed
• 8.41 Recovering spine104 - PASS
• 8.42 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.43 Check whether all flow status are ADDED - PASS
• 8.44 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 8.45 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 8.46 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Recovery2 - FAIL
• IP connectivity failed
• 8.47 Recovering ONOS instance - PASS
• 8.48 Checking if ONOS CLI is ready - PASS
• 8.49 Checking ONOS nodes - PASS
• 8.50 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.51 Check whether all flow status are ADDED - PASS
• 8.52 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 8.53 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 8.54 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 8.55 Killing ONOS instance - PASS
• 8.56 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Failure3 - FAIL
• IP connectivity failed
• 8.57 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Failure3 - FAIL
• IP connectivity failed
• 8.58 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Failure3 - FAIL
• IP connectivity failed
• 8.59 Kill spine104 - PASS
• 8.60 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 8.61 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 8.62 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Failure3 - FAIL
• IP connectivity failed
• 8.63 Recovering spine104 - PASS
• 8.64 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.65 Check whether all flow status are ADDED - PASS
• 8.66 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 8.67 Connectivity for [u'olt10', u'vsg10'] CASE8_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 8.68 Connectivity for [u'olt5', u'vsg5'] CASE8_SWITCH_Recovery3 - FAIL
• IP connectivity failed
• 8.69 Recovering ONOS instance - PASS
• 8.70 Checking if ONOS CLI is ready - PASS
• 8.71 Checking ONOS nodes - PASS
• 8.72 Check whether the flow count is bigger than 350 - FAIL
• Flow count looks wrong: False
• 8.73 Check whether all flow status are ADDED - PASS
• 8.74 Connectivity for [u'h1', u'h2', u'h3', u'h4', u'h5', u'h6', u'h7', u'h8'] CASE8_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 8.75 Connectivity for [u'olt10', u'vsg10'] CASE8_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 8.76 Connectivity for [u'olt5', u'vsg5'] CASE8_ONOS_Recovery3 - FAIL
• IP connectivity failed