<img src="https://jenkins.onosproject.org/view/QA/job/postjob-Fabric3/lastSuccessfulBuild/artifact/SRHighAvailability_onos-2.2_20-builds_graph.jpg", alt="SRHighAvailability", style="width:525px;height:350px;border:0">

commit 1d4756928df452b61a0d51919d9aab80b7447bfe (HEAD -] onos-2.2, origin/onos-2.2)
Author: Robert MacDavid [robertmacdavid@gmail.com]
AuthorDate: Thu May 21 21:32:38 2020 -0400
Commit: Carmelo Cascone [carmelo@opennetworking.org]
CommitDate: Fri May 29 02:30:01 2020 +0000

Rewrite the SPGW P4 pipeline.
--
This commit makes two changes to the fabric.p4 source files:
1. Routing is now done on a metadata field instead of the ipv4 header.
Routing on the ipv4 header is incorrect if the packet is encapsulated
with an outer IPV4 header. The metadata field is updated each time the
ipv4 destination changes, or an outer header is added or removed. These
changes require no control plane modifications due to @name annotations.
2. The spgw control blocks have been rewritten to support the logical UP4
--
(cherry picked from commit bec6b6a7b0f33812c6f6904d9f9b629539446e79)

Case 1: High Availability tests - ONOS failures 2x2 Leaf-spine - FAIL

• 1.1 Constructing test variables - PASS
• 1.2 Apply cell to environment - PASS
• 1.3 Uninstalling Atomix - PASS
• 1.4 Uninstalling ONOS package - PASS
• 1.5 Creating ONOS package - PASS
• 1.6 Installing Atomix - PASS
• 1.7 Installing ONOS package - PASS
• 1.8 Set up ONOS secure SSH - PASS
• 1.9 Checking ONOS service - PASS
• 1.10 Starting ONOS CLI sessions - PASS
• 1.11 Checking ONOS nodes - PASS
• 1.12 Checking ONOS applications - PASS
• 1.13 Starting Mininet Topology - PASS
• 1.14 Check whether the flow count is bigger than 116 - PASS
• 1.15 Check whether all flow status are ADDED - PASS
• 1.16 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1 - FAIL
• IP connectivity failed
• 1.17 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1 - FAIL
• IP connectivity failed
• 1.18 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1 - FAIL
• IP connectivity failed
• 1.19 Killing ONOS instances with index(es): [0] - PASS
• 1.20 Check number of topology elements - FAIL
• Unexpected number of links, switches, and/or controllers
• 1.21 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Failure1 - FAIL
• IP connectivity failed
• 1.22 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Failure1 - FAIL
• IP connectivity failed
• 1.23 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Failure1 - FAIL
• IP connectivity failed
• 1.24 Recovering ONOS instances with index(es): [0] - PASS
• 1.25 Checking if ONOS CLI is ready - PASS
• 1.26 Checking ONOS nodes - PASS
• 1.27 Check number of topology elements - PASS
• 1.28 Check whether the flow count is bigger than 116 - PASS
• 1.29 Check whether all flow status are ADDED - PASS
• 1.30 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 1.31 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 1.32 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Recovery1 - FAIL
• IP connectivity failed
• 1.33 Killing ONOS instances with index(es): [1] - PASS
• 1.34 Check number of topology elements - FAIL
• Unexpected number of links, switches, and/or controllers
• 1.35 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Failure2 - FAIL
• IP connectivity failed
• 1.36 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Failure2 - FAIL
• IP connectivity failed
• 1.37 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Failure2 - FAIL
• IP connectivity failed
• 1.38 Recovering ONOS instances with index(es): [1] - PASS
• 1.39 Checking if ONOS CLI is ready - PASS
• 1.40 Checking ONOS nodes - PASS
• 1.41 Check number of topology elements - PASS
• 1.42 Check whether the flow count is bigger than 116 - PASS
• 1.43 Check whether all flow status are ADDED - PASS
• 1.44 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 1.45 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 1.46 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Recovery2 - FAIL
• IP connectivity failed
• 1.47 Killing ONOS instances with index(es): [2] - PASS
• 1.48 Check number of topology elements - FAIL
• Unexpected number of links, switches, and/or controllers
• 1.49 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Failure3 - FAIL
• IP connectivity failed
• 1.50 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Failure3 - FAIL
• IP connectivity failed
• 1.51 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Failure3 - FAIL
• IP connectivity failed
• 1.52 Recovering ONOS instances with index(es): [2] - PASS
• 1.53 Checking if ONOS CLI is ready - PASS
• 1.54 Checking ONOS nodes - PASS
• 1.55 Check number of topology elements - PASS
• 1.56 Check whether the flow count is bigger than 116 - PASS
• 1.57 Check whether all flow status are ADDED - PASS
• 1.58 Verify full connectivity for [u'h1', u'h2', u'h3', u'h4'] with tag CASE1_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 1.59 Verify full connectivity for [u'olt10', u'vsg10'] with tag CASE1_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 1.60 Verify full connectivity for [u'olt5', u'vsg5'] with tag CASE1_ONOS_Recovery3 - FAIL
• IP connectivity failed
• 1.61 Stopping Mininet - PASS
• 1.62 Copying karaf logs - PASS

Case 2: High Availability tests - ONOS random failures 2x2 Leaf-spine - FAIL