Master - HA

ONOS-HA

**************************************

Jenkins test result for 02:53 on Jan 29, 2015.

Test Plan for HA Test Cases

HATestSanity - Results: 25 Passed, 0 Failed

HATestMinorityRestart - Results: 19 Passed, 6 Failed

HATestClusterRestart - Results: 14 Passed, 12 Failed

HATestSingleInstanceRestart - Results: 26 Passed, 0 Failed

Test HATestSanity

************************************

Result summary for Testcase1

ONOS HA Sanity test - initialization

Removing raft logs

Uninstalling ONOS

commit 35c71e12ee5201f1a2ed041b8b855cce378a4d9b (HEAD, origin/master, origin/HEAD, master)

Author: Ayaka Koshibe [ayaka@onlab.us]

AuthorDate: Tue Jan 27 17:10:04 2015 -0800

Commit: Gerrit Code Review [gerrit@onlab.us]

CommitDate: Thu Jan 29 04:50:07 2015 +0000

NullDeviceProvider now takes configurations for :

Result: Pass

Result summary for Testcase2

Assigning switches to controllers

Switch mastership assigned correctly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.4822468758 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase3

Adding host intents

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase5

Setting up and gathering data for current state

Switch roles are consistent across all ONOS nodes

Intents are consistent across all ONOS nodes

Flow count is consistent across all ONOS nodes

Result: Pass

Result summary for Testcase14

Leadership election tests passed( consistent view of leader across listeners and a leader was elected )

Result: Pass

Result summary for Testcase6

Wait 60 seconds instead of inducing a failure

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.2718119621 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase7

Switch roles are consistent across all ONOS nodes

Mastership of Switches was not changed

Intents are consistent across all ONOS nodes

Intents are consistent with before failure

No changes were found in the flow tables

No loss of dataplane connectivity

Leadership election tests passed( consistent view of leader across listeners and a new leader was re-elected if applicable )

Constant State Tests Passed

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase15

Check that Leadership Election is still functional

Leadership election tests passed( consistent view of leader across listeners and a new leader was elected when the old leader resigned )

Result: Pass

Result summary for Testcase9

Turn off a link to ensure that Link Discovery is working properly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 15 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 15.987598896 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase10

Restore a link to ensure that Link Discovery is working properly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.266122818 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase11

Killing a switch to ensure it is discovered correctly

Deleting s5

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.0970180035 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase12

Adding a switch to ensure it is discovered correctly

Adding back s5

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 15 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 15.9584929943 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase13

Test Cleanup

Result: Pass

Test HATestMinorityRestart

************************************

Result summary for Testcase1

ONOS HA test: Restart minority of ONOS nodes - initialization

Removing raft logs

Uninstalling ONOS

commit 35c71e12ee5201f1a2ed041b8b855cce378a4d9b (HEAD, origin/master, origin/HEAD, master)

Author: Ayaka Koshibe [ayaka@onlab.us]

AuthorDate: Tue Jan 27 17:10:04 2015 -0800

Commit: Gerrit Code Review [gerrit@onlab.us]

CommitDate: Thu Jan 29 04:50:07 2015 +0000

NullDeviceProvider now takes configurations for :

Result: Pass

Result summary for Testcase2

Assigning switches to controllers

Switch mastership assigned correctly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.5745100975 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase3

Adding host intents

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase5

Setting up and gathering data for current state

Switch roles are consistent across all ONOS nodes

Intents are consistent across all ONOS nodes

Flow count is consistent across all ONOS nodes

Result: Pass

Result summary for Testcase14

Leadership election tests passed( consistent view of leader across listeners and a leader was elected )

Result: Pass

Result summary for Testcase6

Killing 3 ONOS nodes

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.3640120029 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase7

Switch roles are consistent across all ONOS nodes

Mastership of some switches changed

Intents are consistent across all ONOS nodes

The Intents changed during failure

Changes were found in the flow tables

Loss of dataplane connectivity detected

Leadership election tests passed( consistent view of leader across listeners and a new leader was re-elected if applicable )

Constant state tests failed

Result: Failed

Result summary for Testcase4

Ping across added host intents

Intents have not been installed correctly, pings failed.

Result: Failed

Result summary for Testcase15

Check that Leadership Election is still functional

Leadership election tests passed( consistent view of leader across listeners and a new leader was elected when the old leader resigned )

Result: Pass

Result summary for Testcase9

Turn off a link to ensure that Link Discovery is working properly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.4471559525 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have not been installed correctly, pings failed.

Result: Failed

Result summary for Testcase10

Restore a link to ensure that Link Discovery is working properly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.3059709072 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have not been installed correctly, pings failed.

Result: Failed

Result summary for Testcase11

Killing a switch to ensure it is discovered correctly

Deleting s5

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.1795449257 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have not been installed correctly, pings failed.

Result: Failed

Result summary for Testcase12

Adding a switch to ensure it is discovered correctly

Adding back s5

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 16 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 16.5209269524 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have not been installed correctly, pings failed.

Result: Failed

Result summary for Testcase13

Test Cleanup

Result: Pass

Test HATestClusterRestart

************************************

Result summary for Testcase1

ONOS HA test: Restart all ONOS nodes - initialization

Removing raft logs

Uninstalling ONOS

commit 0c8be2c6e16adf7a743bd7085918446223dce209 (HEAD, origin/onos-1.0, onos-1.0)

Author: Marc De Leenheer [marc@onlab.us]

AuthorDate: Thu Jan 22 14:50:04 2015 -0800

Commit: Gerrit Code Review [gerrit@onlab.us]

CommitDate: Wed Jan 28 22:15:27 2015 +0000

Fix rewriting of LINC-OE config for long controller IPs (ONOS-834).

Result: Pass

Result summary for Testcase2

Assigning switches to controllers

Switch mastership assigned correctly

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Very crass estimate for topology discovery/convergence( note it takes about 19 seconds for the test to make all the cli calls to fetch the topology from each ONOS instance ): 19.8666329384 seconds, 1 tries

ONOS topology view matches Mininet topology

Result: Pass

Result summary for Testcase3

Adding host intents

Result: Pass

Result summary for Testcase4

Ping across added host intents

Intents have been installed correctly and verified by pings

Result: Pass

Result summary for Testcase5

Setting up and gathering data for current state

Switch roles are consistent across all ONOS nodes

Intents are consistent across all ONOS nodes

Flow count is consistent across all ONOS nodes

Result: Pass

Result summary for Testcase14

Leadership election tests passed( consistent view of leader across listeners and a leader was elected )

Result: Pass

Result summary for Testcase6

Restart entire ONOS cluster

Result: Pass

Result summary for Testcase8

Compare ONOS Topology view to Mininet topology

Switches in MN but not in ONOS:

['0000000000001000', '0000000000003000', '0000000000003004', '0000000000003012', '0000000000003014', '0000000000003015', '0000000000006000', '0000000000006007', '0000000000006018', '0000000000006019', '0000000000006020', '0000000000006022', '0000000000006023', '0000000000006024', '0000000000006025', '0000000000006026', '0000000000006027']

Switches in ONOS but not in MN:

[]

ONOS1 Switches view is incorrect

The list of ports for switch s7(0000000000006007) does not match:

The list of ports for switch s1(0000000000001000) does not match:

The list of ports for switch s14(0000000000003014) does not match:

The list of ports for switch s19(0000000000006019) does not match:

The list of ports for switch s18(0000000000006018) does not match:

The list of ports for switch s12(0000000000003012) does not match:

The list of ports for switch s15(0000000000003015) does not match:

The list of ports for switch s4(0000000000003004) does not match:

The list of ports for switch s20(0000000000006020) does not match:

The list of ports for switch s23(0000000000006023) does not match:

The list of ports for switch s6(0000000000006000) does not match:

The list of ports for switch s27(0000000000006027) does not match:

The list of ports for switch s22(0000000000006022) does not match:

The list of ports for switch s3(0000000000003000) does not match:

The list of ports for switch s26(0000000000006026) does not match:

The list of ports for switch s24(0000000000006024) does not match:

The list of ports for switch s25(0000000000006025) does not match:

ONOS1 ports view is incorrect

Mininet has 29 bidirectional links and ONOS has 13 unidirectional links

ONOS does not have the link 0000000000003004/11 -> 0000000000003016/2

ONOS does not have the link 0000000000003016/2 -> 0000000000003004/11

ONOS does not have the link 0000000000003004/12 -> 0000000000003017/2

ONOS does not have the link 0000000000003017/2 -> 0000000000003004/12

ONOS does not have the link 0000000000005000/4 -> 0000000000006000/2

ONOS does not have the link 0000000000006000/3 -> 0000000000002800/3

ONOS does not have the link 0000000000002800/3 -> 0000000000006000/3

ONOS does not have the link 0000000000006000/4 -> 0000000000006007/2

ONOS does not have the link 0000000000006007/2 -> 0000000000006000/4

ONOS does not have the link 0000000000006018/2 -> 0000000000006007/3

ONOS does not have the link 0000000000006007/4 -> 0000000000006019/2

ONOS does not have the link 0000000000006019/2 -> 0000000000006007/4

ONOS does not have the link 0000000000006007/5 -> 0000000000006020/2

ONOS does not have the link 0000000000006020/2 -> 0000000000006007/5

ONOS does not have the link 0000000000006021/2 -> 0000000000006007/6

ONOS does not have the link 0000000000006007/7 -> 0000000000006022/2

ONOS does not have the link 0000000000006022/2 -> 0000000000006007/7

ONOS does not have the link 0000000000006023/2 -> 0000000000006007/8

ONOS does not have the link 0000000000006007/9 -> 0000000000006024/2

ONOS does not have the link 0000000000006024/2 -> 0000000000006007/9

ONOS does not have the link 0000000000006007/10 -> 0000000000006025/2

ONOS does not have the link 0000000000006025/2 -> 0000000000006007/10

ONOS does not have the link 0000000000006007/11 -> 0000000000006026/2

ONOS does not have the link 0000000000006026/2 -> 0000000000006007/11

ONOS does not have the link 0000000000006007/12 -> 0000000000006027/2

ONOS does not have the link 0000000000006027/2 -> 0000000000006007/12

ONOS does not have the link 0000000000005000/3 -> 0000000000001000/3

ONOS does not have the link 0000000000002000/2 -> 0000000000005000/2

ONOS does not have the link 0000000000002000/4 -> 0000000000003000/2

ONOS does not have the link 0000000000003000/2 -> 0000000000002000/4

ONOS does not have the link 0000000000003004/9 -> 0000000000003014/2

ONOS does not have the link 0000000000003014/2 -> 0000000000003004/9

ONOS does not have the link 0000000000003000/3 -> 0000000000002800/2

ONOS does not have the link 0000000000002800/2 -> 0000000000003000/3

ONOS does not have the link 0000000000003000/4 -> 0000000000003004/2

ONOS does not have the link 0000000000003008/2 -> 0000000000003004/3

ONOS does not have the link 0000000000003009/2 -> 0000000000003004/4

ONOS does not have the link 0000000000003004/5 -> 0000000000003010/2

ONOS does not have the link 0000000000003010/2 -> 0000000000003004/5

ONOS does not have the link 0000000000003011/2 -> 0000000000003004/6

ONOS does not have the link 0000000000003012/2 -> 0000000000003004/7

ONOS does not have the link 0000000000003013/2 -> 0000000000003004/8

ONOS does not have the link 0000000000001000/2 -> 0000000000002000/3

ONOS does not have the link 0000000000002000/3 -> 0000000000001000/2

ONOS does not have the link 0000000000003015/2 -> 0000000000003004/10

ONOS1 links view is incorrect

Switches in MN but not in ONOS:

['0000000000001000', '0000000000003000', '0000000000003004', '0000000000003012', '0000000000003014', '0000000000003015', '0000000000006000', '0000000000006007', '0000000000006018', '0000000000006019', '0000000000006020', '0000000000006022', '0000000000006023', '0000000000006024', '0000000000006025', '0000000000006026', '0000000000006027']

Switches in ONOS but not in MN:

[]

ONOS2 Switches view is incorrect