...

export JAVA_OPTS="-Dds.lock.timeout.milliseconds=15000"

Profiling

Profiling is critical to tuning ONOS for production deployments. Indeed, the above recommendations were based on extensive profiling of ONOS deployments. We recommend YourKit Java Profiler for profiling ONOS.

To enable YourKit profiling, download the YourKit agent from the YourKit webset and modify the ONOS environment variables to start the JVM with the YourKit agent:

export JAVA_OPTS="-agentpath:/path/to/libyjpagent.so=listen=all"

By default, the YourKit agent will bind to port 10001, so ensure that port is open.

Once an ONOS node is started, open YourKit and select the + symbol to add a new remote application:

Image Added

Enter the IP of the ONOS node and optionally assign a name to the connection:

Image Added

Once the connection has been added, select the connection to connect to the YourKit agent:

Image Added

Once the connection is established, the full YourKit profiler view will be opened, including CPU, memory, garbage collection, events, and other statistics. Note: by default CPU usage is sampled. To gather more precise CPU usage statistics, enable CPU tracing.

Sharing Snapshots

The YourKit profiler provides a variety of valuable views for analyzing and debugging the performance of ONOS deployments. But we've found its most invaluable feature to be memory snapshots. YourKit allows users to take a complete snapshot of memory allocations, CPU usage, garbage collection, and all other statistics at a point in time. These snapshots can then be shared with other ONOS developers. Indeed, when reporting a memory leak or debugging a performance issue, ONOS developers may request a YourKit memory snapshot.

To take a memory snapshot, simply click the memory snapshot button in the toolbar:

Image Added

The snapshot will be captured and saved to the client's machine. By default, snapshots will be saved in ~/Snapshots. Snapshots can then be sent to other developers for review and collaboration.