swx-top

Prev Next

WHAT TO EXPECT

swxtch-top is one of the utility applications included in xNIC installation. It can be run from the console of any VM that has the xNIC software installed, displaying real-time statistics of an attached cloudSwXtch instance. This includes data regarding mesh, high availability, multicast and PTP.

In this article, you will learn how to navigate through the different pages in swxtch-top and get better visibility on how data flows in your cloudSwXtch instance. 

Running swxtch-top 

Depending on your operating system, you can use certain commands to run swxtch-top on your VM. 

For both Windows and Linux agents (xNICs), users can enter the following into the terminal: 

swxtch-top --swxtch <cloudSwXtch-IP>

Example: 

swxtch-top --swxtch 10.5.1.6

From the cloudSwXtch, users can enter the following command: 

swxtch-top

Updating cloudSwXtch for swXtch-top

The following screenshots have been taken on the latest version of cloudSwXtch. To learn how to upgrade your cloudSwXtch, please see the article, Upgrading cloudSwXtch.

Navigating swxtch-top Dashboard

swxtch-top - components-map

The swxtch-top dashboard is organized into 3 panels as shown in the screenshot above. While the top panel will remain static, displaying information regarding the cloudSwXtch, the bottom 2 panels will change depending on the selected view. The swxtch-top dashboard has 12 different views:

  1. Components (Default)

  2. Streams

  3. StreamLinks

  4. Adaptors

  5. HA

  6. Lossless

  7. Aliases

  8. PTP

  9. Subscriptions

  10. Notifications

  11. Configuration

  12. Licensing

Switching Between swXtch-top Views

To switch between views, click on the Views option in the lefthand corner of swXtch-top and select the desired view from the menu. Alternatively, users can press CTRL + V to open the menu and then CTRL + the highlighted letter to switch to that view. For example, to toggle to “StreamLinks",” enter CTRL + V and CTRL + K.

Note: Opening the menu with the CTRL key is not necessary if you know the highlighted letter for your desired view. Simply entering CTRL + the highlighted letter will open the view.

swxtch-top_components_views

Using the Setup Function

Users can modify what columns are visible in each of the views and the width of those columns. Please note: These changes will persist between swxtch-top sessions.

To do this:

  1. Select the [+] next to the first column. This will open the Setup menu for that specific panel.

    1. Alternatively, you can click a panel to select it and press F3 to open the Setup menu. You can tell you selected panel by the increased thickness of its border.

      swxtch-top-setup-box

  2. Click one of the blue checkboxes under Visible to highlight the column you wish to add. An X will appear.

    1. You can also change the width of the columns in swXtch-top in this panel.

  3. Hit OK to confirm your selection.

    swxtch-top - components_setup_cloud selected

    The new column should now appear in the swxtch-top view. In this example, the Cloud column was added by the user and is now displayed in the Components view.

    swxtch-top - components_setup_cloud added

Using Setup to Toggle Raw Totals On/Off

By default, swXtch-top will round the total of bits and packets into a simplified number. For example, instead of 1496892, swxtch-top will write 1.5 M (million). To toggle this on/off, go to the Setup menu and check the column that you would like the raw totals for and select OK.

The raw total should appear in a few seconds for the entire column.

Changing swXtch-top Colors

Users can change the colors displayed in swxtch-top by keying in different hex codes in the Colors Set-Up menu.

To do this:

  1. Click Colors at the bottom of swxtch-top or enter CTRL + R.

  2. Select Custom from the Scheme dropdown menu.

  3. Enter the desired hex codes into one of the fields.

  4. Click Confirm when you are happy with your color selections.

  5. swxtch-top should now display the assigned colors.

Exporting Support Logs

swxtch-top_support

The swXtch-top Support menu allows users to export cloudSwXtch logs, similar to the functionality found in the wXcked Eye Support page.

To navigate to this menu, you can enter CTRL + T or select Support in the bottom navigation bar.

Selecting one of the report options will prompt users to enter parameters as shown in the example below. The report will save on the VM’s user directory.

swxtch-top - support_swxtch_info_plain_text

Panel 1: Information

swXtch_top_Components-Panel1

The first panel of the swxtch-top dashboard provides users with information regarding their cloudSwXtch as well as their subscription plan. Each cloud provider will have alternative titles for some of the listed items but for the most part, the information is the same.

On the left side of the section:

  • cloudSwXtch name

  • Cloud (Azure, AWS, GCP, OCI)

  • SubscriptionID (Azure), Account ID (AWS), or ID (GCP)

  • ResourceGroupName (Azure) or Region (AWS and GCP)

  • SwXtchID

  • Status

On the right side of the section:

  • Licensing Type

  • Bandwidth Allotment

  • Number of Clients and Bridges

  • License Expiration Date

The Information panel is fully collapsible by either selecting the (+) or (-) buttons. Users can also open and close the panel using the F2 key on their keyboard. The collapsed view will only display the cloudSwXtch name, the status, the licensing type, and new notifications.

When there are new notifications under the Notifications view, a “NEW NOTIFICATIONS” message will also display in this box.

For more information regarding licensing, please read the cloudSwXtch System Requirements article.

Panel 2 and 3: Views

Panel 2 and 3 defaults to the Components view and is shown in the picture above. However, the display changes based on view selection. To change views, key in CTRL + the associated letter for that view. 

Please note: Not all views have a 3rd panel.  

Filter Bar

At the top of each view’s main panel, there is a white search “Filter” bar. This allows users to filter through the contents in the current view. Simply type the information you would like to filter the list by, followed by an asterisk (*), and hit Enter. For example, v1* to see all the fields that have “v1” in in.

swxtch-top-components-filter

Components

swXtch_top_Components

The Components view gives a detailed list of all the connected nodes in a cloudSwXtch’s network. To navigate to this view, press CTRL+E or click on the view’s name. To switch between components, use the up and down directional keys on your keyboard.

This view includes:

  • Name - The name of the component

  • Class - Either cloudSwXtch, xNIC, or VM (non-xNIC)

  • Type - X1 or X2 for cloudSwXtch or Type 1 or 2 for xNIC

  • RX bps - The total ingress bits per second that the component is receiving.

  • RX pps - The total ingress packets per second that the component is receiving.

  • TX bps - The total egress bits per second that the component is transmitting.

  • TX pps - The total egress packets per second that the xNIC is transmitting.

  • Primary IP - The primary interface IP address

  • OS - Operating System

  • Primary Swxtch - This lists the cloudSwXtch the xNIC was installed against

There are additional column options in the Setup menu:

An Interfaces panel displays when in the Component view, detailing the interfaces of a selected component. This panel includes the following information for each interface:

Filtering Out Virtual Machines

Next to the Filter search bar, there is a checkbox with virtual machines (VMs) selected. This means that non-xNIC virtual machines will display in the components list. If a user wishes to hide these virtual machines and just focus on swXtch.io-based components, they can deselect this VM filter.

swXtch_top_Components-VMs

Streams

swXtch_top_Streams

This tree view shows all the multicast groups that are being received and/or transmitted by the cloudSwXtch and how to interacts with the various endpoints. To navigate to this view, enter CTRL + S or click on the view’s name.

This view includes:

  • Stream - The name of the stream IP and Port. This could also be an Alias set in wXcked Eye. For example, BaseballCamera1 and BaseballCamera2 were Aliases assigned in wXcked Eye.

  • Components: swxtch, xNIC or VM (Virtual Machine w/o xNIC)

  • Protocol: Multicast or Broadcast

  • RX Pkts: The total ingress packets per second being received by the component.

  • RX Bits: The total ingress bits per second that is received by the component.

  • TX Pkts: The total egress packets per second that is transmitted by the component.

  • TX Bits: The total egress bits per second that that is transmitted by the component.

In the example above, stream 224.2.2.3:8400 is listed. It is being transmitted by an endpoint (v1-u22-172-31-64-6) to the cloudSwXtch (v1-c2-172-31-64-14), which is then sent to the other endpoints. That is why there are three additional endpoints listed under the cloudSwXtch.

Stream Links

swxtch-top_streamLinks

The Stream Links view displays all the cloudSwXtch network components (cloudSwXtch or xNIC) and the streams they are linked to. To select this view, press CTRL + K or click on the view’s name. To expand a component, simply use the up and down keys on the keyboard and hit ENTER to select.

In this example, the cloudSwXtch (dsd-core-200) is selected with the singular stream 239.1.1.1:3490 expanded. Here, a user can see a list of endpoints receiving/transmitting the stream to/from the cloudSwXtch, dsd-core-200.

  • Components: cloudSwXtch or endpoints (xNICs)

  • Direction: Either Ingress or Egress, depending on whether data is being ingested or transmitted by the component

  • Protocol: Multicast or Broadcast

  • Bits (bps): Bits per second

  • Pkts (pps): Packets per second

  • Frag-Pkts (fpps): Fragmented packets per second

  • Frags/Pkts: Total # of Fragmented Packets

  • Missing Packets: Packets that did not make their destination

  • OOO Pkts: Out of order packets

Adaptors

swxtch-top_Adaptors

The Adaptors view displays information regarding Protocol Conversion and Fanout. It includes a detailed list of configured protocols (UDP, SRT Caller/Listener, or RIST Caller/Listener), their direction (ingress/egress), the stream IP or stream name, the node, and Listener Port. Users can also add and delete adaptors. For more information, see the Configuring Adaptors for Protocol Fanout section.

To navigate to this view, enter CTRL + A or click on the view’s name.

HA (High Availability)

swXtch_top_Ha

The HA view shows additional details for high availability from the perspective of the consumer. It will only show data if High Availability has been configured.

To navigate to this view, enter CTRL + H or click on the view’s name. Go to High Availability for configuration details.

This view is organized into two sections: HA and Cluster.

  • HA: This section lists the xNIC components in the cloudSwXtch network that are consuming or producing HA paths. Expanding the consumer details will show the stream IP split into two paths (red and blue) and a reconstructed path.

  • Cluster: This section displays the high availability clusters and the associated cloudSwXtches for each path.

The statistics by each line item is further explained below:

  • Path Ingress pps - The total ingress packets per second that is received in the path for the multicast group.

  • Path Ingress bps - The total ingress bits per second that is received in the path for the multicast group.

  • Path Usage % - The percentage that the path is used in the highly available multicast group.

  • Missing packets - The total number of missing packets for the path since the inception of the stream. If you stop the stream or cloudSwXtches, the number will stop increasing but will not reset.

Users can also create and destroy HA clusters in this view. For more information, see Configuring HA in swXtch-top.

Lossless

swxtch-top_lossless

The Lossless view shows statistics for data flow using swXtch Lossless UDP, or SLP, from ground to cloud through the cloudSwXtch Bridge.

To navigate to this view, enter CTRL + L or click on the view’s name.

By default, the view is organized into two sections: SLP Rx and SLP Tx. Both sections will populate with information, illustrating traffic between the cloudSwXtch Bridge and the cloudSwXtch.

By default, the following columns are displayed for SLP Rx:

  • RTT (ns): Round trip time in nanoseconds it takes for our connection to send a ping and receive a response. One way latency can be estimated by dividing this value in half.

  • RTT Variance: The variance of the RTT measurement. A high variance is indicative of an unstable connection.

  • Data Pkts: The total number of packets received (original and retransmitted)

  • Retr Pkts: The number of packets received as a a retransmitted packet.

  • Packets Lost: The total number of packets that were not recovered by the connection and are lost.

By default, the following columns are displayed for SLP Tx:

  • RTT (ns): The round trip time in nanoseconds it takes for our connection to send a ping and receive a response. One way latency can be estimated by dividing this value in half.

  • RTT Variance: The variance of the RTT measurement. A high variance is indicative of an unstable connection.

  • Data Pkts: The total number of packets transmitted (original and retransmitted)

  • Retr Pkts: The total number of retransmitted packets

Please note: Although the Setup feature has additional SLP-related statistics, they are mostly use for troubleshooting purposes by the swXtch.io. We recommend using the default columns above to monitor your SLP connection.

Aliases

swxtch-top -Aliases

The Aliases section provides users with a detailed list of user-assigned names for components and streams. It is split into two sections: Component Aliases and Stream Aliases.

For Component Aliases, the following columns will display:

  • Alias  

  • Primary IP

  • Hostname

  • Class

For Stream Aliases, there are only two columns: Alias and Stream.

To navigate to this view, enter CTRL + E or click on the view’s name.

Adding and Deleting Aliases

Users can add and delete aliases in swxtch-top. This can also be done in the Aliases tab on the wXcked Eye Settings page.

To add an alias:

  1. Click the Add button at the bottom of swxtch-top or hit F7 on your keyboard.

    1. A new window will open.

  2. Select the Alias type: Component or Stream.

    1. For Component, add an IP address and the assign an Alias name.

    2. For Stream, add a Stream IP address, Port and assign an Alias.

  3. Click Confirm once you are happy with your selections.

  4. The new Alias will now appear under the selected alias type.

To delete an alias:

  1. Highlight an Alias from either the Component or Stream Aliases list.

  2. Click Delete at the bottom of the swxtch-top window.

  3. Click Confirm.

  4. The Alias will disappear from list.

PTP

swxtch-top_PTP

The PTP view displays information regarding the clock sync configuration for the cloudSwXtch. The page in swxtch-top will only populate with information if the user has the PTP feature enabled.

To navigate to this view, enter CTRL + P or click on the view’s name.

In the example above, the cloudSwXtch is acting as the Master Node and is the first line item listed. Below the cloudSwXtch are the follower nodes.

  • Master Node- The Master Node is what the PTP configuration sets as the most reliable time source. This will send the true time it receives from the source clock to the Follower Nodes. 

    • Name - The name of the cloudSwXtch

    • Time Sync Service - The source clock

  • Follower Nodes- The Follower Nodes lists the agents/VMs that subscribe to the Master Node for accurate timing. 

    • Name - The name of the endpoints 

    • Status - The status of the endpoints, noting if the node is active in the PTP configuration

    • Local Offset - The local offset denotes the offset in time from the cloudSwXtch to the xNIC. 

    • Root Offset - The root offset denotes the offset in time from the GrandMaster clock to the cloudSwXtch and its follower nodes (xNIC). Note how the root is larger than the local. This is normal behavior since the distance between the follower node and the Grandmaster clock is greater than the offset between a cloudSwXtch and xNIC.

PTP Stabilization 

After upgrading your cloudSwXtch system, you may notice that the local and root offset values are much larger than they actually are. It can take up to 30 minutes for the values to stabilize and return back to normal levels.  

Subscriptions

swxtch-top_subscriptions(1)

The Subscriptions page details the different source list configurations for single source multicast and multiple multicast groups. Here, users will get a detailed lists of included and excluded streams and the source they originate from.

To navigate to this view, enter CTRL + U or or click on the view’s name.

Notifications

swxtch-top_notifications

The Notifications view compiles a list of cloudSwXtch-related notifications.

To navigate to this view, enter CTRL + N or click on the view’s name.

A New Notifications alert will appear in the Information panel in swXtch-top as shown below. Visiting the Notifications view will remove this alert.

swXtch_top_Components-Notification

Configurations

swxtch-top_configurations

The Configurations view gives users a visualization of the xNICs’ configuration JSON files.

To navigate to this view, enter CTRL + F or click on the view’s name.

Licensing

swxtch-top_licensing

The Licensing view gives a breakdown of the entitlements allotted to each of the cloudSwXtches included in the network.

Depending on their license, users will have a set number for their Max Bandwidth, Max Clients and Max bridges. In the example above, the user has a max bandwidthMbps of 236bps with 20 clients max and 1 bridges max. This section will also show if a user has the following features enabled: HA, Fanout, Clock Sync (PTP), wXckedEye, TachyonLive, and Major Version Update.

In addition to the Licensing details, the Details panel lists the expiration date of the license and the license type for a selected cloudSwXtch.

To navigate to this view, enter CTRL + G or click on the view’s name.

Configuring Adaptors for Protocol Fanout

In the Adaptors view, users can configure adaptors for protocol fanout, a functionality that is also available in wXcked Eye. swXtch.io’s protocol fanout feature allows users to configure unicast (UDP, SRT, RIST) for the cloudSwXtch and the cloudSwXtch Bridge and fan it out to multiple instances as UDP, SRT, RIST, or multicast. For swXtch-top, users have the ability to both add and delete an adaptor.

Adding an Adaptor

  1. Navigate to the Adaptors view in swXtch-top.

  2. Highlight a component (cloudSwXtch or cloudSwXtch Bridge) that you would like to add an adaptor. You can do this either by clicking on a listed component or using the navigation up and down navigation keys.

  3. Press F7 or click the Add button at the bottom of swXtch-top.

    1. A new window will open.

  4. Enter the information for your adaptor. For an explainer on what each field means, see Protocol Conversion and Fanout with wXcked Eye.

  5. Select Confirm.

    swXtch_top_adaptor_add

  6. The new adaptor will list under the cloudSwXtch or cloudSwXtch Bridge component.

    swXtch_top_adaptor_after_add

Deleting an Adaptor

  1. Navigate to the Adaptors view in swXtch-top.

  2. Select an adaptor under a cloudSwXtch or cloudSwXtch Bridge component that you would like to delete. You can do this either by clicking on a listed component or using the navigation up and down navigation keys.

  3. Press F7 or click the delete button at the bottom of swXtch-top.

    1. A new window will open.

      swXtch_top_adaptor_delete

  4. Confirm you wish to delete the selected adaptor.

    swXtch_top_adaptor_delete_confirm

  5. The new adaptor will be removed from under the cloudSwXtch or cloudSwXtch Bridge component.

Configuring HA in swXtch-top

In the HA view, users can configure an HA cluster, a functionality that is also available in wXcked Eye. This includes creating or deleting an HA cluster. Note: You cannot reconfigure an HA cluster after creating one. You must delete it to start again.  

Creating an HA cluster

  1. Navigate to the HA view in swXtch-top.

  2. Press F7 or click the Create button at the bottom of swXtch-top.

    1. A Create HA cluster form will open.

  3. Name your HA cluster.

  4. Set the number of Paths.

    1. For each path, assign a Path Name and a cloudSwXtch Control IP address.

  5. Confirm when you are happy with your selections.

    swXtch_top_HA_add

  6. A new HA cluster will now populate in the Cluster panel.

    swXtch_top_HA_added_and_streams_running

    After a cluster is added for this cloudSwXtch, the information will propagate to all relevant cloudSwXtches.

Deleting an HA cluster

  1. Navigate to the HA view in swXtch-top.

  2. Press F7 or click the Delete button at the bottom of swXtch-top.

    1. A Delete HA cluster form will open.

  3. Confirm that you wish to Destroy the HA cluster.

    swXtch_top_HA_delete_confirm

  4. The HA cluster and the displayed HA stats will be removed.

Troubleshooting swxtch-top

  1. If the swxtch-top "Status" is showing that there is a "Connection error:" 

    1. Check that the cloudSwXtch is started. 

    2. Check that you entered in the proper cloudswxtch name or IP when running the swxtch-top command.

    3. If name does not work when running the swxtch-top command then the DNS is not set-up correctly, use the IP address instead.

  2. If an xNIC was installed but is not showing up in swxtch-top:

    1. Navigate to the swxtch-nic.conf file and validate that the "SwxtchSvcAddr" is correct.

      • Windows can be found at "C:\Program Files\SwXtch.io\Swxtch-xNIC"

      • Linux can be found at "/var/opt/swxtch/swxtch-xnic.conf"

    2. Check that the firewall is open for the following ports: 

      subnet

      protocol

      ports

      vm

      ctrl-subnet

      tcp

      80

      cloudSwXtch

      ctrl-subnet

      udp

      10800-10803

      all

      data-subnet

      udp

      9999

      all

  1. If a multicast group is not showing up then check that they have registered.

    • In Linux, run this command:

      Text

       ip maddress show
    • In Windows, run this command in PowerShell:

      Text

      netsh.exe interface ipv4 show joins
    • If the joins are not showing here then the application is not joining the multi-cast group. In this case run swxtch-perf for the same IP:Port combination and then re-try in the program.

    • If the joins are not showing here then the application is not joining the multi-cast group. In this case run swxtch-perf for the same IP:Port combination and then re-try in the program.

    • If using Windows make use of Task Manager and view Performance to know where data is being sent/received.

    • Validate using TCPdump or Wireshark to identify where traffic is going as it could be going to the wrong network interface, it should be going to the Data Interface if xNIC2 and Swxtch-tun0 if xNIC1. An example is below:

$ sudo tcpdump udp -X -i <interface>

NOTE

xNIC1 interface: swxtch-tun0
xNIC2 interface: data nic (usually eth1 for Linux, and "Ethernet 2" for Windows

  • Validate that a firewall is not stopping the multicast and open up the firewall to include port exceptions.

swxtch-top on a cloudSwXtch

swxtch-top should be run from a virtual machine with an xNIC installed, it should be avoided to run it or anything else directly on a cloudSwXtch. That being said it can be done, but you must run it with sudo. Only run it on the cloudSwXtch if doing advanced troubleshooting.

sudo /swxtch/swxtch-top dashboard --swxtch localhost

Alternatively use 127.0.0.1 or swxtch-hostname or swxtch-IP in place of localhost