WHAT TO EXPECT
swxtch-top is one of the utility applications included in xNIC installation. It can be run from the console of any VM that has the xNIC software installed, displaying real-time statistics of an attached cloudSwXtch instance. This includes data regarding mesh, high availability, multicast and PTP.
In this article, you will learn how to navigate through the different pages in swxtch-top and get better visibility on how data flows in your cloudSwXtch instance.
Running swxtch-top
Depending on your operating system, you can use certain commands to run swxtch-top on your VM.
For both Windows and Linux agents (xNICs), users can enter the following into the terminal:
swxtch-top --swxtch <cloudSwXtch-IP>
Example:
swxtch-top --swxtch 10.5.1.6
From the cloudSwXtch, users can enter the following command:
swxtch-top
Updating cloudSwXtch for swXtch-top
The following screenshots have been taken on the latest version of cloudSwXtch. To learn how to upgrade your cloudSwXtch, please see the article, Upgrading cloudSwXtch.
Navigating swxtch-top Dashboard
The swxtch-top dashboard is organized into 3 panels as shown in the screenshot above. While the top panel will remain static, displaying information regarding the cloudSwXtch, the bottom 2 panels will change depending on the selected view. The swxtch-top dashboard has 12 different views:
Components (Default)
Streams
StreamLinks
Adaptors
HA
Lossless
Aliases
PTP
Subscriptions
Notifications
Configuration
Licensing
Switching Between swXtch-top Views
To switch between views, click on the Views option in the lefthand corner of swXtch-top and select the desired view from the menu. Alternatively, users can press CTRL + V to open the menu and then CTRL + the highlighted letter to switch to that view. For example, to toggle to “StreamLinks",” enter CTRL + V and CTRL + K.
Note: Opening the menu with the CTRL key is not necessary if you know the highlighted letter for your desired view. Simply entering CTRL + the highlighted letter will open the view.
Using the Setup Function
Users can modify what columns are visible in each of the views and the width of those columns. Please note: These changes will persist between swxtch-top sessions.
To do this:
Select the [+] next to the first column. This will open the Setup menu for that specific panel.
Alternatively, you can click a panel to select it and press F3 to open the Setup menu. You can tell you selected panel by the increased thickness of its border.
Click one of the blue checkboxes under Visible to highlight the column you wish to add. An X will appear.
You can also change the width of the columns in swXtch-top in this panel.
Hit OK to confirm your selection.
The new column should now appear in the swxtch-top view. In this example, the Cloud column was added by the user and is now displayed in the Components view.
Using Setup to Toggle Raw Totals On/Off
By default, swXtch-top will round the total of bits and packets into a simplified number. For example, instead of 1496892, swxtch-top will write 1.5 M (million). To toggle this on/off, go to the Setup menu and check the column that you would like the raw totals for and select OK.
The raw total should appear in a few seconds for the entire column.
Changing swXtch-top Colors
Users can change the colors displayed in swxtch-top by keying in different hex codes in the Colors Set-Up menu.
To do this:
Click Colors at the bottom of swxtch-top or enter CTRL + R.
Select Custom from the Scheme dropdown menu.
Enter the desired hex codes into one of the fields.
Click Confirm when you are happy with your color selections.
swxtch-top should now display the assigned colors.
Exporting Support Logs
The swXtch-top Support menu allows users to export cloudSwXtch logs, similar to the functionality found in the wXcked Eye Support page.
To navigate to this menu, you can enter CTRL + T or select Support in the bottom navigation bar.
Selecting one of the report options will prompt users to enter parameters as shown in the example below. The report will save on the VM’s user directory.
Panel 1: Information
The first panel of the swxtch-top dashboard provides users with information regarding their cloudSwXtch as well as their subscription plan. Each cloud provider will have alternative titles for some of the listed items but for the most part, the information is the same.
On the left side of the section:
cloudSwXtch name
Cloud (Azure, AWS, GCP, OCI)
SubscriptionID (Azure), Account ID (AWS), or ID (GCP)
ResourceGroupName (Azure) or Region (AWS and GCP)
SwXtchID
Status
On the right side of the section:
Licensing Type
Bandwidth Allotment
Number of Clients and Bridges
License Expiration Date
The Information panel is fully collapsible by either selecting the (+) or (-) buttons. Users can also open and close the panel using the F2 key on their keyboard. The collapsed view will only display the cloudSwXtch name, the status, the licensing type, and new notifications.
When there are new notifications under the Notifications view, a “NEW NOTIFICATIONS” message will also display in this box.
For more information regarding licensing, please read the cloudSwXtch System Requirements article.
Panel 2 and 3: Views
Panel 2 and 3 defaults to the Components view and is shown in the picture above. However, the display changes based on view selection. To change views, key in CTRL + the associated letter for that view.
Please note: Not all views have a 3rd panel.
Filter Bar
At the top of each view’s main panel, there is a white search “Filter” bar. This allows users to filter through the contents in the current view. Simply type the information you would like to filter the list by, followed by an asterisk (*), and hit Enter. For example, v1*
to see all the fields that have “v1” in in.
Components
The Components view gives a detailed list of all the connected nodes in a cloudSwXtch’s network. To navigate to this view, press CTRL+E or click on the view’s name. To switch between components, use the up and down directional keys on your keyboard.
This view includes:
Name - The name of the component
Class - Either cloudSwXtch, xNIC, or VM (non-xNIC)
Type - X1 or X2 for cloudSwXtch or Type 1 or 2 for xNIC
RX bps - The total ingress bits per second that the component is receiving.
RX pps - The total ingress packets per second that the component is receiving.
TX bps - The total egress bits per second that the component is transmitting.
TX pps - The total egress packets per second that the xNIC is transmitting.
Primary IP - The primary interface IP address
OS - Operating System
Primary Swxtch - This lists the cloudSwXtch the xNIC was installed against
There are additional column options in the Setup menu:
An Interfaces panel displays when in the Component view, detailing the interfaces of a selected component. This panel includes the following information for each interface:
Filtering Out Virtual Machines
Next to the Filter search bar, there is a checkbox with virtual machines (VMs) selected. This means that non-xNIC virtual machines will display in the components list. If a user wishes to hide these virtual machines and just focus on swXtch.io-based components, they can deselect this VM filter.
Streams
This tree view shows all the multicast groups that are being received and/or transmitted by the cloudSwXtch and how to interacts with the various endpoints. To navigate to this view, enter CTRL + S or click on the view’s name.
This view includes:
Stream - The name of the stream IP and Port. This could also be an Alias set in wXcked Eye. For example, BaseballCamera1 and BaseballCamera2 were Aliases assigned in wXcked Eye.
Components: swxtch, xNIC or VM (Virtual Machine w/o xNIC)
Protocol: Multicast or Broadcast
RX Pkts: The total ingress packets per second being received by the component.
RX Bits: The total ingress bits per second that is received by the component.
TX Pkts: The total egress packets per second that is transmitted by the component.
TX Bits: The total egress bits per second that that is transmitted by the component.
In the example above, stream 224.2.2.3:8400 is listed. It is being transmitted by an endpoint (v1-u22-172-31-64-6) to the cloudSwXtch (v1-c2-172-31-64-14), which is then sent to the other endpoints. That is why there are three additional endpoints listed under the cloudSwXtch.
Stream Links
The Stream Links view displays all the cloudSwXtch network components (cloudSwXtch or xNIC) and the streams they are linked to. To select this view, press CTRL + K or click on the view’s name. To expand a component, simply use the up and down keys on the keyboard and hit ENTER to select.
In this example, the cloudSwXtch (dsd-core-200) is selected with the singular stream 239.1.1.1:3490 expanded. Here, a user can see a list of endpoints receiving/transmitting the stream to/from the cloudSwXtch, dsd-core-200.
Components: cloudSwXtch or endpoints (xNICs)
Direction: Either Ingress or Egress, depending on whether data is being ingested or transmitted by the component
Protocol: Multicast or Broadcast
Bits (bps): Bits per second
Pkts (pps): Packets per second
Frag-Pkts (fpps): Fragmented packets per second
Frags/Pkts: Total # of Fragmented Packets
Missing Packets: Packets that did not make their destination
OOO Pkts: Out of order packets
Adaptors
The Adaptors view displays information regarding Protocol Conversion and Fanout. It includes a detailed list of configured protocols (UDP, SRT Caller/Listener, or RIST Caller/Listener), their direction (ingress/egress), the stream IP or stream name, the node, and Listener Port. Users can also add and delete adaptors. For more information, see the Configuring Adaptors for Protocol Fanout section.
To navigate to this view, enter CTRL + A or click on the view’s name.
HA (High Availability)
The HA view shows additional details for high availability from the perspective of the consumer. It will only show data if High Availability has been configured.
To navigate to this view, enter CTRL + H or click on the view’s name. Go to High Availability for configuration details.
This view is organized into two sections: HA and Cluster.
HA: This section lists the xNIC components in the cloudSwXtch network that are consuming or producing HA paths. Expanding the consumer details will show the stream IP split into two paths (red and blue) and a reconstructed path.
Cluster: This section displays the high availability clusters and the associated cloudSwXtches for each path.
The statistics by each line item is further explained below:
Path Ingress pps - The total ingress packets per second that is received in the path for the multicast group.
Path Ingress bps - The total ingress bits per second that is received in the path for the multicast group.
Path Usage % - The percentage that the path is used in the highly available multicast group.
Missing packets - The total number of missing packets for the path since the inception of the stream. If you stop the stream or cloudSwXtches, the number will stop increasing but will not reset.
Users can also create and destroy HA clusters in this view. For more information, see Configuring HA in swXtch-top.
Lossless
The Lossless view shows statistics for data flow using swXtch Lossless UDP, or SLP, from ground to cloud through the cloudSwXtch Bridge.
To navigate to this view, enter CTRL + L or click on the view’s name.
By default, the view is organized into two sections: SLP Rx and SLP Tx. Both sections will populate with information, illustrating traffic between the cloudSwXtch Bridge and the cloudSwXtch.
By default, the following columns are displayed for SLP Rx:
RTT (ns): Round trip time in nanoseconds it takes for our connection to send a ping and receive a response. One way latency can be estimated by dividing this value in half.
RTT Variance: The variance of the RTT measurement. A high variance is indicative of an unstable connection.
Data Pkts: The total number of packets received (original and retransmitted)
Retr Pkts: The number of packets received as a a retransmitted packet.
Packets Lost: The total number of packets that were not recovered by the connection and are lost.
By default, the following columns are displayed for SLP Tx:
RTT (ns): The round trip time in nanoseconds it takes for our connection to send a ping and receive a response. One way latency can be estimated by dividing this value in half.
RTT Variance: The variance of the RTT measurement. A high variance is indicative of an unstable connection.
Data Pkts: The total number of packets transmitted (original and retransmitted)
Retr Pkts: The total number of retransmitted packets
Please note: Although the Setup feature has additional SLP-related statistics, they are mostly use for troubleshooting purposes by the swXtch.io. We recommend using the default columns above to monitor your SLP connection.
Aliases
The Aliases section provides users with a detailed list of user-assigned names for components and streams. It is split into two sections: Component Aliases and Stream Aliases.
For Component Aliases, the following columns will display:
Alias
Primary IP
Hostname
Class
For Stream Aliases, there are only two columns: Alias and Stream.
To navigate to this view, enter CTRL + E or click on the view’s name.
Adding and Deleting Aliases
Users can add and delete aliases in swxtch-top. This can also be done in the Aliases tab on the wXcked Eye Settings page.
To add an alias:
Click the Add button at the bottom of swxtch-top or hit F7 on your keyboard.
A new window will open.
Select the Alias type: Component or Stream.
For Component, add an IP address and the assign an Alias name.
For Stream, add a Stream IP address, Port and assign an Alias.
Click Confirm once you are happy with your selections.
The new Alias will now appear under the selected alias type.
To delete an alias:
Highlight an Alias from either the Component or Stream Aliases list.
Click Delete at the bottom of the swxtch-top window.
Click Confirm.
The Alias will disappear from list.
PTP
The PTP view displays information regarding the clock sync configuration for the cloudSwXtch. The page in swxtch-top will only populate with information if the user has the PTP feature enabled.
To navigate to this view, enter CTRL + P or click on the view’s name.
In the example above, the cloudSwXtch is acting as the Master Node and is the first line item listed. Below the cloudSwXtch are the follower nodes.
Master Node- The Master Node is what the PTP configuration sets as the most reliable time source. This will send the true time it receives from the source clock to the Follower Nodes.
Name - The name of the cloudSwXtch
Time Sync Service - The source clock
Follower Nodes- The Follower Nodes lists the agents/VMs that subscribe to the Master Node for accurate timing.
Name - The name of the endpoints
Status - The status of the endpoints, noting if the node is active in the PTP configuration
Local Offset - The local offset denotes the offset in time from the cloudSwXtch to the xNIC.
Root Offset - The root offset denotes the offset in time from the GrandMaster clock to the cloudSwXtch and its follower nodes (xNIC). Note how the root is larger than the local. This is normal behavior since the distance between the follower node and the Grandmaster clock is greater than the offset between a cloudSwXtch and xNIC.
PTP Stabilization
After upgrading your cloudSwXtch system, you may notice that the local and root offset values are much larger than they actually are. It can take up to 30 minutes for the values to stabilize and return back to normal levels.
Subscriptions
The Subscriptions page details the different source list configurations for single source multicast and multiple multicast groups. Here, users will get a detailed lists of included and excluded streams and the source they originate from.
To navigate to this view, enter CTRL + U or or click on the view’s name.
Notifications
The Notifications view compiles a list of cloudSwXtch-related notifications.
To navigate to this view, enter CTRL + N or click on the view’s name.
A New Notifications alert will appear in the Information panel in swXtch-top as shown below. Visiting the Notifications view will remove this alert.
Configurations
The Configurations view gives users a visualization of the xNICs’ configuration JSON files.
To navigate to this view, enter CTRL + F or click on the view’s name.
Licensing
The Licensing view gives a breakdown of the entitlements allotted to each of the cloudSwXtches included in the network.
Depending on their license, users will have a set number for their Max Bandwidth, Max Clients and Max bridges. In the example above, the user has a max bandwidthMbps of 236bps with 20 clients max and 1 bridges max. This section will also show if a user has the following features enabled: HA, Fanout, Clock Sync (PTP), wXckedEye, TachyonLive, and Major Version Update.
In addition to the Licensing details, the Details panel lists the expiration date of the license and the license type for a selected cloudSwXtch.
To navigate to this view, enter CTRL + G or click on the view’s name.
Configuring Adaptors for Protocol Fanout
In the Adaptors view, users can configure adaptors for protocol fanout, a functionality that is also available in wXcked Eye. swXtch.io’s protocol fanout feature allows users to configure unicast (UDP, SRT, RIST) for the cloudSwXtch and the cloudSwXtch Bridge and fan it out to multiple instances as UDP, SRT, RIST, or multicast. For swXtch-top, users have the ability to both add and delete an adaptor.
Adding an Adaptor
Navigate to the Adaptors view in swXtch-top.
Highlight a component (cloudSwXtch or cloudSwXtch Bridge) that you would like to add an adaptor. You can do this either by clicking on a listed component or using the navigation up and down navigation keys.
Press F7 or click the Add button at the bottom of swXtch-top.
A new window will open.
Enter the information for your adaptor. For an explainer on what each field means, see Protocol Conversion and Fanout with wXcked Eye.
Select Confirm.
The new adaptor will list under the cloudSwXtch or cloudSwXtch Bridge component.
Deleting an Adaptor
Navigate to the Adaptors view in swXtch-top.
Select an adaptor under a cloudSwXtch or cloudSwXtch Bridge component that you would like to delete. You can do this either by clicking on a listed component or using the navigation up and down navigation keys.
Press F7 or click the delete button at the bottom of swXtch-top.
A new window will open.
Confirm you wish to delete the selected adaptor.
The new adaptor will be removed from under the cloudSwXtch or cloudSwXtch Bridge component.
Configuring HA in swXtch-top
In the HA view, users can configure an HA cluster, a functionality that is also available in wXcked Eye. This includes creating or deleting an HA cluster. Note: You cannot reconfigure an HA cluster after creating one. You must delete it to start again.
Creating an HA cluster
Navigate to the HA view in swXtch-top.
Press F7 or click the Create button at the bottom of swXtch-top.
A Create HA cluster form will open.
Name your HA cluster.
Set the number of Paths.
For each path, assign a Path Name and a cloudSwXtch Control IP address.
Confirm when you are happy with your selections.
A new HA cluster will now populate in the Cluster panel.
After a cluster is added for this cloudSwXtch, the information will propagate to all relevant cloudSwXtches.
Deleting an HA cluster
Navigate to the HA view in swXtch-top.
Press F7 or click the Delete button at the bottom of swXtch-top.
A Delete HA cluster form will open.
Confirm that you wish to Destroy the HA cluster.
The HA cluster and the displayed HA stats will be removed.
Troubleshooting swxtch-top
If the swxtch-top "Status" is showing that there is a "Connection error:"
Check that the cloudSwXtch is started.
Check that you entered in the proper cloudswxtch name or IP when running the swxtch-top command.
If name does not work when running the swxtch-top command then the DNS is not set-up correctly, use the IP address instead.
If an xNIC was installed but is not showing up in swxtch-top:
Navigate to the swxtch-nic.conf file and validate that the "SwxtchSvcAddr" is correct.
Windows can be found at "C:\Program Files\SwXtch.io\Swxtch-xNIC"
Linux can be found at "/var/opt/swxtch/swxtch-xnic.conf"
Check that the firewall is open for the following ports:
subnet
protocol
ports
vm
ctrl-subnet
tcp
80
cloudSwXtch
ctrl-subnet
udp
10800-10803
all
data-subnet
udp
9999
all
If a multicast group is not showing up then check that they have registered.
In Linux, run this command:
Text
ip maddress show
In Windows, run this command in PowerShell:
Text
netsh.exe interface ipv4 show joins
If the joins are not showing here then the application is not joining the multi-cast group. In this case run swxtch-perf for the same IP:Port combination and then re-try in the program.
If the joins are not showing here then the application is not joining the multi-cast group. In this case run swxtch-perf for the same IP:Port combination and then re-try in the program.
If using Windows make use of Task Manager and view Performance to know where data is being sent/received.
Validate using TCPdump or Wireshark to identify where traffic is going as it could be going to the wrong network interface, it should be going to the Data Interface if xNIC2 and Swxtch-tun0 if xNIC1. An example is below:
$ sudo tcpdump udp -X -i <interface>
NOTE
xNIC1 interface: swxtch-tun0
xNIC2 interface: data nic (usually eth1 for Linux, and "Ethernet 2" for Windows
)
Validate that a firewall is not stopping the multicast and open up the firewall to include port exceptions.
swxtch-top on a cloudSwXtch
swxtch-top should be run from a virtual machine with an xNIC installed, it should be avoided to run it or anything else directly on a cloudSwXtch. That being said it can be done, but you must run it with sudo. Only run it on the cloudSwXtch if doing advanced troubleshooting.
sudo /swxtch/swxtch-top dashboard --swxtch localhost
Alternatively use 127.0.0.1 or swxtch-hostname or swxtch-IP in place of localhost