Test xNIC with K8s

  • Updated on Oct 9, 2024
  • Published on Jun 13, 2023
  • 7 minute(s) read
Prev Next

WHAT TO EXPECT

Before running your application in your preferred cloud, it is a good idea to test with swXtch.io's provided tools/examples.

In this article, you will learn how to test xNIC with K8s. Please complete the installation process outlined in Install xNIC on K8s before you begin testing. 

Prerequisites

For this test to work, a user should have at least two nodes. 

STEP ONE: Create A Consumer 

  1. Create a TestConsumer.yaml file using the example below.
    Replace the XNIC_SWXTCH_ADDR with the cloudSwXtch control address. 

    apiVersion: v1
kind: Pod
metadata:
  name: consumer-a
  labels:
    app: consumer-a
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - producer-a
            - consumer-b
        topologyKey: kubernetes.io/hostname
  containers:
  - name: consumer-a
    image: ubuntu:20.04
    securityContext:
      privileged: true
    env:
    - name: IS_DAEMON
      value: "false"
    - name: PERF_TYPE
      value: "consumer"
    - name: PERF_NIC
      value: "eth0"
    - name: PERF_MCGIP
      value: "239.0.0.10"
    - name: PERF_MCGPORT
      value: "8410"
    - name: XNIC_SWXTCH_ADDR
      value: "10.224.0.115"
    command: ["/bin/bash"]
    args: ["-c", "apt update && apt install curl -y;
                  curl http://$(XNIC_SWXTCH_ADDR)/services/install/swxtch-xnic-k8s-install.sh --output swxtch-xnic-k8s-install.sh; 
                  chmod +x swxtch-xnic-k8s-install.sh;
                  ./swxtch-xnic-k8s-install.sh -v 2;
                  sleep infinity"]

  2. Upload the file into the Azure CloudShell. 

STEP TWO: Create a Producer

  1. Create a TestProducer.yaml file using the example below.
    Replace XNIC_SWXTCH_ADDR with the cloudSwXtch control address.

    Shell

    apiVersion: v1
kind: Pod
metadata:
  name: producer-a
  labels:
    app: producer-a
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - consumer-a
            - producer-b
        topologyKey: kubernetes.io/hostname
  containers:
  - name: producer-a
    image: ubuntu:20.04
    securityContext:
      privileged: true
    env:
    - name: IS_DAEMON
      value: "false"
    - name: PERF_TYPE
      value: "producer"
    - name: PERF_NIC
      value: "eth0"
    - name: PERF_MCGIP
      value: "239.0.0.10"
    - name: PERF_MCGPORT
      value: "8410"
    - name: PERF_PPS
      value: "100"
    - name: XNIC_SWXTCH_ADDR
      value: "10.224.0.115"
    command: ["/bin/bash"]
    args: ["-c", "apt update && apt install curl -y;
                  curl http://$(XNIC_SWXTCH_ADDR)/services/install/swxtch-xnic-k8s-install.sh --output swxtch-xnic-k8s-install.sh; 
                  chmod +x swxtch-xnic-k8s-install.sh;
                  ./swxtch-xnic-k8s-install.sh -v 2;
                  sleep infinity"]

  2. Upload the file into the Azure CloudShell. 

STEP THREE: Run Test

  • Run the producer by running this command in your preferred cloud's cloudShell Bash window.
    Wait for the cursor to return to know it is fully created.

    Shell

    kubectl create -f TestProducer.yaml

  • Run the consumer by running this command in your preferred cloud's cloudShell bash window.
    Wait for the cursor to return to know it is fully created.

    kubectl create -f TestConsumer.yaml

Validate they are running using this command:

kubectl get pods -o wide -A

Below is an example in Azure showing the consumer-a and producer-a running:

donna@Azure:~$ kubectl get pods -o wide -A
NAMESPACE     NAME                                 READY   STATUS    RESTARTS   AGE   IP             NODE                                NOMINATED NODE   READINESS GATES
kube-system   consumer-a                           1/1     Running   0          15m   10.0.1.91      aks-nodepool1-23351669-vmss000006   <none>           <none>
kube-system   producer-a                           1/1     Running   0          15m   10.0.1.90      aks-nodepool1-23351669-vmss000005   <none>           <none>
kube-system   cilium-node-init-kbql4               1/1     Running   0          27h   10.2.128.101   aks-nodepool1-23164585-vmss00000j   <none>           <none>
kube-system   cilium-node-init-sg4vc               1/1     Running   0          27h   10.2.128.100   aks-nodepool1-23164585-vmss00000i   <none>           <none>
kube-system   cilium-nx7vl                         1/1     Running   0          27h   10.2.128.100   aks-nodepool1-23164585-vmss00000i   <none>           <none>
kube-system   cilium-operator-6485c89c66-748tz     1/1     Running   0          27h   10.2.128.101   aks-nodepool1-23164585-vmss00000j   <none>           <none>
kube-system   cilium-vv4qs                         1/1     Running   0          27h   10.2.128.101   aks-nodepool1-23164585-vmss00000j   <none>           <none>
kube-system   cloud-node-manager-mncgk             1/1     Running   0          27h   10.2.128.100   aks-nodepool1-23164585-vmss00000i   <none>           <none>
kube-system   cloud-node-manager-qg5wf             1/1     Running   0          27h   10.2.128.101   aks-nodepool1-23164585-vmss00000j   <none>           <none>
kube-system   coredns-autoscaler-569f6ff56-qtqpr   1/1     Running   0          28h   10.0.0.121     aks-nodepool1-23164585-vmss00000i   <none>           <none>
kube-system   coredns-fb6b9d95f-blk6j              1/1     Running   0          28h   10.0.0.236     aks-nodepool1-23164585-vmss00000i   <none>           <none>
kube-system   coredns-fb6b9d95f-pxzh2              1/1     Running   0          28h   10.0.0.131     aks-nodepool1-23164585-vmss00000i   <none>           <none>

Step Four: Validate The Test Is Running

You can validate it is working by viewing logs with this command:

kubectl logs pods/producer-a -f

2nd screen

Alternatively, you can log into your cloudSwXtch and run this command to see data flowing between nodes:

swxtch-top

swXtch-top should show the traffic coming in and out of the nodes with either the producer or the consumer.

Step Five: Cleaning the Pods

  1.  Stop the test consumer by running this command back in your preferred cloud's CloudShell bash window. 

    1. Wait for the cursor to return to know it is deleted fully.

      kubectl delete -f TestConsumer.yaml

    2. swXtch-top should no longer show the consumer. Additionally, running kubectl get pods -o wide should now show just the test consumer as shown below:

      donna@Azure:~$ kubectl get pods -o wide -A
NAME                READY   STATUS        RESTARTS   AGE   IP            NODE                                NOMINATED NODE   READINESS GATES
consumer-a          1/1     Terminating   0          15m   10.0.1.91     aks-nodepool1-23351669-vmss000006   <none>           <none>
producer-a          1/1     Running       0          15m   10.0.1.90     aks-nodepool1-23351669-vmss000005   <none>           <none>
swxtch-xnic-46qgg   1/1     Running       0          39m   10.2.128.96   aks-nodepool1-23351669-vmss000005   <none>           <none>
swxtch-xnic-szdk7   1/1     Running       0          40m   10.2.128.95   aks-nodepool1-23351669-vmss000004   <none>           <none>

  2. Stop the test producer by running this command in your preferred cloud's CloudShell bash window. 

    1. Wait for the cursor to return to know its fully deleted.

      kubectl delete -f TestProducer.yaml

    2. swXtch-top should no longer show the producer. This may take a minute to display. Additionally, running kubectl get pods -o wide should now show just the test producer as shown below:

      donna@Azure:~$ kubectl get pods -o wide -A
NAME                READY   STATUS        RESTARTS   AGE   IP            NODE                                NOMINATED NODE   READINESS GATES
producer-a          1/1     Terminating   0          15m   10.0.1.90     aks-nodepool1-23351669-vmss000005   <none>           <none>
swxtch-xnic-46qgg   1/1     Running       0          42m   10.2.128.96   aks-nodepool1-23351669-vmss000005   <none>           <none>
swxtch-xnic-szdk7   1/1     Running       0          42m   10.2.128.95   aks-nodepool1-23351669-vmss000004   <none>           <none>

Now that the system is validated using swXtch.io, you can test with your K8s application.