Skip to content

Cilium Performance Test for Cross-Node Communication between Pods

This page describes the performance test results of Cilium in the scenario of cross-node communication between Pods.

The official test report provided by Cilium shows that Cilium leverages eBPF technology to improve the accessibility performance between pods across nodes. The test is based on a bare metal machine and 100G network connection, with Cilium's configuration adjusted for maximum network performance. The adjustment of Cilium including:

  • Using host routing to forward pod data between nodes without any tunneling mode configured.

  • Disabled iptables connection tracking on the host.

Test objects

cilium

Test environment

env

Testing tool and metrics

The tests use netperf to generate workloads and collect metrics. The test metrics are as follows:

  • Throughput

    Maximum transfer rate over a single TCP connection and total transfer rate over 32 concurrent TCP connections.

  • Request/Response Rate (TCP_RR)

    The number of request/response messages that can be transmitted per second over a single TCP connection and 32 concurrent TCP connections.

Test one

For TCP Single Stream, test TCP throughput between two cross-node Pods:

single1

CPU overhead for client and server:

single2

Test two

For 32-core CPU, 32 concurrent TCP connections, test TCP throughput between two cross-node Pods

multi

CPU overhead for client and server:

multi2

Test three

Simulate the test of Layer 7 access based on the TCP_RR mode to repeat the behavior of "sending 1 request, and then waiting for 1 reply" over and over again over the same TCP connection.

The TCP_RR performance between two cross-node Pods is tested with the Pod having only 1 core CPU:

tcprr1

CPU overhead of the client and server:

tcprr2

Test four

Test TCP_RR performance between two cross-node Pods when the Pod has only 32 cores of CPU:

tcprr3

CPU overhead of the client and server:

tcprr4

Comments