Since the Raspberry Pi Compute Module 4 was introduced last year, I've been testing a variety of PCI Express NICs with it. One of the main types of NIC I'm interested in is cheap 2.5 Gigabit Ethernet adapters.

2.5 Gigabits is about the highest reasonable bandwidth you can get through the PCI Express Gen 2.0 x1 lane on the Raspberry Pi, and it's also a lot more accessible than 10 Gigabit networking, especially for home users who might already have Cat5e runs that they are loathe to swap out for Cat6 or better cabling.

In my testing, besides discovering that not all 10 Gbps SFP+ transceivers are created equal, I found out that when it comes to performance, the Linux driver you're using matters—a lot.

And as a stark illustration of this point, I recently pitted an ASUSTOR Drivestor 4 Pro NAS against a Raspberry Pi 'Taco' NAS. Both of these devices use the popular Realtek RTL8125B chip, and both of them were reporting underwhelming iperf3 bandwidth results:

The maximum throughput was eerily similar, just under 1.9 Gbps. And worse, I noticed when using devices through a PCI Express switch on the Pi, like a SATA RAID array, the network throughput took a hit, down to 1.51 Gbps on the Pi.

I wondered if the driver might've seen some improvements in later revisions of the Linux kernel, so I recompiled the kernel using the 5.15.x source:

That made no difference—in fact, the results started to have more jitter in them, so I decided that avenue wasn't worth pursuing further.

So next, I downloaded the driver directly from Realtek's website, and installed version 9.007.01:

$ sudo apt-get install -y raspberrypi-kernel-headers
$ tar vjxf r8125-9.007.01.tar.bz2
$ cd r8125-9.007.01/
$ sudo ./autorun.sh

I re-ran my benchmarks, and wouldn't you know? I'm able to get the full 2.35 Gbps of real-world throughput through the card—regardless of other traffic on the PCIe bus!

So the moral of the story seems to be: if you're not seeing the performance you expect, see if the vendor's driver is better than what's in the kernel tree.

Many vendors take an upstream-first approach, where they make sure any driver optimizations make their way to the kernel quickly... but not all. I'm not sure how the discrepancy is so big, though, in this case. It looks like 8125B support was only added a year and a half ago, and only a few chip-specific commits have been added since.

I have to wonder if maybe since far fewer Linux users have 2.5G-capable networks, these kinds of performance issues aren't discussed as much?

I confirmed with ASUSTOR that their current Drivestor 4 Pro software is running the driver from the kernel, and not the Realtek vendor driver. No word yet on if that could change—but if so, it may be able to unlock more performance from their value-lineup of 2.5G NASes!

For all my debugging details, check out my GitHub issue NAS Comparison - ASUSTOR Drivestor 4 Pro vs Pi CM4.