Netfilter connects to a multi-core system

Question

We wrote LKM, which uses network sniffers to intercept IP packets. The problem is that at a payload of 1 Gbit / s we see that the hooks load only one processor core through soft irq. The other 15 cores are idle. Therefore, I conclude that interceptors are not multithreaded.

So my question is: is there any way I can handle hooks on multiple cores?

Answer 1

Span is not from netfilter, this is how your kernel handles interrupts.

By default, older versions of APIC provide all interrupts for CPU0.

You can check if this is your problem:

cat /proc/interrupts 

You can see if NIC interrupts are handled (and remember that the netfilter hook runs through RX or SoftIRQ TX) are handled by a single core.

In newer versions of the kernel, there is a compilation option (CONFIG_HOTPLUG_CPU) that balances IRQ over existing kernels.

If you cannot upgrade the version or recompile the kernel, you can update the SMP affinity (with a mask that processes more than CPUid) to try to balance the different cores. Or go to ACPI and the correct configuration (here I can no longer help).

Here you can find everything about this material (SMP affininty and proper IRQ processing)

Answer 2

The problem may be that your network adapter has only one interrupt. Some new NICS have several interrupts (the so-called multi-element network adapters) that allow you to distribute the load between many threads.

For single-chassis network adapters, there are some software features available in the new kernels that you can configure to distribute the load. See http://www.spinics.net/lists/linux-doc/msg02975.html for an overview of what is available.