Is Multi Message MSI implemented on Linux / x86?

Question

I am working on a network driver for an FPGA endpoint that supports MSI interrupts with multiple messages (not msix) on the PCIe bus. The host processor is an x86 Intel i7 620LM running on CentOS with a 4.2 kernel.

The FPGA endpoint correctly advertises several msi vectors in it. MSI Capability Register (0x101 = 32 possible possible vectors).

From what I can tell, a multifunctional function has been added in kernel 4.2. Unfortunately, when I call pci_enable_msi_range(pdev, 1, 32); , it returns only 1 . When I call pci_msi_vec_count(pdev); , it returns 32 . I can query the irq handler on a single vector and it works as expected.

Does anyone know if multithreaded MSI vectors are really supported on Linux on the x86 architecture?

UPDATE: I was able to enable all 32 MSI vectors using a different SBC with an i7-4700EQ processor. This is with the 4.4-rc1 core.

UPDATE: Works with 4.2 too.

UPDATE: In this case, the problem was loading. I managed to get some vectors working after I updated the dossier.

Answer 1

Sure. I'm not sure why you got the error. but I wrote some drivers that used more than 1 vector.

I just checked my computer at home, I have different drivers that use more than 1 IRI IRI:

 $ cat /proc/interrupts | grep -i msi 33: 5683962 0 0 0 0 0 0 0 PCI-MSI 524288-edge radeon 34: 0 0 0 0 0 0 0 0 PCI-MSI 1572864-edge xhci_hcd 35: 0 0 0 0 0 0 0 0 PCI-MSI 1572865-edge xhci_hcd 36: 0 0 0 0 0 0 0 0 PCI-MSI 1572866-edge xhci_hcd 37: 0 0 0 0 0 0 0 0 PCI-MSI 1572867-edge xhci_hcd 38: 0 0 0 0 0 0 0 0 PCI-MSI 1572868-edge xhci_hcd 39: 0 0 0 0 0 0 0 0 PCI-MSI 1572869-edge xhci_hcd 40: 0 0 0 0 0 0 0 0 PCI-MSI 1572870-edge xhci_hcd 41: 0 0 0 0 0 0 0 0 PCI-MSI 1572871-edge xhci_hcd 42: 3807594 0 0 0 0 0 0 0 PCI-MSI 512000-edge 0000:00:1f.2 43: 0 0 0 0 0 0 0 0 PCI-MSI 2097152-edge xhci_hcd 44: 0 0 0 0 0 0 0 0 PCI-MSI 2097153-edge xhci_hcd 45: 0 0 0 0 0 0 0 0 PCI-MSI 2097154-edge xhci_hcd 46: 0 0 0 0 0 0 0 0 PCI-MSI 2097155-edge xhci_hcd 47: 0 0 0 0 0 0 0 0 PCI-MSI 2097156-edge xhci_hcd 48: 0 0 0 0 0 0 0 0 PCI-MSI 2097157-edge xhci_hcd 49: 0 0 0 0 0 0 0 0 PCI-MSI 2097158-edge xhci_hcd 50: 0 0 0 0 0 0 0 0 PCI-MSI 2097159-edge xhci_hcd 51: 310762 0 0 0 0 0 0 0 PCI-MSI 5242880-edge 0000:0a:00.0 52: 11 0 0 0 0 0 0 0 PCI-MSI 360448-edge mei_me 54: 38991293 0 0 0 0 0 0 0 PCI-MSI 7340032-edge enp14s0 55: 1 0 0 0 0 0 0 0 PCI-MSI 32768-edge i915 56: 1169 0 0 0 0 0 0 0 PCI-MSI 442368-edge snd_hda_intel 57: 152 0 0 0 0 0 0 0 PCI-MSI 526336-edge snd_hda_intel $ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 42 Model name: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz Stepping: 7 CPU MHz: 1634.257 CPU max MHz: 3800.0000 CPU min MHz: 1600.0000 BogoMIPS: 6825.67 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 8192K NUMA node0 CPU(s): 0-7 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid xsaveopt 

Answer 2

I had problems using multiple MSIs on an Intel computer, regardless of the Linux kernel version. After many attempts, I realized that the inclusion of VT-d in the BIOS fixes the problem (tested on the Linux 4.6 kernel). I think this is due to the fact that in order to support multiple MSIs on x86, the Linux kernel needs support for remapping interrupts, which is one of the features provided by VT-d.

Answer 3

To add to what Adrien mentioned, turning CONFIG_IRQ_REMAP on allowed me to solve the problem with multiple MSIs for the same device. pci_enable_msi_range works successfully with my requested msi interrupts (in this case 8).