On linux, why am I losing UDP packets if I call send () as quickly as possible?

Question

An implicit question: if Linux blocks the send() call when the socket send buffer is full, why should there be lost packets?

Details: I wrote a small utility in C to send UDP packets to the unicast address and port as soon as possible. Each time I send a UDP payload of 1450 bytes, and the first bytes are a counter that increments by 1 for each packet. I run it on Fedora 20 inside VirtualBox on a desktop PC with 1Gb nic (= pretty slow).

Then I wrote a small utility for reading UDP packets from a given port, which checks the packet counter for its own counter and prints a message if they are different (i.e. 1 or more packets are lost). I run it on a Fedora 20 dual-core server with a 1Gb ethernet nic (= super fast). It shows a lot of lost packets.

Both machines are on the local network. I do not know exactly the number of flights between them, but I do not think that there are more than two routers between them.

Things I tried:

• Add a delay after each send() . If I set the delay to 1 ms, then the packets are no longer lost. A delay of 100us will begin to lose packets.
• Increase the receive socket buffer size to 4MiB with setsockopt() . It does not matter...

Please enlighten me!

Answer 1

For UDP, the SO_SNDBUF socket SO_SNDBUF limits only the size of the datagram you can send. There is no explicit socket buffer throttling for sockets, as with TCP. Of course, in the core of the frame queue to the network card.

In other words, send(2) may refuse your datagram without returning an error (see the ENOBUFS description at the bottom of the manual page).

Then the package can be removed almost everywhere on the way:

• sending a network card does not have free hardware resources to service the request, the frame is discarded,
• the intermediate routing device does not have available buffer space or implements some algorithm to avoid congestion, omits the packet,
• the receiving network card cannot receive network frames at a given speed; some frames are simply ignored. In the reading application
• there is not enough space to receive socket buffers to accommodate bursts of traffic, the kernel decompresses datagrams.

From what you said, it is very likely that the virtual machine cannot send packets at high speed. Force wire with tcpdump(1) or wireshark(1) as close to the source as possible, and check your serial numbers - this will tell you if this is the culprit who is to blame.

Answer 2

Even if send() blocked when the send buffer is full (assuming you have not set SOCK_NONBLOCK on the socket to put it in non-blocking mode), the receiver must be fast enough to process all incoming packets, if the receiver or any intermediate system is slower than the sender, packets will be lost when using UDP. Please note that it is applied more slowly not only for the speed of the network interface, but also for the entire network stack plus an application for user space.

In your case, it is possible that the recipient receives all the packets, but cannot process them fast enough in user space. You can verify this by recording and analyzing your traffic through tcpdump or wireshark .

If you do not want to lose packets, switch to TCP.

Answer 3

• Any of the two routers you specify can drop packets if there is congestion,
• and the receiving PC may drop or skip packets also under certain circumstances, such as congestion.

Answer 4

As one of the above posters said, UDP is a simple datagram protocol that does not guarantee delivery. Or because of the local machine, equipment on the network, etc. For this reason, many modern developers recommend switching to TCP if you want reliability. However, if you really want to adhere to the UDP protocol, and there are many good reasons for this, you will need to find a library to help you ensure delivery. Look for SS7 designs, especially in the telephony API, where UDP is used to transmit voice, data, and signaling information. For your only application, I can offer the enet UDP library. http://enet.bespin.org/