Python udp course limits?

Question

Update to original post: colleague indicated that I am doing wrong. I will give an explanation at the bottom of the post, as this may be useful to others.

---

I am trying to get a basic understanding of python program network performance limitations and have run into an anomaly. Code snippet

while 1: sock.sendto("a",target) 

sends UDP packets to the target machine as soon as the host sends. I measure the sending speed of just over 4000 packets per second, or 250 us per packet. This seems slow even for an interpreted language such as python (the program runs on a dual-processor AMD, Linux, python version 2.6.6). I have seen much better performance in python for TCP, so I find this a bit strange.

If I ran this in the background and ran the top, I found that python uses only 25% of the processor, assuming that python might artificially delay UDP packets.

Has anyone else experienced something like this? Does anyone know if python limits the packet transfer rate, and if there is a way this is disabled?

BTW, a similar C ++ program, can send more than 200,000 packets per second, so it is not the internal limit of a platform or OS.

---

So it turns out I made a stupid newbie mistake. I forgot to call gethostbyname explicitly. Therefore, the destination address in the sendto command contained a symbolic name. This caused a name resolution every time a packet was sent. After that, I measure the maximum sending speed of about 120,000 p / s. Much better.

Answer 1

Have you tried connect() first and then send() instead of sendto() ? (UDP connect() just sets the destination address, it doesn’t actually make a “connection”.) I am rusty, but I think Python does more interpretation on the address parameter than C sockets, which may be related to overhead.

Answer 2

You might want to post a more complete code sample so that others can repeat their test. 250 μs per loop iteration is too slow. Based on the daily experience of optimizing Python, I would expect the Python interpreter overhead to be well below 1 µs on a modern machine. In other words, if a C ++ program sends 200K packets per second, I expect Python to be in the same order of magnitude of speed.

(In light of the foregoing, normal optimization suggestions, such as moving the search for the sock.sendto attribute out of a loop, do not apply here because slowness comes from a different source.)

A good first step is to use strace to check what Python actually does. Is it a single-threaded program or a multi-threaded application that can lose latency on the GIL? Is sock regular Python socket or is it part of a more complex API? Does the same thing happen when os.write called directly on the os.write socket?