I need a TCP (ioctl) option to send data immediately

Question

I have an unusual situation: I use the Linux system in the embedded situation (Intel box currently using the 2.6.20 kernel), which should interact with the embedded system, which has a partially broken TCP implementation. As far as I can tell right now, they expect each message from us to appear in a separate Ethernet frame! They seem to have problems when messages are split into Ethernet frames.

We are on the local network with the device, and there are no routers between us (just a switch).

We, of course, are trying to get them to fix their system, but this may not be possible.

I already set TCP_NODELAY on my sockets (I connect to them), but this only helps if I am not trying to send more than one message at a time. If I have several outgoing messages per line, these messages usually fall into one or two Ethernet frames, which causes problems on another system.

As a rule, I can avoid the problem by using a timer so as not to send messages too close to each other, but this clearly limits our bandwidth. In addition, if I turn off the time too low, I risk overloading the network that supports packet transmission and ending with the fact that it allows you to send more than one message to the same packet.

Is there any way to determine if the driver has data in the queue or not? Is there a way to get the driver to send independent write calls to independent transport layer packets? I looked at the socket pages (7) and tcp (7) and I did not find anything. Maybe I don't know what I'm looking for.

Obviously, UDP will be one way, but then again, I don't think we can get the other end to change anything at this point.

Any help is greatly appreciated.

Answer 1

IIUC, setting the TCP_NODELAY parameter should clear all packets (i.e. tcp.c implements NODELAY configuration with tcp_push_pending_frames call). Therefore, if you set the socket parameter after each call, you should get what you want.

Answer 2

You cannot solve the problem if you are not sure what the problem is.

If they made a novice mistake by assuming that recv () receives exactly one message, I see no way to solve it completely. Sending only one message to one Ethernet port is one thing, but if several Ethernet frames arrive before the receiver calls recv (), it will still receive several messages in one call.

Network congestion makes it almost impossible to prevent this (while maintaining decent bandwidth), even if they can tell you how often they call recv ().

Answer 3

Perhaps set TCP_NODELAY and set your MTU low enough to have no more than 1 message per frame? Oh and add the dont-fragment flag to outgoing packets

Answer 4

Did you try to open a new socket for each message and immediately close it? Overhead can be disgusting, but this should limit your messages.

Answer 5

In the worst case scenario, you can go one level down (raw sockets), where you have better control over the sent packets, but then you have to deal with all nitty-gritty TCP.

Answer 6

Perhaps you could try putting the tcp stack in low latency mode:

echo 1 > /proc/sys/net/ipv4/tcp_low_latency 

This should help remove packages as quickly as possible when combining data. Read the person on tcp (7) for more information.