I have a real-time process that periodically sends messages through RS232 to a high-speed camera. I have several other real-time processes that take up a lot of CPU time, doing image processing on multiple GPU boards using CUDA. Typically, serial communication is very fast, with a message and response taking about 50 ms each time. However, when background processes are busy processing images, serial communication slows down, often taking several seconds (sometimes more than 10 seconds).
Thus, during serial communication, process A is delayed if processes B, C, etc. very busy, although process A has the highest priority:
- Process A (Real Time, Highest Priority): Random Serial Communication
- Processes B, C, D, etc. (in real time, lower priority): processing of large processors and GPUs.
When I change background processes to SCHED_OTHER processes (not real-time), serial communication is fast; however, this is not a solution for me, because background processes should be in real time (when this is not the case, the GPU processing does not meet the requirements of a high-speed camera).
Apparently, the serial connection relies on some process that is not related to real time in the system, which is anticipated by my background processes in real time. I think if I knew which process is used for serial communication, I could increase its priority and solve the problem. Does anyone know if serial communication relies on any particular process running on the system?
I am running RHEL 6.5 with a standard kernel (not PREEMPT_RT). It is equipped with two 6-core processors.
In Erki's suggestion, I grabbed strace. Apparently, this is the select () system call, which is slow ("set roi2" is the command for the camera, and "Ok!" In the end is the response from the camera):
write(9, "set roi2"..., 26) = 26 <0.001106>
ioctl(9, TCSBRK, 0x1) = 0 <0.000263>
select(10, [9], NULL, NULL, {2, 0}) = 1 (in [9], left {0, 0}) <2.252840>
read(9, "Ok!\r\n", 4096) = 5 <0.000092>
select() , . , , , . select() ?
select() read(), read() .