Linux precision timerfd

Question

I have a system that requires a timer of at least 10 ms.
I went to timerfd, because it suits me perfectly, but found that even for up to 15 milliseconds this is not entirely accurate, either that or I don’t understand how it works.

The time I measured was up to 21 ms on a 10 ms timer.
I put together a quick test that shows my problem.
Here's the test:

#include <sys/timerfd.h> #include <time.h> #include <string.h> #include <stdint.h> int main(int argc, char *argv[]){ int timerfd = timerfd_create(CLOCK_MONOTONIC,0); int milliseconds = atoi(argv[1]); struct itimerspec timspec; bzero(&timspec, sizeof(timspec)); timspec.it_interval.tv_sec = 0; timspec.it_interval.tv_nsec = milliseconds * 1000000; timspec.it_value.tv_sec = 0; timspec.it_value.tv_nsec = 1; int res = timerfd_settime(timerfd, 0, &timspec, 0); if(res < 0){ perror("timerfd_settime:"); } uint64_t expirations = 0; int iterations = 0; while( res = read(timerfd, &expirations, sizeof(expirations))){ if(res < 0){ perror("read:"); continue; } if(expirations > 1){ printf("%lld expirations, %d iterations\n", expirations, iterations); break; } iterations++; } } 

And it is executed as follows:

 Zack ~$ for i in 2 4 8 10 15; do echo "intervals of $i milliseconds"; ./test $i;done intervals of 2 milliseconds 2 expirations, 1 iterations intervals of 4 milliseconds 2 expirations, 6381 iterations intervals of 8 milliseconds 2 expirations, 21764 iterations intervals of 10 milliseconds 2 expirations, 1089 iterations intervals of 15 milliseconds 2 expirations, 3085 iterations 

Even assuming some possible delays, a 15 millisecond delay is too much for me.

Answer 1

Try changing it as follows, it should be quite a lot so that it never misses waking up, but be careful with it, since running a priority in real time can lock your machine if it is not sleeping, you may also need to set your user to have the ability to run material in priority in real time (see /etc/security/limits.conf )

 #include <sys/timerfd.h> #include <time.h> #include <string.h> #include <stdint.h> #include <stdio.h> #include <sched.h> int main(int argc, char *argv[]) { int timerfd = timerfd_create(CLOCK_MONOTONIC,0); int milliseconds = atoi(argv[1]); struct itimerspec timspec; struct sched_param schedparm; memset(&schedparm, 0, sizeof(schedparm)); schedparm.sched_priority = 1; // lowest rt priority sched_setscheduler(0, SCHED_FIFO, &schedparm); bzero(&timspec, sizeof(timspec)); timspec.it_interval.tv_sec = 0; timspec.it_interval.tv_nsec = milliseconds * 1000000; timspec.it_value.tv_sec = 0; timspec.it_value.tv_nsec = 1; int res = timerfd_settime(timerfd, 0, &timspec, 0); if(res < 0){ perror("timerfd_settime:"); } uint64_t expirations = 0; int iterations = 0; while( res = read(timerfd, &expirations, sizeof(expirations))){ if(res < 0){ perror("read:"); continue; } if(expirations > 1){ printf("%ld expirations, %d iterations\n", expirations, iterations); break; } iterations++; } } 

If you use threads, you should use pthread_setschedparam instead of sched_setscheduler .

In real time, it’s also not about low latency, about guarantees, RT means that if you want to wake up exactly once a second on the second, you WILL, normal planning does not give you this, it may decide to wake you up to 100 ms later, because in anyway, at that time he had a different job. If you want to wake up every 10 ms, and you really need to, then you must configure yourself as a task in real time, then the kernel will wake you up every 10 ms without crashing. If a task with a higher priority in real time is busy.

If you need to ensure that your wake-up interval is exactly some time, it does not matter if it is 1 ms or 1 second, you will not receive it if you do not perform the task in real time. There are good reasons why the kernel will do this with you (energy saving is one of them, higher throughput is the other, there are others), but it has rights to it, since you never talked about it You need the best warranty. Most things do not really have to be so precise or you don’t need to miss, so you should consider whether you really need it.

quote from http://www.ganssle.com/articles/realtime.htm

A tough task or real-time system where an activity simply needs to be completed - always - by a specified deadline. The deadline may be a specific time or time interval, or may be the arrival of an event. Hard real-time tasks fail, by definition, if they miss such a deadline.

Note that this definition does not make an assumption about the frequency or duration of tasks. A microsecond or a week - if there is not enough time, it induces a failure, then the task is strict requirements for real time.

Soft real time is almost the same, except that a missed deadline, while undesirable, is not the end of the world (for example, playing video and audio are real-time tasks in real time, you do not want to skip the frame display, or the buffer ends, but if you do it just an instant hiccup, and you just continue). If what you are trying to do is soft in real time, I would not work with real-time priority, since you usually need to get your awakenings on time (or at least close to it).

EDIT:

If you do not work in real time, the kernel will by default give you some timers that you will make "weak" so that it can combine your request in order to wake up with other events that happen sometimes close to the one you asked (if this other event is within your "weak" time, it will not wake you up at the time you asked, but a little earlier or later, at the same time he is already going to do something else, it saves energy) .

For more information, see High (but not too high) permission timeouts and Timer slack (note that I'm not sure that either of these things is exactly what is really in the kernel, since both of these articles are related to discussions on the lkml mailing list, but something like the first is really in the kernel.

Answer 2

I have the feeling that your test is very hardware dependent. When I ran your trial program on my system, it seemed to freeze in 1 ms. To make your test meaningful on my computer, I had to switch from milliseconds to microseconds. (I changed the multiplier from 1_000_000 to 1_000.)

  $ grep 1000 test.c
     timspec.it_interval.tv_nsec = microseconds * 1000;

  $ for i in 1 2 4 5 7 8 9 15 16 17 \
  31 32 33 47 48 49 63 64 65;  do \
  echo "intervals of $ i microseconds"; \
  ./test $ i; done
 intervals of 1 microseconds
 11 expirations, 0 iterations
 intervals of 2 microseconds
 5 expirations, 0 iterations
 intervals of 4 microseconds
 3 expirations, 0 iterations
 intervals of 5 microseconds
 2 expirations, 0 iterations
 intervals of 7 microseconds
 2 expirations, 0 iterations
 intervals of 8 microseconds
 2 expirations, 0 iterations
 intervals of 9 microseconds
 2 expirations, 0 iterations
 intervals of 15 microseconds
 2 expirations, 7788 iterations
 intervals of 16 microseconds
 4 expirations, 1646767 iterations
 intervals of 17 microseconds
 2 expirations, 597 iterations
 intervals of 31 microseconds
 2 expirations, 370969 iterations
 intervals of 32 microseconds
 2 expirations, 163167 iterations
 intervals of 33 microseconds
 2 expirations, 3267 iterations
 intervals of 47 microseconds
 2 expirations, 1913584 iterations
 intervals of 48 microseconds
 2 expirations, 31 iterations
 intervals of 49 microseconds
 2 expirations, 17852 iterations
 intervals of 63 microseconds
 2 expirations, 24 iterations
 intervals of 64 microseconds
 2 expirations, 2888 iterations
 intervals of 65 microseconds
 2 expirations, 37668 iterations

(It's a little interesting that I got the longest runs with 16 and 47 microseconds, but 17 and 48 were terrible.)

time (7) contains some suggestions on why our platforms are so different:

  High-resolution timers
        Before Linux 2.6.21, the accuracy of timer and sleep system
        calls (see below) was also limited by the size of the jiffy.

        Since Linux 2.6.21, Linux supports high-resolution timers
        (HRTs), optionally configurable via CONFIG_HIGH_RES_TIMERS.  On
        a system that supports HRTs, the accuracy of sleep and timer
        system calls is no longer constrained by the jiffy, but instead
        can be as accurate as the hardware allows (microsecond accuracy
        is typical of modern hardware).  You can determine whether
        high-resolution timers are supported by checking the resolution
        returned by a call to clock_getres (2) or looking at the
        "resolution" entries in / proc / timer_list.

        HRTs are not supported on all hardware architectures.  (Support
        is provided on x86, arm, and powerpc, among others.)

All the "permission" lines in my / proc / timer _list are 1 ns on my (admittedly incredibly powerful) x86_64 system.

I decided to try to find out where the "breakpoint" is on my computer, but refused to execute 110 microseconds:

  $ for i in 70 80 90 100 110 120 120 \
  ;  do echo "intervals of $ i microseconds"; \
  ./test $ i; done
 intervals of 70 microseconds
 2 expirations, 639236 iterations
 intervals of 80 microseconds
 2 expirations, 150304 iterations
 intervals of 90 microseconds
 4 expirations, 3368248 iterations
 intervals of 100 microseconds
 4 expirations, 1964857 iterations
 intervals of 110 microseconds
 ^ C

90 microseconds ran three million iterations before they failed several times; which is 22 times better than your first test, so I would say that with the right equipment, 10 ms should not be somewhere close to difficulties. (90 microseconds is 111 times better than 10 milliseconds).

But if your equipment does not provide timers for high-resolution timers, Linux will not be able to help you without resorting to SCHED_RR or SCHED_FIFO. And even then, perhaps another kernel could better provide you with the necessary software timer support.

Good luck. :)

Answer 3

Here is the theory. If HZ is set to 250 for your system (as usual), then you have a resolution of 4 milliseconds for the timer. After your process is replaced by a scheduler, it is likely that a number of other processes will be scheduled and started before your process receives another time slice. This may explain that you see timer resolutions in the range of 15 to 21 milliseconds. The only way around this is to start the real-time kernel.

A typical high-resolution synchronization solution for non-real-time systems, mainly for lively waiting with a challenge to choose from.

Answer 4

Depending on what else the system does, this may be a little slow when switching back to your task. Unless you have a “real” real-time system, there is no guarantee that it will be better than what you see, although I agree that the result is a little disappointing.

You can (basically) exclude task / scheduler switching times. If you have processor power (and electrical power!), To get rid of, a cruel but effective solution would be a busy waiting loop.

The idea is to run your program in a narrow loop that continuously checks the clock for what time, and then calls your other code when the time is right. Due to the fact that your system is very slow for everything else and heats up your processor, you will end up planning tasks that are mostly free from jitter.

I wrote a system like this once under Windows XP to rotate a stepper motor, providing evenly distributed pulses up to 40K times per second, and it worked fine. Of course, your mileage may vary.