Client client error "Performing a previous run check"

Question

I am using the latest version of the sensational version 0.26.1, configured sensu server on one CentOS machine, configured one sensu client on another.

There are about 500 checks in one sensu client, I see the "previous check command execution in progress" in sensu-client.log, but each check actually ends very quickly (most of them are less than 0.1 seconds and the interval is 60 seconds), I can confirm this by running it under sensu sudo su sensu -c "{run my check}" .

However, the actual situation on the uchiwa control panel showed that many of my checks were performed for more than 1 minute , it seems that the sensu client was seriously stuck / slowed down, I tried the following approaches

• Remove some lengthy checks and restart sensu server / client
• Add a timeout for my checks, limit the wait time to 10, and then lead to many "Run time out" checks.
• I tried only to run 1, 10, 50 checks, it seems that everything is working fine, however, as soon as the number of checks reaches a certain number, maybe 200-300, a problem arose.

None of the above work exists, is there a way to debug, which checks actually block? Or can I configure sensu to just kill the check when it exceeds the timeout definition? So I will not see this message in the log "previous execution of the validation command"

I am blocked by this and need help :)

Answer 1

To debug this, you can obviously check sensu-client.log. You can set the log level in the default file for sensu

also: ps aux | grep sensu ps aux | grep sensu will show you that each check will be displayed as a separate process, and you can easily manually confirm whether and what checks may hang. A common problem here is that there are flaws in the checks that make them freeze than the sense client itself.

I experienced this with some checks.

To get more detailed information about your problem, you will need at least some log output and the ps command [if a problem occurs].