I am very curious to learn about a recent problem that I encountered with all instances of EC2 with MySQL. On July 1, at 12:00 AM UTC, both of my Amazon EC2 instances (operating in the US-EAST region) issued an alarm indicating a high processor load. I researched to find that it is MySQL that eats the entire processor. I logged in and ran SHOW PROCESSLIST to find that there were no requests (these servers do not hit hard after working hours). I stopped MySQL, CPU usage dropped to 1-3% (as usual). I restarted MySQL and it started to eat a lot of CPU again. Then I restarted the shutdown -r now server and the problem disappeared. Both servers had the same problem (running Ubuntu 12.04 LTS).
The only relevant item that I see in syslog:
Jun 30 23:59:59 hostname kernel: [14152976.187987] Clock: inserting leap second 23:59:60 UTC
I don’t think I have registration enabled for MySQL; regardless of the default settings for Ubuntu 12.04 - this is what I use. If that matters, most InnoDB databases.
The day before, one instance was affected by an EBS connection problem, which is why the server reacted very slowly when accessing the disk. I do not think that another instance was implemented (it is in a different AWS account and responded well during the “disconnect”), but I cannot be 100% sure. I wonder if this could make MySQL go crazy? But why wait until midnight? (Beware! It could be the "red herring" in this secret)
Any thoughts on what could cause this problem?
source share