GAE: process terminated because the backend took too long to close backends

Question

My backend work is based on a cron job (every 4 hours). But it ends without data processing. The server log is displayed as follows:

500 15377121ms 0kb instance=0 AppEngine-Google; (+http://code.google.com/appengine) E 2012-10-05 01:50:18.044 Process terminated because the backend took too long to shutdown. 

How to deal with such an error in my program

Answer 1

This error occurs when the App Engine needs to close your backend, but the backend cannot exit within 30 seconds. Some reasons this may happen are listed here . Depending on the type of error, App Engine may notify your server of an impending shutdown, so it is recommended that you register a shutdown handler so that you can collect more data about the status of your application when this happens.

If you see this regularly, there may be a systematic explanation, such as job memory, exceeding the maximum for the base class.

Answer 2

Work with the same problem. I looked at the reasons listed in the official documents. Memory consumption looks normal from statistics. My code also handles issues with Datastore spores. Timeouts too. Changing the task mechanism to work in recoverable pieces seems to be the only way out.

After pursuing this error for a while, it seems the AppEngine development paradigm revolves around URL handlers with time, memory, etc. restrictions. This applies to operations with a long start. I gave my long-term task to small tasks. Target queues run smaller tasks that run before ending the queue with the next task. Never failed before!

The advantage is that taskqueuus is better fault tolerant / handover than just a huge cron job. One of the failed tasks does not mean that the rest of the huge list of tasks fails.