Geek Answers Handbook

Tasks for handling HardDeadlineExceededError

I have a series of tasks / servlets that click on a HardDeadlineExceededError that leaves everything that hangs in the "still executable" state.

Work in progress can easily exceed the 29 second threshold.

I am trying to catch a DeadlineExceededException and a base exception in order to save the output but none of these exception handlers get caught ...

Is there a way to determine which tasks are in the queue or are currently running?

Are there other strategies to deal with this situation?

The situation I'm dealing with is documented in the Request Timer .

// task handler for retrieving information from external web services protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException { String taskRetryCountParam = req.getParameter( "X-AppEngine-TaskRetryCount" ); int taskRetryCount = (taskRetryCountParam==null) ? 0 : Integer.parseInt(taskRetryCountParam); // look up the persistent 'task' and mark it as 'running' logger.info( this.getClass().getName() + ".doPost( "+ taskId + " ) retryCount=" + taskRestryCount ); // Do lots of heavy lifting here // like calling external web services using URL fetch service // and saving the contents into our database. // look up the persistent 'task' and mark it as 'completed' } catch ( DeadlineExceededException deadline ) { // got this deadline exception // look up the persistent 'task' and mark it as 'errored - try again' logger.warning( "DeadlineExceeded Exception while loading content " + deadline.getMessage() ); resp.setStatus( HttpServletResponse.SC_REQUEST_TIMEOUT ); } } catch ( Exception unknown ) { // got some unknown exception // look up the persistent 'task' and mark it as 'errored - cancelled' logger.severe( "General Exception while loading content exception:" + unknown.getMessage() ); resp.setStatus( HttpServletResponse.SC_OK ); } }

Here are the entries in the log file when I get into this situation ... It seems that my database transactions are too long when the time comes.

  W 05-30 12:42PM 09.535 Error for /loadstatus com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC. at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:443) at java.util.concurrent.TimeUnit.timedWait(Unknown Source) at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60) at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326) at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217) at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131) at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102) at java.security.AccessController.doPrivileged(Native Method) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43) at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58) at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42) at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56) at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66) at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61) at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88) at org.datanucleus.transaction.Transaction.commit(Transaction.java:149) at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95) at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390) at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258) at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83) at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) <snip> C 05-30 12:42PM 09.629 Uncaught exception from servlet com.google.apphosting.runtime.HardDeadlineExceededError: This request (083793d1091c2ca3) started at 2010/05/30 19:41:39.814 UTC and was still executing at 2010/05/30 19:42:09.529 UTC. at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:443) at java.util.concurrent.TimeUnit.timedWait(Unknown Source) at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:60) at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:326) at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:217) at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:131) at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:43) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:104) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:102) at java.security.AccessController.doPrivileged(Native Method) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:102) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:43) at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:58) at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:42) at com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java:56) at com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java:66) at org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java:61) at org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java:88) at org.datanucleus.transaction.Transaction.commit(Transaction.java:149) at org.datanucleus.transaction.TransactionManager.commit(TransactionManager.java:95) at org.datanucleus.TransactionImpl.internalCommit(TransactionImpl.java:390) at org.datanucleus.TransactionImpl.commit(TransactionImpl.java:258) at org.datanucleus.jdo.JDOTransaction.commit(JDOTransaction.java:83) at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.commit(DatastoreJDOTransaction.java:56) <snip> W 05-30 12:42PM 09.644 A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)
+7
java google-app-engine scheduled-tasks servlets
source share
1 answer

http://groups.google.com/group/google-appengine-java/msg/e3fd2b621bb96013

HDEE can be thrown without DEE if this happens in your own code.
Usually the most time-consuming thing is waiting for API calls on
return, so a timeout here will cause the API call to stop using DEE. Therefore, if you do not call the API often, you can directly get into HDEE.

I also have long-running tasks that iterate through data processing
and saving results. I use an Iterator that stops returning results
after 20 seconds and saves the last processed object, and then kicks
with a new task to continue processing.

My original solution caught DEE and then cleared, but it
stopped working reliably.

+3
source share

More articles:


All Articles