Automatically Scaling Pull Queue in Engine Application

Question

I am trying to implement a Push Notification system in PHP, which should send mass notifications with minimal delay, as described here

The drawback that I see in this architecture is how to automatically scale Notification Workers. As far as I know, there is no way to count pending tasks in the unload queue and not count active workers. How do you do this?

Answer 1

I solved this count of pending tasks in the notification queue using this method, and then added workers depending on the number of pending tasks:

public static function task_count($queue) { $request = new google\appengine\TaskQueueFetchQueueStatsRequest(); $response = new google\appengine\TaskQueueFetchQueueStatsResponse(); $request->addQueueName($queue); google\appengine\runtime\ApiProxy::makeSyncCall('taskqueue', 'FetchQueueStats', $request, $response); return $response->getQueueStats(0)->getNumTasks(); } 

Answer 2

There are queue statistics, where you can get the number of tasks in the queue https://cloud.google.com/appengine/docs/python/refdocs/google.appengine.api.taskqueue . We use it to control the number of tasks, perhaps it can be used for your purpose.