How to disable delayed work so as not to annoy the Facebook API

Question

I am creating an application that will hit graphics on Facebook badly. I found out that they have a speed limit of 600 requests every 600 seconds.

I use delayed work for all my background processing. What is a good way to schedule deferred work in order to stay under the fb api limit? Are there any tricks with a delayed task or do I need to create a separate background task processor so as not to go over my bid limit?

thanks

Answer 1

600 requests every 600 seconds - 1 time per second on avg.

Not very fast!

1) Depending on the size of your company and level, I would check the FB to see if you can get the limit raised for you.

2) You can stick with DelayedJob, no need to reinvent the wheel. You just need to change the scheduler.

In my DelayedJob installation, I use the "run_at" column more than just setting the time to repeat jobs. I also use it as a time to start a task in the first place. You can also use it to throttle your jobs.

Changed in file DelayedJob job.rb:

# added run_at param # eg Delayed::Job.enqueue NewsletterJob.new('lorem ipsum...'), 0, # Delayed::Job.db_time_now + 15.minutes def self.enqueue(object, priority = 0, run_at = nil) unless object.respond_to?(:perform) raise ArgumentError, 'Cannot enqueue items which do not respond to perform' end Job.create(:payload_object => object, :priority => priority, :run_at => run_at) end 

For your purpose, I would track the last time the FB api call was called, and assign the next version of run_at to the time, at least for a second.

Advantage: you can alternate other non-FB tasks using FB-api calls.

Answer 2

A bit shameless plugin, but you can try SimpleWorker, a cloud-based background processing / work queue for Ruby applications. You can schedule one or more tasks to break away from the queue and press FB api when you need to. All schedule and queue management is handled by SimpleWorker, and processing is also done in the cloud.

It is designed for this type of use only.

Suggest you also check out the mini_fb gem for working with FB (Appoxy is the creator and maintainer).

Let us know if you need any help.

Ken @SimpleWorker