Is it possible to initiate multiple concurrent HTTP requests using EventMachine with Ruby 1.8

Question

em-synchrony.rb implements this function using Fibers, but I would go for a non-Fiber version with 1.8 MRI.

EM.run do http = EM::Protocols::HttpClient2.connect("www.google.com", 80) request = http.get("/") request.callback do puts request.status EM.stop end end 

Answer 1

See em-http-request :

 EM.run do http1 = EventMachine::HttpRequest.new('http://example.com/1').get http1.callback do p http1.response end http2 = EventMachine::HttpRequest.new('http://example.com/2').get http2.callback do p http2.response end end 

Answer 2

If you can look outside EventMachine, Typhoeus is an easy-to-use HTTP client that comes with Hydra, which makes it possible to process multiple requests in parallel.

I used it for several things, and it is easy to set up. This is unverified code torn from what I wrote the other day:

 require 'typhoeus' hydra = Typhoeus::Hydra.new(:max_concurrency => 10) urls.each do |url| request = Typhoeus::Request.new(url) request.on_complete do |resp| filename = resp.request.url.split('/').last puts "writing #{filename}" File.open(filename, 'w') do |fo| fo.write resp.body end end puts "queuing #{ url }" hydra.queue(request) end hydra.run