Hydra manages making parallel HTTP requests. This is achieved by using libcurls multi interface: curl.haxx.se/libcurl/c/libcurl-multi.html The benefits are that you don't have to worry running the requests by yourself.
Hydra will also handle how many requests you can make in parallel. Things will get flakey if you try to make too many requests at the same time. The built in limit is 200. When more requests than that are queued up, hydra will save them for later and start the requests as others are finished. You can raise or lower the concurrency limit through the Hydra constructor.
Regarding the asynchronous behavior of the hydra, it is important to know that this is completely hidden from the developer and you are free to apply whatever technique you want to your code. That should not conflict with libcurls internal concurrency mechanism.
@example Use the hydra to do multiple requests.
hydra = Typhoeus::Hydra.new requests = (0..9).map{ Typhoeus::Request.new("www.example.com") } requests.each{ |request| hydra.queue(request) } hydra.run
@note Callbacks are going to delay the request
execution.
@example Set max_concurrency.
Typhoeus::Hydra.new(max_concurrency: 20)
@api private
Returns a memoized hydra instance.
@example Get a hydra.
Typhoeus::Hydra.hydra
@return [Typhoeus::Hydra] A new hydra.
@deprecated This is only for convenience because so
much external code relies on it.
# File lib/typhoeus/hydra.rb, line 69 def hydra @hydra ||= new end
Create a new hydra. All {rubydoc.info/github/typhoeus/ethon/Ethon/Multi#initialize-instance_method Ethon::Multi#initialize} options are also available.
@example Create a hydra.
Typhoeus::Hydra.new
@example Create a hydra with max_concurrency.
Typhoeus::Hydra.new(max_concurrency: 20)
@param [ Hash ] options The options hash.
@option options :max_concurrency [ Integer ] Number
of max concurrent connections to create. Default is 200.
@see rubydoc.info/github/typhoeus/ethon/Ethon/Multi#initialize-instance_method
Ethon::Multi#initialize
# File lib/typhoeus/hydra.rb, line 92 def initialize(options = {}) @options = options @max_concurrency = @options.fetch(:max_concurrency, 200) @multi = Ethon::Multi.new(options.reject{|k,_| k==:max_concurrency}) end