Faster Net::HTTP for Ruby 1.8.6

by john on December 20, 2008

We’ve been a bit frustrated at work with Net::HTTP performance (as have so many others) so here’s a monkeypatch for 1.8.6 that combines the buffer size increase in 1.8.7 with Aaron Patterson’s recent tweak to use non-blocking IO (unfortunately, the non-blocking IO patch doesn’t work with HTTPS, which is why we use the buffer size tweak when the @io variable suggests that HTTPS is happening. No guarantee implied, etc.


class Net::BufferedIO #:nodoc:

  alias :o ld_rbuf_fill :rbuf_fill

  def rbuf_fill

    BUFSIZE = 1024 * 16

    # HTTPS can't use the non-blocking strategy below in 1.8.6; so at least
    # increase buffer size over 1.8.6 default of 1024
    if !@io.respond_to? :read_nonblock
      timeout(@read_timeout) {
        @rbuf << @io.sysread(BUFSIZE)
      }
      return
    end

    # non-blocking
    begin
      @rbuf << @io.read_nonblock(BUFSIZE)
    rescue Errno::EWOULDBLOCK
      if IO.select([@io], nil, nil, @read_timeout)
        @rbuf << @io.read_nonblock(BUFSIZE)
      else
        raise Timeout::TimeoutError
      end
    end

  end

end

{ 1 comment… read it below or add one }

pete December 1, 2009 at 1:11 pm

I’ve tried this with some threaded net/http code on ruby 1.8.7 and I don’t see any difference in speed. Does this actually make net/http non-blocking?

Leave a Comment

Previous post: DNS failover: Short TTL vs Multiple A Records?

Next post: Juliana Hatfield, When I Grow Up (Book Review)