Rails and concurrent request handling

In the kerfluffle over Heroku’s routing logic and Rails, a lot of people have been saying “The problem is Rails does not support concurrent request handling, a single Rails process can only handle one request at a time.” Many people use this as a reason to say how ridiciulous Rails is.

And Heroku repeated this in their explaination of the problem.

The problem is, it’s not really true to say “Rails doesn’t support concurrent request handling.”  It used to be true of Rails. But Rails has said it supported concurrent request handling using threads for quite some time, with the `config.threadsafe!` directive (existing for over 4 years, since rails 2.2 at least), which is also currently slated to be on by default in Rails4. 

Now, it may be that Rails still has issues under multi-threaded request handling. Certainly Rails has had problems with multi-threading before, has had bugs, including some rather confusing ones I’ve spent way too much time myself with ActiveRecord concurrency bugs or mis-designs.  So it’s surely possible there are still bugs and problems, especially since config.threadsafe! has historically not gotten a huge amount of use.

Rails apparently thinks it handles multi-threaded request handling fine now, with the plan to make `config.threadsafe!` on by default in Rails4. Anyone that has reason to believe otherwise should let us all (including Rails core team) know, not just irresponsibly repeat some rumor you heard that “Rails can’t do concurrent request handling”, that’s just FUD.

And it’s also true that the app server stacks for Rails (Passenger, thin, etc) haven’t historically handled multi-threaded request concurrency well. You have limited options (and more limited confidence in the maturity of the options) if you want multi-threaded concurrent requests in your app stack. (For instance, the free Passenger does not do it. Passenger Enterprise either does or will in 4, I forget which. But they have no plans to support such in free Passenger, which is awfully sad).

So, yeah, it’s complicated. And yeah, Rails has been a bit later to this party than some of us would like, and, from being burned before, I am not entirely confident Rails and Rails-compatible app stacks are completely mature here yet (neither am I sure they are not).

(I’d also like to make it clear that I don’t think Heroku’s current design is particularly idiotic as some people are saying. It has become clear in retrospect that it is not sufficient, but I understand how smart people got to where they got without realizing it. They probably should have noticed the problem earlier though, since customers were trying to report it. But at this point, what matters is how they handle it.)

Here’s my response to Heroku’s explanation I posted as a comment on their blog.

Thanks for the update. I think it’s important to clarify one thing though. You say:

Rails, in fact, does not yet reliably support concurrent request handling.

I am not sure what you mean by this. Rails is documented to support concurrent request handling with the `config.threadsafe!` directive, which is also slated to be on by default in Rails 4.

Are you saying you have reason to believe this setting does not in fact lead to reliable behavior? (I could believe that, but I’m not sure if that’s what you’re saying). Or were you unaware of config.threadsafe! ?

Or, it is true that in addition to config.threadsafe!, one needs a web/app server stack that can handle concurrent request handling, rather than handling requests only one-by-one serially. Heroku’s documented suggested example deployment stacks probably do not do this. Puma or even thin with multi-threaded handling turned on theoretically do — but are you saying that you have reason to believe this app server stacks are not yet reliable?

It’s also often said that under MRI, multiple threads can’t actually use multiple CPU’s concurrently. But this shouldn’t be an issue for the heroku use case, since a single dyno only has access to one cpu core, I think?

Now even if you have an app stack that supports multi-threaded concurrent request handling with Rails config.threadsafe!, it’s not entirely clear that the Heroku routing algorithms you describe would work even still. Some people are suggesting on the net that even with concurrent Rails request handling, they’ve done simulations that show that an effectively ‘random’ routing algorithm would still run into the same problems — just at a higher level of traffic.

It’s really hard for the non-expert to figure this stuff out. I suggest that Heroku should figure this stuff out, and document it for your users. Figure out what method of concurrent rails request handling, if any, at the moment IS reliable (Puma with config.threadsafe!) by doing your own research/tests. Figure out how well configuration performs under Heroku’s current routing — if you get Rails doing concurrent request handling, IS that enough to make Heroku’s current cedar routing perform and scale properly? Figure this all out and let us all know, rather than leave less expert customers to fend for themselves each individually trying to figure out this admittedly very confusing topic.

And stop repeating vague things like “Rails, in fact, does not yet reliably support concurrent request handling,” which just sounds like FUD, or passing the buck, when stated without context or explanation here. Rails says it’s supports concurrent request handling, you have to explain why you are saying it doesn’t.


9 thoughts on “Rails and concurrent request handling”

  1. Following this issue, I’m happy with my stack choice: C#, ASP.NET, IIS, Azure. Very efficient and solid with strong concurrency and scalability support. I suppose the “just throw more servers at it” mindset can only take you so far.

  2. If you’re happy with what you’ve got, that’s great. I’m happy with Rails, myself, and am always happier with open source where I can see the code to diagnose bugs and submit patches when needed. But I suppose the “everything from Microsoft and only from Microsoft” mindset can take you far enough, for the right kinds of projects.

    It’s my blog, so I’ll delete any further random misinformed Rails trashing int the comments here that I don’t feel to be productive or interesting (yeah, I’ve got editorial discretion over what’s productive or interesting on my own blog), but feel free to randomly trash Rails on various HN or reddit threads, if you feel so called!

  3. Interesting, thanks Stefan, I did not know that. So jruby with concurrent request handling probably WOULD perform better than MRI with concurrent request handling. Due to jruby’s ability to run threads in simultaneous parallel accross cores (no GIL).

    Now, if you run thin with four process forks (for each of the cores), with each process then dispatching multi-threaded concurrent requests to Rails…. it seems quite possible that would have the same or nearly the same capacity and latency as jruby. But it would have to be simulated and tested in practice. This stuff IS confusing.

    I don’t neccesarily think Heroku’s design is wrong, it may very well turn out to be the closest to optimal compromise. What they’re mostly guilty of is insufficient documention on performance characteristics, best practices, and options — and insuffiicent research into their own stack’s actual performance with actual rails ecosystem (a large part of their customer base and what they’re most famous for supporting) to write that documentation.

  4. Phusion Passenger 4 Enterprise supports multithreading. Phusion Passenger 4 open source and Phusion Passenger 3 Enterprise don’t support it.

    Regarding the maturity multithreading support in Ruby + Rails: it works, and it works well. There are no known multithreading bugs in Rails. In my mind it’s pretty simple, but maybe I’m too familiar with the ecosystem to see how others can perceive it as confusing. Here’s a (as far as I know) complete checklist of using multithreading. If anything’s still unclear, please feel free to ask because I would love to explain further.

    – Enable `config.thread_safe`.
    – Use a multithreading-capable app server, for example Phusion Passenger 4 Enterprise. :) Thin is *evented*, not multithreaded, so that doesn’t work if you want multithreading. Thin has experimental multithreading support but I’m not sure how reliable it is.
    – When using MRI, use multithreading *and* multiprocessing, e.g. what Phusion Passenger 4 Enterprise offers. Multithreading is for handling concurrency, multiprocessing is for handling multiple CPU cores.
    – When using non-MRI (e.g. JRuby) using multithreading and a single process is enough.
    – Do not use the ‘mysql’ database driver, which is freezes threads while doing its work. Use the ‘mysql2’ database driver instead.

  5. Hongli, thanks for your comment.

    I think the most confusing thing at the moment is figuring out what app server stack supports multi-threaded request handling (reliably and robustly).

    For instance, you are mistaken — thin does have a multi-threaded mode (as opposed to evented mode) too, which at one point was marked ‘experimenta’l, but I think is not any longer (or soon won’t be?). But it IS poorly documented. See the end of this thread with an exchange between me and thin dev: https://groups.google.com/d/msg/thin-ruby/0hMub7_dWzQ/UtgmNeLKOf8J

    Few app server stacks are as easy to get going with as Passenger though. I love Passenger.

    Your choice to reserve multi-threaded concurrent request dispatch for the ‘enterprise’ version… will either be great for you, a push for your customers to pay for ‘enterprise’, or it’ll be a sad drag on uptake of concurrent request dispatch in Rails in the community (since many will stick with free passenger and not have it available). Or a little bit of both.

  6. I’ve followed your work on threading in Ruby with interest, though I have to say it’s sometimes over my head. :) There’s a new book coming out on threading in Ruby that might be worth a look given your interest in the topic: http://www.workingwithrubythreads.com. Not trying to sell you anything, just thought you might be interested.

    Keep the blog posts coming. They’re great. Enjoyed your Code4Lib talk as well.

  7. Yeah, I was pleased to see that book, more evidence of rising interest in writing multi-threaded code in ruby.

    On Thu, Feb 21, 2013 at 8:29 PM, Bibliographic Wilderness

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s