Rails and concurrent request handling

In the kerfluffle over Heroku’s routing logic and Rails, a lot of people have been saying “The problem is Rails does not support concurrent request handling, a single Rails process can only handle one request at a time.” Many people use this as a reason to say how ridiciulous Rails is.

And Heroku repeated this in their explaination of the problem.

The problem is, it’s not really true to say “Rails doesn’t support concurrent request handling.”  It used to be true of Rails. But Rails has said it supported concurrent request handling using threads for quite some time, with the `config.threadsafe!` directive (existing for over 4 years, since rails 2.2 at least), which is also currently slated to be on by default in Rails4. 

Now, it may be that Rails still has issues under multi-threaded request handling. Certainly Rails has had problems with multi-threading before, has had bugs, including some rather confusing ones I’ve spent way too much time myself with ActiveRecord concurrency bugs or mis-designs.  So it’s surely possible there are still bugs and problems, especially since config.threadsafe! has historically not gotten a huge amount of use.

Rails apparently thinks it handles multi-threaded request handling fine now, with the plan to make `config.threadsafe!` on by default in Rails4. Anyone that has reason to believe otherwise should let us all (including Rails core team) know, not just irresponsibly repeat some rumor you heard that “Rails can’t do concurrent request handling”, that’s just FUD.

And it’s also true that the app server stacks for Rails (Passenger, thin, etc) haven’t historically handled multi-threaded request concurrency well. You have limited options (and more limited confidence in the maturity of the options) if you want multi-threaded concurrent requests in your app stack. (For instance, the free Passenger does not do it. Passenger Enterprise either does or will in 4, I forget which. But they have no plans to support such in free Passenger, which is awfully sad).

So, yeah, it’s complicated. And yeah, Rails has been a bit later to this party than some of us would like, and, from being burned before, I am not entirely confident Rails and Rails-compatible app stacks are completely mature here yet (neither am I sure they are not).

(I’d also like to make it clear that I don’t think Heroku’s current design is particularly idiotic as some people are saying. It has become clear in retrospect that it is not sufficient, but I understand how smart people got to where they got without realizing it. They probably should have noticed the problem earlier though, since customers were trying to report it. But at this point, what matters is how they handle it.)

Here’s my response to Heroku’s explanation I posted as a comment on their blog.

---

Thanks for the update. I think it’s important to clarify one thing though. You say:

Rails, in fact, does not yet reliably support concurrent request handling.

I am not sure what you mean by this. Rails is documented to support concurrent request handling with the `config.threadsafe!` directive, which is also slated to be on by default in Rails 4.

Are you saying you have reason to believe this setting does not in fact lead to reliable behavior? (I could believe that, but I’m not sure if that’s what you’re saying). Or were you unaware of config.threadsafe! ?

Or, it is true that in addition to config.threadsafe!, one needs a web/app server stack that can handle concurrent request handling, rather than handling requests only one-by-one serially. Heroku’s documented suggested example deployment stacks probably do not do this. Puma or even thin with multi-threaded handling turned on theoretically do — but are you saying that you have reason to believe this app server stacks are not yet reliable?

It’s also often said that under MRI, multiple threads can’t actually use multiple CPU’s concurrently. But this shouldn’t be an issue for the heroku use case, since a single dyno only has access to one cpu core, I think?

Now even if you have an app stack that supports multi-threaded concurrent request handling with Rails config.threadsafe!, it’s not entirely clear that the Heroku routing algorithms you describe would work even still. Some people are suggesting on the net that even with concurrent Rails request handling, they’ve done simulations that show that an effectively ‘random’ routing algorithm would still run into the same problems — just at a higher level of traffic.

It’s really hard for the non-expert to figure this stuff out. I suggest that Heroku should figure this stuff out, and document it for your users. Figure out what method of concurrent rails request handling, if any, at the moment IS reliable (Puma with config.threadsafe!) by doing your own research/tests. Figure out how well configuration performs under Heroku’s current routing — if you get Rails doing concurrent request handling, IS that enough to make Heroku’s current cedar routing perform and scale properly? Figure this all out and let us all know, rather than leave less expert customers to fend for themselves each individually trying to figure out this admittedly very confusing topic.

And stop repeating vague things like “Rails, in fact, does not yet reliably support concurrent request handling,” which just sounds like FUD, or passing the buck, when stated without context or explanation here. Rails says it’s supports concurrent request handling, you have to explain why you are saying it doesn’t.