gem depends on Rails engine gem gotcha: Need explicit require

Let’s say you have a gem WidgetGem, which may or may not be a Rails engine.

Let’s say you have another gem, which is a Rails engine, WidgetSupport.

Let’s say WidgetGem includes WidgetSupportEngine as a gem dependency, using the ordinary means of listing it in widget_gem.gemspec with add_dependency.

Now you’ve got a Rails app MyApp, which uses WidgetGem, just lists it in the MyApp Gemfile, as usual.

You would expect that WidgetSupportEngine would be loaded as a Rails Engine into MyApp, since it’s a dependency of your dependency, it should just work, right? Nope, you’d be wrong. WidgetSupportEngine will be available as a gem, but it won’t actually have been ‘activated’ as an engine, any engine-specific features won’t have been executed. (For instance, if it declared Rails initializers, they won’t have run; for Rails engine gems, the engine’s ./app/* is usually on the load path for the host app, but not now, etc.)

What do you have to do to fix this? You have to have WidgetGem explicitly require WidgetSupportEngine in some file that is loaded on gem require for WidgetGem. For instance, typically WidgetGem’s ./lib/widget_gem.rb, or any file that widget_gem.rb explicitly requires (like, if WidgetGem is a Rails engine, typically ./lib/widget_gem/engine.rb): require 'widget_support_engine'.

Then it’ll work as expected again. When WidgetGem, a gem itself, has a dependency on a Rails engine gem, and you want that rails engine gem to be succesfully loaded in Rails apps that use WidgetGem — the first gem needs to not only express the gem as a dependency, but also explicitly require it.

How come?

When your Rails app boots, all of it’s immediate gem dependencies are not only in the load path, but those first order gems are also “require”d on boot, they are loaded. (I am not quite sure if it’s rubygems directly that’s doing this, or bundler, or some part of Rails, or what).

However, second-order or N-order gems, dependencies of your dependencies, are not require’d on boot. Their lib directories are in the load path, sure, but they aren’t required on boot.

If you load up the rails irb console in MyApp in the above scenario, you can type WidgetGem, and, yep, that’s defined. But if you type WidgetSupportEngine, no such class. It has not automatically been require’d by the boot process. Oh, okay, type require 'widget_support_engine', that works fine (and returns true to tell you indeed it hadn’t been reuqired before), the WidgetSupportEngine is in the load path, and now it exists, type WidgetSupportEngine.

But at this point, Rails has already booted, loading WidgetSupportEngine now is too late. No exceptions will be raised, but all of WidgetSupportEngine’s Rails engine behavior is just no-op, it’s just an ordinary gem now, not a Rails engine gem, it’s got no engine powers. Oops.

Having the first order dependency WidgetGem explicitly require makes sure that WidgetSupportEngine gets loaded before Rails boot — at the point WidgetGem itself is automatically required — so it’s engine powers can be activated.

Has rubygems or rubygems/bundler changed?

I could swear this used to work differently. And all gems in your dependency tree got require‘d on boot.

But now only the “immediate” dependencies got require‘d on boot — dependencies of those dependencies are on the load path, but not required.

Am I imagining things, and it never used to work that other way? Or has something changed? In rubygems, in bundler, in rails, in something else? Anyone know, and when/why?

The effect of this change is to break my own mental model of gem dependencies a bit. Ordinarily, it doesn’t matter if a gem is expressed as a first-level dependency, or a dependency of one of your first-level dependencies, and so on, down many levels. It behaves exactly the same either way.

Sure, load order can matter sometimes, and lead to odd bugs. And in fact, I spent a while assuming my problem was a load order problem, as well as a bunch of other hypotheses, before realizing it wasn’t load order, it was that the second-order gem wasn’t being require’d at all until app code did the requiring.

This change means that it’s not always the same semantics if a gem is expressed as first-order dependency, or a dependency of a dependency. Because the first-order ones get automatically require‘d, and the other ones don’t, and that can sometimes lead to differences in behavior, like with Rails engines. If you start having more than two levels in your dependency tree, with potential Rails engines out near the bottom… things can get kind of awful.

It makes the gem dependency abstraction a bit less foolproof, a bit pricklier around the edge cases. I spent about 3.5 hours figuring this one out, hopefully this post will show up on google and save someone else the trouble. (I was googling around for any help on this issue, unsuccesfully. I did find people discussing Rails engine gems as dependencies of other gems, but none mentioned this gotcha, which further makes me think at some point in the past gem dependency require worked differently).


3 thoughts on “gem depends on Rails engine gem gotcha: Need explicit require

  1. Google pointed me here. I just me the same issue and I had to ‘require my_engine’ in the configure/application.rb. Thanks you :)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s