Dealing with legacy and externally loaded code in webpack(er)

I’ve been mostly a ruby and Rails dev for a while now, and I’ve been a ‘full-stack web dev’ since that was the only kind of web dev. I’ve always been just comfortable enough in Javascript to get by — well, until recently.

The, I don’t know what you call it, “modern JS” (?) advances and (especially) tooling have left me a bit bewildered. (And I know I’m not alone there).  But lately I’ve been pulled (maybe a bit kicking and screaming) into Webpacker with Rails, because you really need modern npm-based tooling to get some JS dependencies you want — and Webpacker will be the default JS toolchain in Rails 6 (and I think you won’t have access to sprockets for JS at all by default).

If the tooling wasn’t already confusing enough — and Webpacker makes webpack a little bit easier to use with Rails-friendly conventions over configuration, but also adds another layer of indirection on understanding your tools — I frequently have to deal with projects where not all the code is managed with webpacker.

I might have dependencies to JS provided only via Rails engine gems (no npm package). I might have legacy projects where not all the code that could be transitioned to webpack(er) control has been yet. And other reasons.  So I might have some code being included via a webpack and javascript_pack_tag, but some code being included in a separate compiled JS via sprockets and javascript_include_tag, and maybe other code doing other odd things.

  • Might need webpacker-packed code that uses dependencies loaded via external mechanisms (sprockets, or a raw <script> tag to a CDN host).
  • Might need non-webpacker-packed code (ie, sprockets-managed usually) that uses a dependency that is loaded by webpacker (because npm/yarn is the best way to get a lot of JS dependencies)
  • Might have “vendored” third-party code that is old and doesn’t play well with ES6 import/export.

So I decided to take some time and understand the webpack(er) patterns and features relevant here. Some webpack documentation calls these techniques for “shimming”, but I think they are relevant for cases beyond what I would consider “shimming”. These techniques are generally available in webpack, but my configuration examples will be Webpacker, cause lack of webpacker examples was a barrier to newbie me in figuring this out.

I am not an expert in this stuff and appreciate any corrections!

“Externals” — webpack code depending on a library loaded via other means

Let’s say we load Uppy via a manual script tag to CDN (so it’s available old-style via window.Uppy after load), but have webpacker-packed code that needs to refer to it. (Why are we loading Uppy that way? I dunno, we might have Reasons, or it might just be legacy in mid-point in being migrated to webpacker).

You want to use the webpack externals feature.

In your config/webpack/environment.js: (after “const { environment } = require(‘@rails/webpacker’)” and before “module.exports = environment”)

environment.config.externals = {
  uppy: 'Uppy'
}

And now you can import Uppy from 'uppy'; in a webpacker source just like you would if Uppy was a local yarn/npm dependency.

The typical examples do this with jQuery:

  externals: {
    jquery: 'jQuery'
  }

Note: In my experimentations, I found that I can apparently just use Uppy (when it’s loaded in window.Uppy by non-webpacker sources) in my webpacker sources without doing the externals setup and the import. I’m not sure why or if this is expected, but externals seems like better practice.

Note: Every time the pack is loaded, if window.Uppy is not available when you have that external, you’ll get a complaint in console “Uncaught ReferenceError: Uppy is not defined”, and your whole JS pack won’t be loaded due to aborting on the error — this tripped me up when I was trying to conditionally load Uppy from CDN only on pages that needed it, but other pages had the pack loaded. I guess the right way to do this would be having separate pack files, and only register the externals with the pack file that actually uses Uppy.

Note: I don’t have Uppy in my package.json/yarn.lock at ALL, so I know webpacker isn’t compiling it into the pack. If I did, but for some reason still wanted to rely on it from an ‘external’ instead of compiling it into the pack, I’d want to do more investigative work to make sure it wans’t in my pack too, resulting in a double-load in the browser since it was already being loaded via CDN.

“Expose” — make a webpack(er) loaded dependency available to external-to-webpack JS

Let’s say you have openseadragon being controlled by webpacker and included in your pack. (Because how else are you going to get the dependency? The old method of creating a rails engine gem with a vendored asset, and keeping it up to date with third-party releases, is a REAL DRAG).

But let’s say the code that uses openseadragon is not controlled by webpacker and included in your pack. It’s still being managed and delivered with sprockets. (Why? Maybe it’s just one step along a migration to webpacker, in which you want to keep everything working step by step.)

So even though OpenSeadragon is being included in your pack, you want it available at window.OpenSeadragon “old-style”, so the other code that expects it there old-style can access it. This is a task for the webpack expose-loader.

You’ll need to yarn add expose-loader — it doesn’t come with webpack/webpacker by default. (You don’t seem to need any configuration to make it available to webpack, once you’ve added it to your package).

So you’ve already yarn add openseadragon-ed. Now in your config/webpack/environment.js: (after “const { environment } = require(‘@rails/webpacker’)” and before “module.exports = environment”)

environment.loaders.append('expose', {
  test: require.resolve('openseadragon'),
  use: [{
          loader: 'expose-loader',
          options: 'OpenSeadragon'
  }]
})

Now window.OpenSeadragon will be set, and available to JS sources that came from somewhere else (like sprockets) (or just accessed as OpenSeadragon in that code).

That is, as long as openseadragon is included in your pack. The “expose” loader directive alone won’t put it in your pack, and if it’s not in your pack, it can’t be exposed at window. (and webpacker won’t complain).

So if you aren’t already including it in your pack, over in (eg) your app/javascript/packs/application.js, add one of these:

// You don't need all of these lines, any one will do:
import 'openseadragon'
import OpenSeadragon from 'openseadragon'
require('openseadragon')

Now OpenSeadragon is included in your pack file, and exposed at window.OpenSeadragon for non-packed JS (say in a sprockets-compiled file) to access.

If you are loading jQuery in your pack, and want to make it available to “external” JS at both jQuery and $ to support either as ordinary JQuery, you want:

environment.loaders.append('expose', {
  test: require.resolve('jquery'),
  use: [{
    loader: 'expose-loader',
    options: 'jQuery'
  }, {
    loader: 'expose-loader',
    options: '$'
  }]
})

“Provide” — automatic “import” for legacy code that doesn’t

Let’s say you are including jQuery with webpacker, in your pack. Great!

And you have some legacy code in sprockets you want to move over to webpacker. This legacy code, as legacy code does, just refers to $ in it, expecting it to be available in window.$ . Or maybe it refers to jQuery. Or a little bit of both.

The “right” way to handle this would be to add import jQuery from 'jquery' at the top of every file as you move it into webpacker. Or maybe import $ from 'jquery'. Or if you want both… do you do two imports? I’m not totally sure.

Or, you can use the webpack ProvidePlugin to avoid having to add import statements, and have $ and jQuery still available (and their use triggering an implicit ‘import’ so jQuery is included in your pack).

In the middle of your config/webpack/environment.js:

const webpack = require('webpack');
environment.plugins.append('Provide', new webpack.ProvidePlugin({
  $: 'jquery',
  jQuery: 'jquery'
}));

Now you can just refer to $ and jQuery in a webpacker source, and it’ll just magically be as if you had imported it.

For code you control, this may be just a convenience, there ought to be a way to get it working without ProvidePlugin, with the proper import statements in every file. But maybe it’s vendored third-party code that was written for a “pre-modern” JS world, and you don’t want to to editing it. ProvidePlugin magically makes it automatically “import” what it needs without having to go adding the right ‘import’ statements everywhere.

Other WebPack plugins of note for legacy code

The ImportsLoader works very much like the ProvidePlugin above. But while the ProvidePlugin makes it magic happen globally — any file in the pack that references an “auto-imported” constant will trigger an import — the ImportsLoader lets you scope that behavior to only specific files.

That seems better overall — avoid accidentally using automatic import in some non-legacy code where you intend to be doing things “right” — but for whatever reason ImportsLoader is discussed a lot less on the web than ProvidePlugin, and I didn’t discover it until later, and I haven’t tried it out.

The ExportsLoader seems to be a way of magically getting legacy code to do ES6 exports (so they can be imported by other webpack sources), without actually having to edit the code to have exports statements. I haven’t played with it.

More Sources

In addition to the docs linked to above for each feature, there are a couple WebPack guides on ‘shimming’ that try to cover this material. I’m not sure why I found two of them and they don’t quite match in their recommendations, not sure which is more up to date. 1) Shimming in Webpack docs. 2) “Shimming modules” in webpack github wiki

My favorite blog post covering converting a sprockets-based Rails app to be webpacker-based instead is “Goodbye Sprockets. Welcome Webpacker” by Alessandro Rodi, although there are other blog posts covering the same goal written since I discovered Rodi’s.

In Rails6, you may not have sprockets available at all for managing Javascript, unless you hack (or politely just “configure”) it back in. This reddit comment claims to have instructions for doing so in Rails6, although I haven’t tried it yet (nor have I confirmed that in RC2 you indeed need it to get sprockets to handle JS; leaving this in part for myself, when I get to it). See also this diff.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s