An inside scoop on harvard library reorg

Dailykos published a useful short essay by a former harvard librarian, reflecting on the Harvard reorg/layoff news. 

I see a couple interesting points here.

Harvard has a famously byzantine library system comprising over forty libraries, and administratively divided into two separate library systems (confusingly called the Harvard University Library or HUL, and the Harvard College Library, or HCL) has changed very little in terms of organizational structure since the late 19th century.

Harvard is not alone here. In fact, I’d suggest that the oldest academic libraries, and ironically especially the old ones that really excelled 80+ years ago, are most likely to have completely dysfunctional organizational structures and organizational behavior today.

Libraries today aren’t the same as libraries 80+ years ago, especially with regard to electronic content we purchase, which has different workflows to manage and different economies to purchase; and in terms of metadata maintenance as well, something which the blog author rightly points out libraries realized the benefits of cooperating/coordinating/sharing many years ago — but sharing cards (or data to print cards) through LC is a different beast than than modern metadata control needs.

I also generally agree with the blogger’s conclusion — but with less optimism:

But second, the importance of catalogers, and more broadly speaking, librarians is not necessarily diminishing into nothingness.  The environment has changed radically, and there are sure to be plenty of future “massacre-like” events that will painfully remind us of these changes.  But librarians do have a future, and I think it may even be a bright one: they just need to accept that it won’t be quite the same as the past.

I fully agree that there is still as much of a need for the tasks librarians have always done as ever — most definitely and even especially including cataloging/metadata control.

However, despite agreeing with that, I am actually not optimistic, like that blogger is.  We are running out of time to demonstrate that our profession, community, and industry is capable of meeting the metadata control needs of the 21st century.  We are not doing a good job of it. We do not seem to be capable of changing our priorities, expertise, organizational structures, and inter-organizational collaborative infrastructures, to deal with it.

The traditional goals of libraries have traditionally are still useful and needed just as much as ever, but with different ways of accomplishing them. There is still a great need for an organization specializing in information management on behalf of a user community, and without trying to make a profit off that user community.  But I am, sadly, no longer particularly optimistic that libraries as they are are actually capable of accomplishing those goals.  However, even in the best of cases, trying will result in some painful organization reorgs — nobody likes change. (It’s of course also possible for painful reorgs to end up entirely useless or even counter-productive, or simply admissions of defeat as libraries slowly die).

Hint: If you or your organization thinks if we can just put all our metadata into RDF as quickly as possible and therefore be “doing linked data”, that this is necessary and sufficient to handle modern metadata control needs — you have not only missed the boat, you are on the wrong boat.   I have lately been seeing a worrying increase of people suggesting “oh, we just need linked data to solve that problem”, with “linked data” meaning “the data we’ve already got expressed in RDF”, with a worrying ignorance/disregard for what good data actually entails in the 21st century systems environment.

Popular press on ebooks on libraries

I posted a few days ago about my worries that publisher unwillingness to allow library ebook lending (made possible by the fact that publishers have more legal right to block such activities than with print) imperils the future of public lending libraries. 

I worry that there isn’t enough patron education on this issue. Patrons need to know that it’s publishers standing in the way, not library traditionalism or incompetence (well, there might be some of that too).

I was heartened to see a popular press article highlighting publisher resistance to library ebook lending, and the barriers publishers put in place. I hope this starts getting more coverage (and wish the ALA’s advocacy and popular education wings would work on it; what’s the ALA for, anyway?)

As demand for e-books soars, libraries struggle to stock their virtual shelves. By Christian Davenport, Published: January 14 . Washington Post.

And in a very related topic, an Amazon press release (blogged and analyzed here) suggests that making a title available through the Kindle lending program increases ebook sales for that title compared to if it were not available for lending, as well as resulting in royalties from Amazon’s library program.  Perhaps the publishers don’t need to be scared of library lending?

Of course, Amazon (as well as perhaps the publishers), would like to see an Amazon-controlled platform take control with no intermediation by pesky non-profit public libraries.  And with a new per-use royalty payment model. (which the US First Sale Doctrine makes unenforceable for print, but not ebooks).  Remember, not-for-profit libraries (public, academic, and sometimes special use)  are pretty much the only institutions in the publishing chain/universe, whose only interest is the benefit of their patrons/customers, rather than squeezing as much profit as possible out of their customers.

web app security

I just discovered the Rails Security Guide, which is actually a pretty darn good intro text to web application security issues and attack vectors, whether you work in rails or not.

(In fact, there are some places which don’t contain useful Rails content I’d expect it to, it seems to mostly be a general text!  The chapter on HTML Injection oddly doesn’t mention Rails 3.x auto-escaping in ERB <%= %>, and #html_escape, etc. )

But anyhow, recommended if you want to brush up on your knowledge of categories of attacks and types of defenses for web apps.

mobile and control

The nytimes allows you to read only 20 articles a month by direct browsing (unlimited for articles you follow from links from another site, presumably implemented by checking the http referer).

It is quite easy for a user to get around this. For instance, with a simple javascript bookmarklet that deletes or modifies the cookies the site uses to track how many views you’ve had.

Being a programmer, I started thinking of ways the nytimes site could try defend against this. Right now it has no defenses at all. I thought of possible defenses that would at least require much more complicated javascript, and possibly ways that would require an actual browser plugin (say, to modify an spoof the Referer header) to defeat.

The nytimes doesn’t seem too interested in this game of cat and mouse, their defenses haven’t changed much since it was deployed. Perhaps because it’s enough that many/most users are ‘honest’, and because they realize that unless they change their usage policies to be much more locked down (perhaps requiring a login to view any articles at all, if not ending the lack of limits from external referrers), it will always be possible to defeat.

Mobile lockdown

But then I also thought of iOS.  I haven’t checked to see if the bookmarklet approach to delete cookies works on iOS. It is possible to install clickable js bookmarklets on iOS (iPhone or iPad) Mobile Safari, although it’s a pain, requiring either manual copy-paste/editing of JS code, or syncing with your desktop safari bookmarks.  It’s accessible to many fewer users than saving a bookmarklet on a desktop device.

It is not possible to spoof Referer headers on an iPad or an iPhone.

Unlike a desktop computer where you can install any software you want (including, say,  an open source browser with a feature to spoof referers, or a browser that takes plugins, and a plugin that does this), on iOS you can only install software approved by Apple.

[Unless you 'jailbreak' your device, which at least temporarily for the moment is not actually illegal (Apple would surely like it to be, does it surprise you that it quite likely would be without the special exemption from the Librarian of Congress?), but is something that typical users won't want to do, for various reasons.]

The App Store rules prohibit any alternate non-Safari/built-in-Webkit browsers.  Apple sometimes approves alternate browsers — only when they use the built in Webkit.  I am confident that part of the approval process for any app that involves a browser component is ensuring that it doesn’t let users do ‘untoward’ things like spoof referer headers or other parts of an http request.

The reasons for these restrictions are not just (or even mostly) about a consistent UI experience. They are about making sure websites that want to have DRM-like restrictions like the nytimes (or netflix, or hulu), have those restrictions be airtight on the iOS.  (These restrictions implemented by a website may or may not technically be ‘DRM’, under the DMCA etc. The actual implementation by the website probably is not, although it serves the same ends as DRM. The restrictions on the device itself or it’s built in software to try to keep you from an end-run around the website’s implementation probably would count as DRM, thus the LoC’s specific exemption for jailbreaking your phone).

The nytimes may or may not have yet implemented paywall protection that is impossible to get around on an iPhone.  But hulu already has. You can’t watch hulu on an iPhone for free, although you can on a desktop.  If the iPhone were a platform that gave users control, it would be easy to install a browser or browser plugin that hid from the hulu servers the fact that it was an iPhone.

But instead, the iPhone is a platform where users can only install software that Apple approves, and Apple’s policies and approval processes are in part designed to protect and enforce content provide restrictions.  Note that not all content owner technical restrictions simply enforce the law — DRM keeps users from doing things that would be legal, for fair use or other reasons, too. 

A future where most people have a mobile device as their main or only web browsing computer seems quite plausible.  If the iOS ‘closed-shop’ platform model becomes prevalent (as also seems quite plausible, as it’s been quite succesful — and I wouldn’t be shocked to see larger form factor non-mobile OSs adopt this model too, perhaps the Apple desktop app store is an exploratory shot) — This could be the end of the era where computer owners have the freedom to install whatever they want on their computers, and the beginning of an era where computer owners can only install what the platform vendors say they can install.  And their permission to install will be subject to their own business models and interests, and the business models and interests of their business partners.  This is not a welcome course.

(Note the paucity of open source software on the iOS app store — can anyone find me any examples?  I don’t think the app store rules actually prohibit open source, but the nature of the ecosystem discourages it or makes it less attractive to developers for several reasons. It’s pretty difficult to hack on a fork of an open source project for iOS, let alone distribute your mods to others.)

Re-introducing Umlaut, Again

An alpha release of Umlaut 3.0 is now available.

Umlaut is an “open source front-end for a link resolver”, or:

Umlaut is a just-in-time aggregator of “last mile” specific citation services, taking input as OpenURL, and providing an HTML UI as well as an api suite for embedding Umlaut services in other applications.

What the heck does this mean? Read more.

The 3.0 release of Umlaut will not add any new features, but instead modernizes Umlaut’s architecture to be based on Rails 3.1+ as an engine gem, and work on modern ruby versions. Lots of unsupported cruft was also removed from the codebase. (Umlaut actually began as a Rails 1.x application!).

Why this matters to you is that Umlaut should be easier to install and maintain than it ever was before. See Installation/Getting Started instructions. 

This is still an alpha release at present. It likely has some not yet discovered bugs, missing features, or performance issues. But it should be much easier to work with than Umlaut 2.x, if you are looking to get started with Umlaut, definitely start with the 3.x alpha.  Alpha tester feedback very welcome, please let me know of any difficulties you have with it, suggestions, questions, etc.

Umlaut 3.x source code is available in the umlaut3dev branch in the github project (eventually it will move to master).

Why a shift to ebooks imperils libraries

It isn’t because libraries can’t figure out, technically, how to loan out ebooks. It’s because publishers don’t want them to, and may be able to prevent it.

A shift to ebooks has been predicted for a while, and seems to be happening. I’ve talked to many people who wonder why their public libraries don’t offer more ebooks they can download on their e-reader of choice, assuming it’s because the public libraries don’t want to, or are not technically competent to. The first is, i think, definitely no longer true — libraries want to. The second, technical competence, may be a barrier, but it’s not the prime one.

The prime barrier is that publishers by and large don’t want libraries to. I don’t think most library patrons realize the threat to libraries here — I think it’s high-time library organizations like the ALA start educating them. Most people like public libraries and want them to continue; a public that realizes they may be threatened will be more likely to support policy to make them do so, and that’s what’s needed.

With a print book, a library can buy a book and loan it to as many borrowers as they like, without any permission from the publisher at all. In the US, the right to do this is protected by the first sale doctrine. 

That does not apply to ebooks. I do not have the right to buy an ebook and loan it out again. I need the publisher’s permission — I may also need enabling technology to make it possible, vs Digital Rights Management that actively seeks to prevent it (and the DMCA which makes it illegal to violate that DRM technology).  A publisher can charge dearly for that permission, or withhold it entirely. With a print book, a publisher might still be worried that being able to borrow for free from a library can hurt sales — but they can’t do anything about it. With an ebook, they can do something about it, the balance of powers has shifted tectonically in favor of publishers.

Here’s a recent new york times article about this battle between publishers and libraries: Publishers vs. Libraries: An E-Book Tug of War

Here’s an account by Patrick Berry of California State University Chico on the difficulties in running a kindle lending program.   Patrick investigates the feasibility of loaning out actual physical kindles loaded with titles purchased by the library — this used to be somewhat feasible, but recently Amazon made technical changes that require a purchased title to be registered to a particular kindle device when purchased, and only readable on that device. Actually, to be fair, Amazon allows up to six devices to be registered for a title. But if a library wants to purchase more than six physical kindle devices, then a given title they purchase they can only load on six of them, unless they buy multiple copies. Note that this isn’t only six at once , it’s only six specific physical devices, pretty much ever.

What if you instead of loaning out actual physical kindles, a library wanted to loan out kindle titles for patrons to load on their own kindles?  Well, that same restriction makes it impossible for a library to simply buy the book normally at a normal rate and loan it out themselves directly to patrons (as libraries do with print books).  Now, you may have heard that Amazon recently announced a program with Overdrive to support library lending. The details of this program are vague and don’t seem available on the open internet. It seems likely to me that not every title available for kindle is in the Overdrive library program, probably only titles that publishers opted in to. (Compare to ordinary print books, where a library can buy any book at all and lend it out).  Likewise, it seems likely that libraries pay more over the lifetime of use for a kindle Overdrive title than they would for a print title. I don’t know what the library pricing model is here, and would be very very curious if anyone does — does a library pay an up front ‘purchase’ fee, is it more than the usual kindle purchase fee? Does the library pay a per-checkout fee as well?  (something a library does not do with a print book, and there’d be no way for a publisher to require it).

Libraries can offer ebooks, unlike print books, only at the sufferance of publishers, and publishers may charge whatever they like for this ‘privilege’. Publishers, not liking the idea of libraries much, are not providing that permission in some cases, and are providing pricing models in other cases which make it much more expensive for a library to offer an ebook than a print book.  If reader preferences continue to shift to ebooks as we expect, we may very well see the end of libraries as book lending institutions. (That’s of course not all a library does, and they may continue in other roles). Not because patrons don’t want to borrow ebooks from a library same as they did print ebooks, not because libraries don’t want to loan ebooks or can’t figure out the technology, but because publishers simply don’t want it to happen, and our laws give them the right to prevent it. 

Overviews of commercial library discovery products

Library Journal has a very useful article with case studies of four commercial library-sector “discovery interface” layers.  Four sections written by librarians at institutions implementing each of four products:

• EBSCO EDS (Amanda Clay Powers, Mississippi State University)
• OCLC WorldCat (Zinthia C. Briceño-Rosales, Washington State University) [Haven't figured out if this is the WorldCat Local product, or simply the free-to-any-OCLC-member use of worldcat.org]
•  Ex Libris Primo (Rebecca Fernandez, Midwestern State University), and
• Serial Solutions Summon (Ken Varnum, University of Michigan).

Discovering What Works: Librarians Compare Discovery Interface Experiences

A very useful overview of what’s out there, why institutions are choosing these products, and what their experiences are.

One negative critique I have is that, like many such articles, these are kind of short on people sharing the negatives, what didn’t work as expected, what they wished worked better, what they could have done differently.  Surely any product and any implementation process has some negatives too. I think people are still too reluctant to admit that not everything went perfectly (nothing ever does), that there were unexpected c consequences of choices made by the library (there always are) or to criticize their vendors in public (library customer/vendor relationships are kind of sado-masochistic).  But making informed decisions requires knowing upsides and downsides, things that worked and things that didn’t, and we provide tremendous value to our community when we share the full picture. Still, just sharing the experiences as they are in this article is still a great step, we need more of this.

Thanks to local colleague Agnes Flannery-Denner  for alerting me to this article. 

update: (15:33) Ken Chad on the Code4Lib listserv draws our attention to  The Higher Education Library Technology (HELibTech) wiki ‘Discovery’ entry with some actual compare and contrast between products. Although it seems to me to mostly just be marketting brochure text for the commercial products. Still useful to have em all in one place, I guess.

And Peter Murray tells about the Unified Resource Discovery Comparison chart on Google Docs.

Neither of these resources look all that useful to me, honestly.

Bad news on patent protection of the obvious

…A federal agency ruled on Monday that a set of important features commonly found in smartphones are protected by an Apple patent, a decision that could force changes in how Google’s Android phones function….

…For example, the case decided Monday involves the technology that lets you tap your finger once on the touch screen to call a phone number that is written inside an e-mail or text message. It also involves the technology that allows you to schedule a calendar appointment, again with a single tap of the finger, for a date mentioned in an e-mail.

http://www.nytimes.com/2011/12/20/technology/apple-wins-partial-victory-on-patent-claim-over-android-features.html?ref=technology

“Clicking” (tapping) on a phone number is a novel idea protected by an apple patent?

…Apple applied for one of the patents at issue in the HTC case — for detecting phone numbers and other forms of data — in 1996, 11 years before the iPhone was released….

I wonder if Skype is currently paying Apple royalties for use of essentially the same idea in their browser plugin, or if the patent is somehow unique to mobile, or to “tap” instead of “click”.

The routine granting of patents for techniques that would be obvious to any software engineer confronted with a particular context of technological ‘affordances’ is deeply harmful to software innovation, competition, and freedom.

Q: best practices for *simple* contributor IP/licensing management for open source?

So, like many non-huge non-corporate-supported open source projects, many of the open source projects I contribute to go something like this (some of which I was original author, others not):

• Someone starts the project in an publicly accessible repo.
• If she works for a company, in the best case she got permission with her employer (who may or may not own copyright to code she writes) to release it as open source.
• She sticks some open source License file in the repo saying “copying Carrie Coder” and/or the the name of the employer.

Okay, so far so good, but then:

• She adds someone else as a committer, who starts committing code. And/or accepts pull requests on github etc, committing code by other authors.
• Never even thinks about licensing/intellectual property issues.

What can go wrong?

•  Well, the license file probably still says ‘copyright Carrie Coder’ or ‘copyright Acme Inc’, even though the code by other authors has copyright held by them (or their employers). So right away something seems not all on the up and up.
• One of those contributors can later be like “Wait, I didn’t mean to release that open source, and I own the copyright, you don’t have my permission to use it, take it out.”
• Or worse, one of the contributors employers can assert they own the copyright and did not give permission for it to be released open source and you don’t have permission to use it (and neither does anyone else that’s copied or forked it from you).

Heavy weight solutions

So there’s a really heavy-weight solution to this, like Apache Foundation uses in their Contributor License Agreement.  This is something people have to actually print out and sign and mail in. Some agreements like this actually transfer the copyright to some corporate entity, presumably so the project can easily re-license under a different license later. (I thought Apache did this, but apparently not).

This is kind of too much over-head for a simple non-corporate-sponsored open source project. Who’s going to receive all this mail, and where are they going to keep the contracts? There is no corporate entity to be granted a non-exclusive license to do anything. (And the hypothetical project isn’t nearly so important or popular to justify trying to get umbrella stewardship from Apache or the Software Freedom Conservancy or whatever.(If it were, the Software Freedom Conservancy is a good option, but still too much overhead for the dozens of different tiny-to-medium sized projects anyone may be involved in. )

Even so far as individuals, over the life of the project who the committers are may very well change, and not include the original author(s) anymore.

And you don’t want to make someone print out sign and wait for you to receive something before accepting their commits, that’s not internet-speed.

Best practices for a simpler solution that’s not nothing?

So doing it ‘right’ with that heavy-weight solution is just way too much trouble, so most of us just keep ignoring it.

But is there some lighter-weight better-than-nothing probably-good-enough approach?  I am curious if anyone can provide examples, ideally lawyer-vetted examples, of doing this much simpler. 

Most of my projects are MIT-style licensed, which already says “do whatever the heck you want with this code”, so I don’t really care about being able to re-license under a different license later (I don’t think I do? Or maybe even the MIT license would already allow anyone to do that). So I definitely don’t need  and can’t really can’t handle paper print-outs.

I’m imagining something where each contributor/accepted-pull-request-submitter basically just puts a digital file in the repo, once,  that says something like “All the code I’ve contributed to this repo in past or future, I have the legal ability to release under license X, and I have done so.” And then I guess in the License file, instead of saying ‘copyright Original Author’, it would be like ‘copyright by various contributors, see files in ./contributors to see who.’

Does something along those lines end up working legally, or is it worthless, no better than just continuing to ignore the problem, so you might as well just continue to ignore the problem?  Or if it is potentially workable, does anyone have examples of projects using such a system, ideally with some evidence some lawyer has said it’s worthwhile, including a lawyer-vetted digital contributor agreement (that is itself licensed such that someone else can re-use it, ha)?

Any ideas?

JQuery-UI css and images, and Rails Asset Pipeline

tl;dr version

For Rails 3.1 (3.1.3 is the latest release as I write this).

If you have JQuery-UI CSS generated by the theme roller, say for the ‘cupertino’ theme. One option is exploding the contents a bit, put the `jquery-ui-1.8.16.custom.css` file directly at ./app/assets/stylesheets/jquery-ui-1.8.16.custom.css, and put the corresponding jquery-ui theme ./images subdir directly at `./app/assets/stylesheets/images.`  (Or ./vendor/assets or ./lib/assets). Now just include the .css file in your application.css, “ *= require jquery-ui-1.8.16.custom.css “.

If instead you really do want to leave the JQuery-UI theme source in it’s own subdir by itself, say at `./app/assets/stylesheets/jquery-ui/cupertino` (or ./vendor/assets or ./lib/assets), then you’ve got to add this to your config/application.rb, matching the file path you’ve chosen to put it at:

initializer :after_append_asset_paths,
            :group => :all,
            :after => :append_assets_path do
   config.assets.paths.unshift Rails.root.join("app", "assets", "stylesheets", "jquery-ui", "cupertino").to_s
end

Your JQuery-UI theme images will now work, both in standard development config, standard production, as well as other combinations of asset piepline configuration.  Works well as far as I can tell in Rails 3.1.3, probably will continue working in the future. (can put in ./vendor/assets or ./lib/assets  instead of ./app/assets, same thing).

For more info and a couple other options, read on.

The problem

The jquery-rails gem, which is added to a new app’s Gemfile by default in rails 3.1, comes with JS for JQuery-UI.  You can include JQuery-UI JS just by adding this to your application.js:

//= require jquery-ui

Okay, but what about the JQuery-UI theme, which is CSS and some images referenced by that CSS?  The jquery-rails README kind of unhelpfully suggests:

In order to use the themed parts of jQuery UI, you will also need to supply your own theme CSS. See jqueryui.com for more information.

Okay, but if you do this the most straightforward way, you’ll find out it doesn’t work. I download a theme, say for the theme ‘cupertino’, and I get a zipfile `jquery-ui-1.8.16.custom.zip`.  Inside there, is a ./css directory.

Okay, I take this ./css directory and actually rename it into my app, say at: `./app/assets/stylesheets/jquery-ui`.  (Can put it in ./vendor/assets too, or even ./lib/assets, all will have the same outcome).

So now inside my local `./app/assets/stylesheets/jquery-ui`, I’ve got `./cupertino`, `./cupertino/jquery-ui-1.8.16.custom.css`, and `./cupertino/images`

So I go into my ./app/assets/application.css, and add:

*= require jquery-ui/cupertino/jquery-ui-1.8.16.custom.css

Start up my app, with some JQuery-UI code activated, and everything works fine in standard ‘development’ mode configuration. But in ‘production’ mode, with pre-compiled assets, none of the JQuery-UI theme images show up. (Which in some cases degrades usably but not as aesthetic, in other cases is a usability problem like lack of a ‘close’ button on a dialog).

What happened? Well, when compiled under sprockets, the contents of jquery-ui-1.8.16.custom.css is actually combined into the application.css file, which lives at the URL (for an app mounted at host root URL): `/assets/application.css`.   But that JQuery-UI CSS contains relative URLs to image assets, of the form `url(images/something.png)`

And relative to `/assets/application.css`, that means `/assets/images/something.png`.  But sprockets actually compiled the images to file path `./public/assets/jquery-ui/cupertino/images/something.png`, where it’ll be at url `/assets/jquery-ui/cupertino/images/something.png`  Not the right place for the CSS.

If I look around on the web for solutions to this, you get a lot of conflicting and confusing answers, perhaps because they were written for different versions of the asset pipeline (inc. some pre-release versions).  I’m not even going to link you to the various reddits, stackoverflows, etc, because it’s just too confusing and depressing. The most popular one seems to involve making symbolic links in your source. Come on really? Symbolic links, that tend to go wrong if you don’t do things perfectly when zipping up your source or copying it to another place or on Windows? I figured there’s got to be a better way.

In fact, I’m going to give you not just one alternate/better way, but four (count them, one, two, three, four) alternate/better ways.  Well, that’s kind of a lie, I’ll give you two better way that actually work with the asset pipeline, and two more that don’t use the pipeline for jquery-ui theme, but you can still use in your app that otherwise uses the pipeline.  For typical cases, any of em could work, but weird cases (say, wanting to have more than one theme available at runtime) each has trade-offs.

One way that works with the pipeline, take apart the subdir structure

Just take the files out of the jquery-ui generated directory stucture, and  put `jquery-ui-1.8.16.custom.css` directly  at ./app/assets/stylesheets/jquery-ui-1.8.16.custom.css, and put the corresponding jquery-ui theme ./images subdir _directly_ at `./app/assets/stylesheets/images.`

(Or put em in ./vendor/assets or ./lib/assets, will work identically in any of those. Maybe vendor is best, I dunno).

Now in your application.css, just

 *= require jquery-ui-1.8.16.custom.css

Now the relative paths in the CSS will resolve correctly both with and without pre-compiled assets, as well as even with `config.assets.debug = false` in dev.

Great, that’s pretty simple. I have to admit I didn’t think of this until I had spent quite a bit of time figuring out the next way, which will work with whatever subdirectory structure you want. Hey, maybe you really want to keep the subdir structure to keep your jquery-ui theme stuff nicely segregated. Or maybe you just want to learn a bit more about how Rails and Sprockets work. Then read, on.

Second way that works with the pipeline, whatever subdirs you want

So maybe you really still want to keep your jquery-ui theme all nicely grouped together in a subdir,  `./(app|vendor|lib)/assets/stylesheets/jquery-ui/themename`.  Just cause it’s tidier, or becuase you need multiple themes installed in your source (although that latter gets trickier).

First I thought, okay, well, maybe we can just add `./app/vendor/assets/stylesheets/themename`  to config.assets.paths in application.rb, now we’re telling sprockets to treat that as a ‘base’ dir, and compile the stuff (images in this case) in there directly to ./public/assets without the parent dirs, keeping the relative paths working.

Turns out I was on the right track, but that doesn’t work. Because  it turns out the order of paths matters to the Sprockets device that compiles assets. And Rails insists on adding ./(app|lib|vendor)/assets to the paths before anything you add yourself in config.assets.paths .  And Sprockets will then compile your stuff at that path, and when it sees a different subsequent path that also matches those same files, Sprockets is smart enough to know it already compiled that stuff from the previous path, and not compile it again.  So your addition to config.assets.paths of a subdir of one of the paths listed earlier essentially does nothing.

Turns out the place Rails insists on adding it’s own default paths first is in an append_assets_path initializer.    (Yes, despite the name “append_”, it actually prepends em with Array#unshift. Maybe it used to append but was changed to prepend to fix some other desired behavior?). Okay, great, the Rails 3.x initializer framework let’s us hook in right after Rails does this, to put what we want first in config.assets.paths instead.

Say we have our jquery-ui theme at `./vendor/assets/stylesheets/jquery-ui/cupertino`.  Add that path to the beginning of config.assets.paths, even before the stuff Rails prepended, by putting this in your config/application.rb:

initializer :after_append_asset_paths,
            :group => :all,
            :after => :append_assets_path do
   config.assets.paths.unshift Rails.root.join("vendor", "assets", "stylesheets", "jquery-ui", "cupertino").to_s
end

This works, to get the theme css’s relative URLs to images working,  in default (as installed by Rails) development mode asset config. It works in default production mode asset config, with precompiled assets.  It even works in development mode but with `config.assets.debug = false`. Each of these setups winds up with different ways of serving assets and potentially different path relationships between assets, but this works in every combo I tried. (Some of them probably work because config.assets.paths, depending on assets config, is sometimes used to create routing to assets too).

Will this keep working in the future? I dunno, I’m not technically using any internal API or anything, but I doing something not documented or popular, and counting on internal implementation not changing that much, the :append_assets_path initiailizer remaining there and named the same and used in the same way, etc. Guess it depends on how much Rails changes, like usual.

How the heck did I figure that out

I was not previously familiar with the inner workings of Sprockets or the Rails 3.1 asset pipeline. Instead, figuring that out took me about 5 hours of iteratively doing a bunch of this stuff:

• Looking at source code for both Rails and Sprockets in like 6 open tabs of github source, trying to understand how they are interacting where.
• Using github search function (wish it worked better), or grep’ing through checked out source to try and figure out where a particular method or module or class is defined. (Rails, why do you namespace classes in Rails source with ::Sprockets , fooling me into thinking I’d find em in Sprockets source? Couldn’t you have namespaced em Rails::Sprockets instead?)
• Using ruby-debug to investigate live source, and put breakpoints at Rails or Sprockets source lines to investigate em there. One reason to investigate is to figure out what the heck class a given variable is (Rails.application.assets anyone?). Another trick I like to do is drop into debug, call #freeze on an object, continue, and then wait for the stacktrace to figure out exactly who/where tried to modify that object next. (It is a travesty that ruby 1.9.3 does not work with ruby-debug, a travesty I say)
• Looking at the Rails config and asset pipeline guides to try and understand at least the big picture design.
• Googling for hints from other people, or sometimes just hints as to what source file a particular method was actually defined in when I was having trouble finding it.
• Wash, rinse, repeat, do these steps in a different order, go down blind alleys and realize that and backtrack in my investigation, etc.

Rails 3.x is really complicated code. Composed of multiple independent decoupled modules which interact with each other in complicated ways, occasionally even monkey-patching each other. (And Sprockets, would it kill you to add some one-two liner rdoc-style comments above your methods and classes, even non-public ones, telling the reader what they do?)

On the other hand, 4-6 hours ain’t that much time for getting to the bottom of fairly complicated functionality. And the solution I arrived at is fairly tidy, Rails 3.x is designed/factored pretty nicely these days to support surgical interventions (thanks Rails for just modifying config.assets.path once in an initiailizer instead of hard-coding it into the places Rails.application.assets is init’d and routes are init’d), and the 3.x initializer chain system is pretty sweet. (But Rails can’t decide whether to use strings or symbols for initializer names, it uses both, but you need to match the form used originally for the initializer :after or :before stuff. Grr.)

Third way, why host the theme yourself at all?

If you are taking a stock JQuery-UI theme and not customizing it at all, did you know that the Google ajax/open source tools CDN hosts all the stock JQuery-UI themes? (Although it’s not documented well, they’re there).  Why host it yourself at all, just link to the Google CDN. Sure it’ll be one more browser request since you haven’t combined the jquery-ui CSS into your application.css with the pipeline, but just one more request, to the Google CDN which is presumably relatively reliable and high performance, to save you the trouble of dealing with this crap or hosting it yourself at all?

Put in your layout:

<%= stylesheet_link_tag "http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/themes/cupertino/jquery-ui.css" %>

Replace ‘cupertino’ with whatever theme you want. Some legacy versions of themes also available, replace “1.8.16″ with “1.7.2″ for instance.

Or, do an import in your application.css instead:

@import url(http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/themes/cupertino/jquery-ui.css)

It’ll still be an extra browser request, but it seems pleasant to keep all your CSS stuff, internal and external,  in your application.css, rather than splitting some of it out to the layout and then getting confused later.  CSS @import seems to be supported in all non-ancient browsers including IE6.

Fourth way, host it but skip the asset pipeline

Just cause you’re using the Rails 3.1 asset pipeline doesn’t mean you have to use it for all your assets. The Rails asset pipeline guide even says:

This is not to say that assets can (or should) no longer be placed in public; they still can be and will be served as static files by the application or web server. You would only use app/assets if you wish your files to undergo some pre-processing before they are served.

While we don’t really want any pre-processing of the source itself, if we just put the assets into public we do give up a couple things. We’d give up combining the CSS itself into assets.css to save a browser request (no big deal to save a headache in this special case, sez me. I ain’t running netflix here.) And we’d give up the cache-busting fingerprinting (again, seems tolerable to me — you aren’t going to be changing your jquery-ui theme very often, when you do, just make the changed theme a new theme in a new directory location in ./public, seems okay to me for most cases).

Now, the rails asset pipeline guide says you can do this, but it ends up being a pain, because when you have the asset pipeline turned on, you don’t have any helper methods anymore to generate URLs to assets in ./public, all the helper methods assume they’ll want an asset-pipeline-controlled asset. (Unless there’s a helper method I can’t find somewhere? Come on Rails, if you’re going to tell us it’s an option, can’t you support us doing it?)

You could I suppose just write hard-coded literal URLs — but that breaks if you are deploying an app at a sub-dir of your webserver, as Rails supports, and as I like to be able to do sometimes.

Okay, so create your own helper method, here you go, stick it in a helper somewhere:

# Much like usual rails route helpers,
# whether you realized it or not, you can pass
# option :only_path => false to get a
# full URL instead of a
# relative path.
 def public_static_path(path, options = nil)
   root_path(options) + path
 end

Now stick the jquery-UI theme at ./public/jquery-ui/cupertino, and in your layout:

<%= stylesheet_link_tag public_static_path("jquery-ui/cupertino/jquery-ui-1.8.16") %>

It’s not in the asset pipeline, but no problem.

What if you want to stick it in application.css with an “@import” instead? Make it into application.css.erb so we can use ERB to call out to a helper to generate the URL… Oops, again we run into the problem of having no way to generate the URL to the asset in public.  A helper method you add to your usual app helpers isn’t available in an erb asset. Yeah, there is a way to monkey-patch helpers into the asset context too, but I went down that path and it started getting more complicated there were a couple other tricks, so I just gave up. Oh well.

The end

The end. Any improvements, feedback, alternate methods, welcome. I wasn’t actually finding any of these suggestions on google myself (maybe i’m just a bad googler), so hopefully having em here now will make it easier for others trying to figure out what’s going on. Symbolic links, geez, you don’t have to do that.