Bibliographic Wilderness

Blacklight Community Survey Results


On August 20th I announced a Blacklight Community Survey to the blacklight and code4lib listservs, and it was also forwarded on to the hydra listserv by a member.

Between August 20th and September 2nd, I received 18 responses. After another week of no responses, I shut off the survey. It’s taken me until now to report the results, sorry!

The Survey was implemented using Google Docs. You can see the survey instrument here, access the summary of results from Google Docs here, and the complete spreadsheet of responses here.  The survey was intentionally anonymous.

My own summary with limited discussion follows below. 

Note: The summary of results incorrectly reports 24 responses rather than 18; I accidentally didn’t delete some test data before releasing the survey, and had no way to update the summary count. However, the spreadsheet is accurate; and the summaries for individual questions are accurate (you’ll see they each add up to 18 responses or fewer), except for the Blacklight version questions which have a couple test answers in the summary version. Sorry!

I am not sure if 18 responses should be considered a lot or a little, or what percentage of Blacklight implementations it represents. It should definitely not be considered a valid statistical sample; I think of it more like getting together people who happen to be at a conference to talk about their experiences with Blacklight, but I think such a view into Blacklight experiences is still useful.

I do suspect that Hydra gets more use than these results would indicate, and Hydra users of Blacklight are under-represented. I’m not sure why, but some guesses might be that Hydra implementations of blacklight are disproportionately done by vendors/contractors, or are more likely to be “release and forget about it” implementations — in either case meaning the host institutions are less likely to maintain a relationship to the Blacklight community, and find out about or care to respond to the survey.

Institutional Demographics and Applications

The majority (12 out of 18) respondents are Academic Libraries. Along with one public library, one museum, one vendor/contractor, two national libraries or consortiums, and one ‘other’.

I was unsurprised to see that the majority of use of Blacklight is for “special collection” or “institutional repository” type use. Only 1/3rd of respondents use Blacklight for a “Library catalog/discovery” application, with the rest “A Single special-purpose collection” (5 of 18), “Institutional/Digital collections repository (multiple collections)” (11, the majority of 18 respondents), or “Other” (4).

At my place of work, when we first adopted Blacklight the primary use case for existing implementations and developers were library catalog/discovery, but I had seen the development efforts mostly focusing on other use cases lately, and it makes sense to see a shift in uses to majority “repository” or “special-purpose collection” uses along with that.

A majority (1o of 18) respondents run more than 1 Blacklight application, which I did find a bit surprising, but may go along with “repository” type use, where each repo or collection gets it’s own BL app?  6 respondents run only one BL app, and 2 respondents are working on BL app(s) in development not yet in production.

Only 3 respondents (including myself) use Blacklight to host “No digital content, just metadata records”; 3 more just digital content, and the remaining 12 (the majority) some of each.

A full 8 of 18 include at least some MARC-origin metadata in their apps, 2 more than the number reporting using their app for “Library catalog/discovery”. Not quite a majority, but it seems MARC is definitely not dead in BL-land. “Dublin Core” and “Content from a Fedora Repository”, at 9 respondents each, only barely beat out MARC.

With 9 respondents reporting using “Content from a Fedora Repo”, and 11 reporting “Institutional/Digital collections repository” I expected this would mean lots of Hydra use. But in a later question we’ll examine in more detail later, only 4 respondents reported using “hydra-head (hydra framework)” in their app, which I find surprising. I don’t know if this is accurate, or respondents missed or didn’t understand the checkbox at that later question.

Versions of Blacklight in Use, and Experience with Upgrading

Two respondents is actually still deploying an app with Blacklight 3.x.

Two more are still on Blacklight 4.x — one of those runs multiple apps with some of them already on 5.x but at least one not yet upgraded; the other runs only one app on BL 4.30.

The rest of respondents on all on a Blacklight 5.x, but they are diverse 5.x releases from 5.5 to 5.14.  At the time the survey data was collected, only four of 18 respondents had crossed the BL 5.12 boundary, where lots of deprecations and refactorings were introduced. 5.12 had been released for about 5 months at that point.  That is, many months after a given BL version was released, most BL implementations (at least in this sample) still had not upgraded to it.

Just over half of respondents, 10 of 18 have never actually upgraded a Blacklight app across a major version (eg 3.x to 4.x or 4.x to 5.x); the other 8 have.

Oddly, the two respondents reporting themselves to be still running at least one BL 3.x app also said they did have experience upgrading a BL app across major versions. Makes me wonder why some of their apps are still on 3.x. None of the respondents still deploying 4.x said they had experience upgrading a BL app across a major version.

It seems that BL apps are in general not being quickly upgraded to keep up with BL releases. Live production BL deployments in the wild use a variety of BL versions, even across major versions, and some may have never been upgraded since install.

Solr Versions In Use

Only 16 of 18 respondents reported the version of Solr they are using (actually we asked for the lowest version of Solr they were using, if they had multiple Solrs used with BL).

A full 14 of these 16 are using some variety of Solr 4.x, with a large variety of 4.x Solrs in use from 4.0 to 4.10.

No respondents were still running Solr 3.x, but one poor soul is still running Solr 1.4. And only one respondent was running a Solr 5.x. It sounds like it may be possible for BL to drop support for Solr 3.x (or has that already happened), but requiring Solr 5.x would probably be premature.

I’m curious how many people have upgraded their Solr, and how often; it may be that the preponderance of Solr 4.x indicates that most installations were first deployed when Solr was in 4.x.

Rails Versions in Use

Four of 18 respondents are still using Rails 3.x, the rest have upgraded to 4.x — although not all to 4.2.

Those using Rails 3.x also tended to be the ones still reporting old BL versions in use, including BL 3.x.  I suspect this means that a lot of installations get deployed and never have any dependencies upgraded. Recall 10 of 18 respondents have never upgraded BL across a major version.  Although many of the people reporting running old Rails and old BL have upgraded BL across a major version (I don’t know if this means they used to be running even older versions of BL, of that they’ve upgraded some but not others).

If it isn’t broke don’t fix it might sometimes work, for a “deploy and done” project that never receives any new features or development. But I suspect a lot of these institutions are going to find themselves in trouble when they realize they are eventually running old unsupported versions of Rails, ruby, or BL, especially if a security vulnerability is discovered.  Even if a backport security patch is released for an old unsupported Rails or ruby version they are using (no guarantee), they may lack local expertise to actually apply those upgrades; or upgrading Rails may require upgrading BL as well to work with later Rails, which can be a very challenging task.

Local Blacklight Development Practices and Dependencies

A full 16 of 18 respondents report apps that include locally-developed custom features. 1 more respondent didn’t answer, only 1 said their app(s) did not.

I was surprised to see that only 2 respondents said they had hired a third-party vendor or contractor to install, configure, or develop a BL app. 2 more had hired a contractor; and 2 more said they were vendors/contractors for others.

I know there are people doing a healthy business in Blacklight consulting, especially Hydra; I am guessing that most of their clients are not enough involved in the BL community to see and/or want to answer this survey. (And I’m guessing many of those installations, unless the vendor/contractor has a maintenance contract, were also “deploy and ignore” installations which have not been upgraded since release).

So almost everyone is doing local implementation of features, but not by hiring a vendor/contractor, actually doing them in-house.

I tried to list every Blacklight plugin gem I could find distributed, and ask respondents which they used. The leaders were blacklight_advanced_search (53%) and blacklight_range_limit (39%).  Next were geoblacklight and hydra-head, each with 4 respondents (31%) claiming use. Again, I’m mystified how so few respondents can be using hydra-head when so many report IR/fedora uses. No other plugin got more than 3 respondents claiming use. I was surprised that only one respondent claimed sufia use.

Blacklight Satisfaction and Evaluation

Asking how satisfied you are with blacklight, on a scale of 1 (least) to 5 (most), the median score was 4, pretty respectable.

Looking at free form answers for what people like, don’t like, or want from Blacklight.

A major trend in what people like is Blacklight’s flexibility, customizability, and extensibility:

Major trends in what people don’t like or find most challenging about Blacklight is difficulty of upgrading BL:

As well as difficulty in understanding the Blacklight codebase:

What does it mean?

I’ve separated my own lengthy interpretation, analysis, and evaluation based on my own personal judgement into a subsequent blog post.