On August 20th I announced a Blacklight Community Survey to the blacklight and code4lib listservs, and it was also forwarded on to the hydra listserv by a member.
Between August 20th and September 2nd, I received 18 responses. After another week of no responses, I shut off the survey. It’s taken me until now to report the results, sorry!
The Survey was implemented using Google Docs. You can see the survey instrument here, access the summary of results from Google Docs here, and the complete spreadsheet of responses here. The survey was intentionally anonymous.
My own summary with limited discussion follows below.
Note: The summary of results incorrectly reports 24 responses rather than 18; I accidentally didn’t delete some test data before releasing the survey, and had no way to update the summary count. However, the spreadsheet is accurate; and the summaries for individual questions are accurate (you’ll see they each add up to 18 responses or fewer), except for the Blacklight version questions which have a couple test answers in the summary version. Sorry!
I am not sure if 18 responses should be considered a lot or a little, or what percentage of Blacklight implementations it represents. It should definitely not be considered a valid statistical sample; I think of it more like getting together people who happen to be at a conference to talk about their experiences with Blacklight, but I think such a view into Blacklight experiences is still useful.
I do suspect that Hydra gets more use than these results would indicate, and Hydra users of Blacklight are under-represented. I’m not sure why, but some guesses might be that Hydra implementations of blacklight are disproportionately done by vendors/contractors, or are more likely to be “release and forget about it” implementations — in either case meaning the host institutions are less likely to maintain a relationship to the Blacklight community, and find out about or care to respond to the survey.
Institutional Demographics and Applications
The majority (12 out of 18) respondents are Academic Libraries. Along with one public library, one museum, one vendor/contractor, two national libraries or consortiums, and one ‘other’.
I was unsurprised to see that the majority of use of Blacklight is for “special collection” or “institutional repository” type use. Only 1/3rd of respondents use Blacklight for a “Library catalog/discovery” application, with the rest “A Single special-purpose collection” (5 of 18), “Institutional/Digital collections repository (multiple collections)” (11, the majority of 18 respondents), or “Other” (4).
At my place of work, when we first adopted Blacklight the primary use case for existing implementations and developers were library catalog/discovery, but I had seen the development efforts mostly focusing on other use cases lately, and it makes sense to see a shift in uses to majority “repository” or “special-purpose collection” uses along with that.
A majority (1o of 18) respondents run more than 1 Blacklight application, which I did find a bit surprising, but may go along with “repository” type use, where each repo or collection gets it’s own BL app? 6 respondents run only one BL app, and 2 respondents are working on BL app(s) in development not yet in production.
Only 3 respondents (including myself) use Blacklight to host “No digital content, just metadata records”; 3 more just digital content, and the remaining 12 (the majority) some of each.
A full 8 of 18 include at least some MARC-origin metadata in their apps, 2 more than the number reporting using their app for “Library catalog/discovery”. Not quite a majority, but it seems MARC is definitely not dead in BL-land. “Dublin Core” and “Content from a Fedora Repository”, at 9 respondents each, only barely beat out MARC.
With 9 respondents reporting using “Content from a Fedora Repo”, and 11 reporting “Institutional/Digital collections repository” I expected this would mean lots of Hydra use. But in a later question we’ll examine in more detail later, only 4 respondents reported using “hydra-head (hydra framework)” in their app, which I find surprising. I don’t know if this is accurate, or respondents missed or didn’t understand the checkbox at that later question.
Versions of Blacklight in Use, and Experience with Upgrading
Two respondents is actually still deploying an app with Blacklight 3.x.
Two more are still on Blacklight 4.x — one of those runs multiple apps with some of them already on 5.x but at least one not yet upgraded; the other runs only one app on BL 4.30.
The rest of respondents on all on a Blacklight 5.x, but they are diverse 5.x releases from 5.5 to 5.14. At the time the survey data was collected, only four of 18 respondents had crossed the BL 5.12 boundary, where lots of deprecations and refactorings were introduced. 5.12 had been released for about 5 months at that point. That is, many months after a given BL version was released, most BL implementations (at least in this sample) still had not upgraded to it.
Just over half of respondents, 10 of 18 have never actually upgraded a Blacklight app across a major version (eg 3.x to 4.x or 4.x to 5.x); the other 8 have.
Oddly, the two respondents reporting themselves to be still running at least one BL 3.x app also said they did have experience upgrading a BL app across major versions. Makes me wonder why some of their apps are still on 3.x. None of the respondents still deploying 4.x said they had experience upgrading a BL app across a major version.
It seems that BL apps are in general not being quickly upgraded to keep up with BL releases. Live production BL deployments in the wild use a variety of BL versions, even across major versions, and some may have never been upgraded since install.
Solr Versions In Use
Only 16 of 18 respondents reported the version of Solr they are using (actually we asked for the lowest version of Solr they were using, if they had multiple Solrs used with BL).
A full 14 of these 16 are using some variety of Solr 4.x, with a large variety of 4.x Solrs in use from 4.0 to 4.10.
No respondents were still running Solr 3.x, but one poor soul is still running Solr 1.4. And only one respondent was running a Solr 5.x. It sounds like it may be possible for BL to drop support for Solr 3.x (or has that already happened), but requiring Solr 5.x would probably be premature.
I’m curious how many people have upgraded their Solr, and how often; it may be that the preponderance of Solr 4.x indicates that most installations were first deployed when Solr was in 4.x.
Rails Versions in Use
Four of 18 respondents are still using Rails 3.x, the rest have upgraded to 4.x — although not all to 4.2.
Those using Rails 3.x also tended to be the ones still reporting old BL versions in use, including BL 3.x. I suspect this means that a lot of installations get deployed and never have any dependencies upgraded. Recall 10 of 18 respondents have never upgraded BL across a major version. Although many of the people reporting running old Rails and old BL have upgraded BL across a major version (I don’t know if this means they used to be running even older versions of BL, of that they’ve upgraded some but not others).
If it isn’t broke don’t fix it might sometimes work, for a “deploy and done” project that never receives any new features or development. But I suspect a lot of these institutions are going to find themselves in trouble when they realize they are eventually running old unsupported versions of Rails, ruby, or BL, especially if a security vulnerability is discovered. Even if a backport security patch is released for an old unsupported Rails or ruby version they are using (no guarantee), they may lack local expertise to actually apply those upgrades; or upgrading Rails may require upgrading BL as well to work with later Rails, which can be a very challenging task.
Local Blacklight Development Practices and Dependencies
A full 16 of 18 respondents report apps that include locally-developed custom features. 1 more respondent didn’t answer, only 1 said their app(s) did not.
I was surprised to see that only 2 respondents said they had hired a third-party vendor or contractor to install, configure, or develop a BL app. 2 more had hired a contractor; and 2 more said they were vendors/contractors for others.
I know there are people doing a healthy business in Blacklight consulting, especially Hydra; I am guessing that most of their clients are not enough involved in the BL community to see and/or want to answer this survey. (And I’m guessing many of those installations, unless the vendor/contractor has a maintenance contract, were also “deploy and ignore” installations which have not been upgraded since release).
So almost everyone is doing local implementation of features, but not by hiring a vendor/contractor, actually doing them in-house.
I tried to list every Blacklight plugin gem I could find distributed, and ask respondents which they used. The leaders were blacklight_advanced_search (53%) and blacklight_range_limit (39%). Next were geoblacklight and hydra-head, each with 4 respondents (31%) claiming use. Again, I’m mystified how so few respondents can be using hydra-head when so many report IR/fedora uses. No other plugin got more than 3 respondents claiming use. I was surprised that only one respondent claimed sufia use.
Blacklight Satisfaction and Evaluation
Asking how satisfied you are with blacklight, on a scale of 1 (least) to 5 (most), the median score was 4, pretty respectable.
Looking at free form answers for what people like, don’t like, or want from Blacklight.
A major trend in what people like is Blacklight’s flexibility, customizability, and extensibility:
- “The easily extendable and overridable features make developing on top of Blacklight a pleasure.”
- “…Easy to configure faceting and fields.”
- “…the ability to reuse other community plugins.”
- “The large number of plugins that enhance the search experience…”
- “We have MARC plus lots of completely randomly-organized bespoke cataloging systems. Blacklight committed from the start to be agnostic as to the source of records, and that was exactly what we needed. The ability to grow with Blacklight’s feature set from back when I started using it, that was great…”
- “Easily configurable, Easily customizable, Ability to tap into the search params logic, Format specific partial rendering”
Major trends in what people don’t like or find most challenging about Blacklight is difficulty of upgrading BL:
- “When we have heavily customized Blacklight applications, upgrading across major versions is a significant stumbling block.”
- “Being bound together with a specific Bootstrap causes enormous headaches with updating”
- “Upgrades and breaking of backwards compatibility. Porting changes back into overridden partials because much customization relies on overriding partials. Building custom, complicated, special purpose searches using Blacklight-provided methods [is a challenge].”
- “Upgrading is obviously a pain-point; although many of the features in newer versions of Blacklight are desirable, we haven’t prioritized upgrading our internal applications to use the latest and greatest.”
- “Varied support for plugins over versions [is a challenge].”
- “And doing blacklight upgrades, which usually means rewriting everything.”
- “Rapid pace of development. New versions are released very quickly, and staying up to date with the latest version is challenging at times. Also, sometimes it seems that major changes to Blacklight (for example, move from Bootstrap 2 to Bootstrap 3) are quasi-dictated by needs of one (or a handful) of particular institutions, rather than by consensus of a wider group of adopters/implementors. Also, certain Blacklight plugins get neglected and start to become less and less compatible with newer versions of Blacklight, or don’t use the latest methods/patterns, which makes it more of a challenge to maintain one’s app.”
- “Getting ready for the upgrade to 6.0. We’ve done a lot of local customizations and overrides to Blacklight and some plugins that are deprecated.”
As well as difficulty in understanding the Blacklight codebase:
- “Steep learning curve coming from vanilla rails MVC. Issues well expressed by B Armintor here: http://github.com/barmintor/ansr.”
- “Code churn in technical approach (often I knew how something was done but find out it has changed since the last time I looked). Can sometimes be difficult to debug the source given the layers of abstraction (probably a necessary evil however).”
- “Too much dinking around and mucking through lengthy instructions and config files is required to do simple things. BL requires someone with substantial systems skills to spend a lot of time to use — a luxury most organizations don’t have. Skinning BL is much more painful than it needs to be as is making modifications to BL behaviors. BL requires far more time to get running and has more technical/skill dependencies than other things we maintain. In all honesty, what people here seem to like best about BL is actually functionality delivered by solr.”
- “Figuring out how to alter blacklight to do our custom development.”
- “Understanding and comprehension of how it fits together and how to customise initially.”
- “Less. Simplicity instead of more indirection and magic. While the easy things have stayed easy anything more has seemed to be getting harder and more complicated. Search inside indexing patterns and plugin. Better, updated, maintained analytics plugin.”
- “A more active and transparent Blacklight development process. We would be happy to contribute more, but it’s difficult to know a longer-term vision of the community.”
Bibliographic Wilderness 
2 thoughts on “Blacklight Community Survey Results”