“Forking” vs open source success

Karen Schneider remarks about some misconceptions about negative aspects of open source, which leads me to respond with something I’ve been meaning to write for a while, about when some of those negative asepcts come true from our own lack of care. Karen says:

Second, this article doesn’t really get its head around the concept that in open source development, development rarely stays “local” (even if it starts there).

Sadly, in  libraryland, open source development often DOES stay local too often. The library approach to ‘open source’ has often been: Fork early, fork often. That is, take the code, when you need it to do something slightly different, just hack it to do what you need, without regard for maintaining a common codebase.

This has happened with lots of library-used open source software. One prime example is the “Fac-Back-OPAC” and its numerous forked progeny.(I was wrong)

Why does this happen? Because, in the short-term, it’s easier and quicker to get something up and running this way that meets your perceived local needs. To maintain a common codebase capable of accomodating diverse local needs takes both more initial time, and, significantly, some software development expertise. Library developers are frequently short on time, pressured by their bosses to meet local needs as quickly as possible–and are also frequently self-taught and not very experienced in complex coordinated software projects.

In the long-term, this is a very inefficient use of programming resources, which leads to the negatives outlined in the report Karen responds to. Everybody’s got their own copy of the code, enhancements and bug fixes can not be easily shared, we are not collaborating.

Not all open source will be succesful open source

Karen says:

Every major open source project I know of has a development process, a project development timeline, and well-orchestrated development.

Right, and few library open source projects I know have this. (Koha and Evergreen being exceptions to be sure).  Without figuring out how to achieve this, these library projects are not going to become succesful (let alone ‘major’) open source projects. The library considering open source development would do well to realize this, and evaluate a project accordingly, and figure out how to allocate their institutional resources to changing this state of affairs.

Certainly, this is not an inherent part of open source, it is instead counter-indicative of successful open source, which does exist. But to change this state of affairs in libraryland is not trivial. The first step is acknowledging the problem—rather than either pretending the problem does not exist or incorrectly believing it is inevitable with open source software.

Next steps would, in my opinion, involve major libraries realizing that they need to have actual software development expertise in-house if they are going to participate in a sustainable non-vendor-supported open source development.

(Again, Koha and Evergreen are exceptions in having vendor support, which indeed protects them from many of the problems noted in that report). There is no such thing as a free lunch with open source, and if libraries think there is, they are making a mistake.

(One significant organizational culture problem too many libraries suffer from is the opposite of ‘not invented here’ syndrome. You can call it “I’d rather have someone else to blame” syndrome.  Certainly not unique to libraryland, as exhibited by corporations paying exhorbitant amounts to ‘consultants’ who really have no more expertise than their local staff. But managers get what they paid for–someone else to blame when it doesn’t work.)

I do love open source

But that also doesn’t mean that open source is inherently problematic. It depends on the product. It depends on if there is paid support available, and you want to pay for it. It depends on the strength of the development community–and if you want to build a strong development community, that takes resources.   These things don’t come without effort, cleverness, experience, and allocation of resources.

And again, lest anyone misunderstand, Evergreen and Koha are the very best of breed of libraryland open source.  It would be a mistake to assume the flaws of less mature products are their flaws–but it would also be a mistake to assume that their success will automatically accrue to other open source.

And Karen makes one excellent point about staff time that goes into localizing, customizing, and maintaining proprietary software too. I know at my institution, we spend a whole lot of staff time on (usually unsupported) localizations and customizations of proprietary software too:

First, I’ve watched large teams of library developers struggled to “adapt” proprietary software, or really, to develop around its inadequacies and hidden source code. For systems of any significant size and political complexity, “turnkey” is a fantasy. What would you rather “adapt”: code that is free to view, share, and download — and discuss and debug on public lists and chatrooms — or some vendor’s super-secret code you can’t entirely view and are often bound by contract not to discuss in the open?

I agree with Karen, but we need to do it right, and too many library managers and library programmers don’t realize what it takes to do it right yet.

This entry was posted in General. Bookmark the permalink.

7 Responses to “Forking” vs open source success

  1. Dan Scott says:

    Eh? Where are all these forks of “Fac-Back-OPAC” and its ilk? Casey’s original “OSS Endekwa” throwaway code formed the base of the work that I did with Michael Beccaria. The original commit represented the code Casey handed off at code4lib 2007 (http://code.google.com/p/fac-back-opac/source/list?start=2). We merged our changes into the code base. Gabriel Farrell and Mark Matienzo became more active on the project and introduced the pymarc indexing code in a feature branch, Kate Lynch committed various enhancements, and Casey contributed his more serious effort as another branch – all part of the same repository. Over time, Gabriel merged the features from the various branches into trunk and did a hell of a lot of work improving the code base. Recently, Ross Singer committed some Jangle integration code. All of this is in the same repository; the mature features are merged together in trunk. So… where are all these forks? How is this a “prime example” of not having “regard for maintaining a common codebase”?

  2. jrochkind says:

    Huh, maybe I got the wrong idea about that one then.

  3. I’m with Dan… in reading this post, I wondered where these instances of Fac-Back-OPAC were, as well as all of this forking in LibraryLand. Did the forks run away with the spoons?

    I’d say an awful lot — the majority — of library OSS hews to the correct OSS ethos… many chefs, one kitchen.

    Still, it’s good to have a long ruminative post about the value of OSS. Thank you for that!

  4. Pingback: More on doing open source right « Bibliographic Wilderness

  5. Robert Copelan says:

    “…Next steps would, in my opinion, involve major libraries realizing that they need to have actual software development expertise in-house if they are going to participate in a sustainable non-vendor-supported open source development.”

    I would contend that in any industry (I work in automotive) we need to move back to having a core group of in-house development expertise. BUT unlike the 70s and 80s where the group did it all, the development groups of the 21st century are there to quickly meed the organizations innovation requirements and to coordinate with the larger community. If, as is seen in the successful Open Source projects, these developers understand the power of “community” then there will be a tremendous advantage to the software as knowledge and improvements are SHARED. The major Enterprise systems of today exist based on “knowledge is power and money”. That will NOT be the future.

  6. jrochkind says:

    A better example than Fac-Bac-OPAC may have been VuFind. It is my understanding that most VuFind implementers customize the VuFind code to meet local needs, directly customizing the original shared codebase, essentially creating a local fork.

    Some ideas for how you architect code to avoid this can be found in a more recent post: https://bibwild.wordpress.com/2009/01/06/how-to-build-shared-open-source/

  7. Pingback: keeping local code cleanly seperated from shared: three case studies « Bibliographic Wilderness

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s