Here are some notes on near/medium future directions of the library systems environment/architecture, and a sketch of requirements on where we want to go. These notes may or may not end up as part of an internal white paper here, as we analyze where we want to be headed (something good to do anyway, but we got a kick in the pants when the vendor ended development on our current ILS).
The challenge here for me was to produce something that took the set of assumptions that are obvious to many of us library tech geeks, and make them both clear and convincing to those they aren’t already obvious to. While at the same time being useful even to those of us in the ‘in’ group, by making things explicit on paper, helping us be clear to and among ourselves about what we mean, and confirm we actually agree and have thought it through. While at the SAME time being concise. I’m not sure I succeeded, especially on that last one, but here you go.
Notes on directions of Library Systems
1. Architecture for the Library Systems Environment: Framework for common understanding.
A. Defining our Terms and Concepts
The library systems environment has grown from being composed of a single “Integrated Library System” at the origin of library systems in the 1980s, to being composed of an ‘ecology’ of systems in most libraries today. These different systems could be from various sources (proprietary and open source), fulfill various–both overlapping and distinct–functions, are used in various ways by different subsets of users (both library staff and our end-users), and interact with each other to varying degrees (some well, some poorly).
Let us consider the “Library Systems Environment” to be the collection of all these information systems. Systems in this environment include systems that are used by library staff to perform their duties; systems used by library patrons/users in performing the tasks the library supports through its services; systems that hold library data; systems that perform some combination of these and other library functions.
Let’s divide this Library Systems Environment into several components to analyze and discuss it better. The first division is between “Back End” and “Front End”
B. Back End and Front End
(My colleague Sue Woodson suggests this should instead be called “Business End” (or components, or part) and “Presentation End”. I think she’s right, but I haven’t changed it in these notes.)
The “Back End” is all those systems that library staff interact with to support their workflow and tasks. For instance, when library staff interact with systems to perform acquisitions, licensing, budgeting, cataloging, document delivery, etc. “Back End” systems include Horizon (staff client portion), ILLiad (staff interface portion), the ERM. Systems not managed by the library can be back-end systems too, such as SAP, to the extent it is used by library staff. Some systems outside even JHU management might be considered ‘back end’, like OCLC’s cataloging and ILL interfaces.
The “Front End” is all systems that our end-users interact with in their use of library services. This includes the OPAC, the Link Resolver (SFX/FindIt), our Metasearch product (Metalib/JHSearch). But also ILLiad (for placing an ILL request and receiving electronic documents), any web forms we have that users need to interact with to use library services, as well as library content-providing website(s) themselves. If we had a virtual reference product of some kind which users interacted with, this too would be the front end. The front-end is not limited to ‘discovery’. Later in this paper we will discuss that the front-end too is not limited to systems managed by the libraries.
Of course, some systems products have both front-end and back-end participation. The traditional ILS of course does. The Horizon Information Portal is the vendor’s name for the front-end component of Horizon. But as our systems environment has diversified, we have seen specialization here. SFX is primarily a front-end product, although staff workflow is necessary to keep SFX working properly, and SFX does provide back-end interfaces for that workflow. “Alternate OPAC” products such as AquaBrowser, Endeca, and Worldcat Local are almost exclusively “front end” products. They have little or no interfaces to support back-end workflow, and also do not hold any data as a ‘database of record’, although they may have copies of data from ‘databases of record’. The ERMS product we are looking at exists almost exclusively in the ‘back end’—although we want/need much of the information controlled by ERMS to be available to front-end systems. We will return to some of these ideas shortly.
[A diagram might be helpful here].
C. Loose Coupling
As our library systems environment has become more diverse, we are finding problems in the interaction of the various components in this environment. Our existing systems– which have both overlapping and distinct functionality and data—often do not work well together, which leads to inefficiencies in library staff work, loss of effective control (in the cataloging sense of ‘control’) of library data, cumbersome interfaces for end-users, and confusing and inefficient task-flow processes for both staff and end-users. A concept from computer science concept is useful in the analysis of this situation, that of coupling:
“In computer science, coupling or dependency is the degree to which each program module relies on each one of the other modules… Coupling can be “low” (also “loose” and “weak”) or “high” (also “tight” and “strong”). Low coupling refers to a relationship in which one module interacts with another module through a stable interface and does not need to be concerned with the other module’s internal implementation (see Information Hiding). With low coupling, a change in one module will not require a change in the implementation of another module. Low coupling is often a sign of a well-structured computer system, and when combined with high cohesion, supports the general goals of high readability and maintainability.”
http://en.wikipedia.org/wiki/Coupling_(computer_science)
( See also http://en.wikipedia.org/wiki/Loose_coupling )
Loose coupling between components is desirable because it allows individual components to be modified, enhanced, or replaced, without destabilizing the entire environment.
In our current environment, we are caught between very tight coupling on one hand and a complete lack of coupling on the other (that is, minimal interaction between components at all!). The traditional ILS did/does not allow you to replace any of its modules, they are all tightly bound together as a single unit, they are very tightly coupled. On the other hand our components from different vendors often communicate or interact with each other very minimally or not at all, requiring ‘double-keying’ and confusing and inefficient task-flows for both staff and users.
Neither of these alternatives are desirable. What we need instead is loose coupling, especially between the back-end and the front-end. The goal is being able to mix and match components (from multiple sources, including multiple vendors and open source), to pick the ‘best of breed’ for each function. Libraries are already being forced by circumstances to attempt this. Most academic libraries have link resolvers separate from their ILSs, because this was front-end functionality required to serve users, but the ILS product did not provide it.
A library’s ILS may simply not provide sufficient modern front-end tools, leading to the library choosing to implement (eg) AquaBrowser as a front-end component. But meanwhile, the library still has its traditional ILS necessary to support back-end work.
Right now, having no choice but to move to a more diverse systems environment, we are harmed by the fact that these diverse systems do not work well together. To the extent they work together at all, it is the result of customized ad-hoc or jury-rigged solutions which also make the coupling between them tighter than is desirable, and make it difficult to switch out one component for another (say, AquaBrowser for Endeca). But we would not choose to return to the days of the monolithic tightly coupled system either, for the very reasons we left it—it locks us into one product from one vendor, which may do some functions well but others poorly, while making the business cost of switching products too high.
So again, what we envision for a library systems architecture is an ecology of loosely coupled components, allowing us to use the best-of-breed for each component, able to switch out any given component when a superior alternative presents itself. These loosely coupled components together in concert must provide an integrated and unified interface both for back-end tasks (staff workflow) and front-end tasks (user task-flow in using library services).
To be sure, this description is an ideal. Although it is possible only to move towards it, not to achieve it in one step, it is important to describe the ideal so we know what we are trying to move toward, with each step oriented towards that direction. (And each step will carry immediate benefits).
D. En Elaboration of the Architectural Model
There is one topic we need to return to, to elaborate our architectural model: Front-ends outside of library management.
Increasingly, we expect that the “front ends”–the computer interfaces that users interact with to use library services—will include interfaces that are not library managed software at all. They can include both enterprise-wide software (Portal software; courseware; etc.), as well as software outside of the enterprise entirely (Search engines, social networking sites, iTunes, various other existing and future web and desktop applications). (See, for instance, an argument in this direction near the end of this essay: http://www.librarywebchic.net/wordpress/2007/09/16/the-future-of-web-services-isnt-the-library-website/)
To make that kind of environment work, this loose coupling is absolutely vital. We need to be able to provide our library services via an ever-changing array of “front-end” interfaces, both managed internally and not. (Note that our vendor supplied databases are already an important front-end which is a big part of our services. Link resolver technology provides some degree of loose coupling, but we need more.)
[Something else about data store being a potentially separate component which again should be loosely coupled to both front and back end? Maybe not necessary].
[Another diagram? Complete audit of all front-end participating systems? Don’t forget licensed databases, course management, portal, google scholar. ]
2. Principles of Front End Requirements
[Some notes, this needs to be sketched out quite a bit more. ].
Our front-end user-facing interfaces must have be both Unified/Integrated as well as Multipresent.
-
Unified/Integrated
- Even when there are several different software products involved in the user’s task, to the user it must appear to be an integrated system. The user can not be sent to one interface for one task (or part of their task), and another for another.
- Rational task-flow aligned to users goals:
i. Integrated task-based functionality. One example to illustrate the point: If the user wants to know if we have a given issue of a given serial, she should not have to look in one place to answer the question physically, and another electronically.
ii. Goal-based task-flow: For example, after the user ‘discovers’ items of interest, they should be immediately, at the point of need, presented with interfaces for accessing/acquiring these items, in an integrated task-flow that appears to the user to be an integrated environment.
-
Multipresent
- However, the tools will be in multiple places at once.
- The tasks need to be provided to the user at the point of need: In courseware, in Portal software, in external web searches, in vendor-databases, etc.
- While library tools and digital services need to be presented in multiple places, they should work the same way in all places, so a user’s skill in using the tools in one context can easily transfer to another.
- In all contexts, the tools and digital services (the “front end”) need to present the unified interface that is rationally related to the tasks to be accomplished, as above.
The twin principles of Unification/Integration and Multipresence may initially seem to be contradictory, but in fact they are not at all, they are complementary.
[ What’s not here yet is exactly what we want to support the user in _doing_ in a unified and multipresent way. A list of tasks, rather than of software. Something needs to be said about that, if only an inventory of what tasks we currently support, but ideally looking to the future as well. ]
3. Principles of Back End Requirements
It is indisputably important that back-end participating software supports rational, efficient, and integrated work-flow for library staff. This back-end work is important for the library, and important for front-end services—after all, the goal of this back-end work is supporting these front-end services, they are the product of back-end work. So let there be no confusion, back-end support is important for the front-end as well.
However, it is not the charge of this paper to discuss this back-end work, or what is necessary in back-end systems to support library staff work flows, so we’ll leave that for now.
It is, however, important here to lay out some requirements that the back end must have to support the front-end environment we envision, and the loosely coupled architecture we anticipate as necessary for that front-end environment.
It is vital that both data and services in the back-end be exposed through Application Programming Interfaces (APIs[1]), to allow well-functioning independent front-end services (eg, AquaBrowser and Endeca etc.; also Portal, courseware, etc.; also Google Scholar etc.). Front-end services need to be able to get access holdings: Including serial coverage; regardless of format; licensing terms; including current circulation status; and more; all not just in format that can be presented to a human but in format that software can make decisions based on. But needs are not limited to holdings. Front-end systems also need access to circulation and document delivery policies (again in format that software can make decisions based on), to document delivery services (allow the user to request document delivery from various loosely coupled front-ends), etc.
A complete list of front-end task-flows to be supported, with corresponding back-end data and services needed to support, should be compiled here at Hopkins.
Ideally, front-end software needing to use these services and data from the back-end would have a unified and integrated standard API environment to call them from. That would allow front-end software to work ‘off the shelf’ with back-end software from disparate sources. More realistically, for the near term, even when a back-end does offer an API, it will be a custom API, and a given front-end will need to consult several custom APIs to access a service which should be unified. This tightens the level of coupling, and will require more in-house programming, and more work to substitute a component. But better to have components more tightly coupled than we’d like, than to have components which can not integrate at all (which is where we sometimes find ourselves presently).
The Digital Library Federation ILS Discovery Interface Taskforce (https://project.library.upenn.edu/confluence/display/ilsapi/Home ) is charged to “analyze the issues involved in integrating ‘integrated library systems’ and discovery systems, and create a technical proposal for how such integration should be accomplished.” This means developing ideas for common service and data requirements the ‘back-end’ needs to provide through APIs, and possibly even common APIs which could minimize the need for customized case-by-case coupling/integration of disparate projects. We should watch this group with interest, and help/participate where possible.
[1] “An API is an Interface which is used for accessing an application or a service from a program. An API makes it possible to use programs from within programs, therefore it is the foundation for modular systems with clearly defined Interfaces between separate components.” http://dret.net/glossary/api
“You often have to rely on others to perform functions that you may not be able or permitted to do by yourself, such as opening a bank safety deposit box. Similarly, virtually all software has to request other software to do some things for it. To accomplish this, the asking program uses a set of standardized requests, called application programming interfaces (API), that have been defined for the program being called upon.” http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=43487
Bibliographic Wilderness 
Jonathan, kudos! This is some terrifically important thinking-out-loud. Thanks also for the link to the DLF TF.