Metadata vocab re-use question

I have a technical question about metadata vocab reuse, and the best way to do something I’m doing.

I’m working on an API for returning a list of scholarly articles.

I am trying to do as much as I can with already existing technical metadata devices.

In general, I am going to do this as an Atom XML response, with some ‘third party’ XML namespaces in use too for full expression of what I want to express. Using already existing vocabularies, identified by URI.

In general, this is working fine — especially using the PRISM vocabulary for some scholarly citation-specific metadata elements. Also some things that were already part of Atom, and may be a bit of DC here or there.

I am generally happy with this approach, and plan to stick to it.

But there are a few places where I am not sure what to do. In general, there’s a common pattern where I need to express a certain ‘element’ using multiple vocabularies simultaneously (and/or no vocabulary at all, free text).

For instance, let’s take the (semantically vague, yes) concept of type/genre. I have a schema.org type URI that expresses the ‘type’. I can also express the ‘type’ using the dcterms ‘type’ vocabulary. I could theoretically have a couple more format/type vocabularies I’d like to expose, but let’s stop there as an example. And on top of this, I also have a free text ‘type’ string (which may or may not be derivable from the controlled vocabs), which I’d like to make available to API consumer too.

Any individual item may have some, all, or none of these data associated with it.

Now, the dcterms ‘type’ element is capable of holding any or all of these. http://dublincore.org/documents/dcmi-terms/#terms-type

“Recommended best practice is to use a controlled vocabulary such as the DCMI Type Vocabulary [DCMITYPE]. To describe the file format, physical medium, or dimensions of the resource, use the Format element.”

See, recommended is to use a controlled vocab such as DCMI Type Vocab, but this makes it clear you can also use the ‘type’ element for another controlled vocab, or no controlled vocab at all.

So it’s legal to simply do something like this:

<!-- schema.org: -->
<dcterms:type>http://schema.org/ScholarlyArticle</dcterms:type>
<!-- dcterms type vocab: -->
<dcterms:type>http://purl.org/dc/dcmitype/Text</dcterms:type>
<!-- free text not from a controlled vocab: -->
<dcterms:type>Scholarly Book Review</dcterms:type>

And I’ve been the consumer of API’s which do something like that: Just throw a grab bag of different things into repeated dcterms:type elements, including URIs representing values from different vocabs, and free text. They figure, hey, it’s legal to use dcterms:type that way according to the docs for the dcterms vocab.

And as a consumer of services that do that… I do not want to do it. It is too difficult to work with as a consumer, when you don’t know what the contents of a dcterms:type element might be, from any vocab, or none at all. It kind of ruins the utility of the controlled vocabs in the first place, or requires unreasonably complex logic on the client side.

So. Another idea that occurs is just to add some custom attributes to the dcterms:type element.

<dcterms:type vocab="schema.org">http://schema.org/ScholarlyArticle</dcterms:type> 
<dcterms:type vocab="dcterms">http://purl.org/dc/dcmitype/Text</dcterms:type>
<dcterms:type>Scholarly Book Review</dcterms:type>

Now at least the client can a lot more easily write logic for “Is there a dcterms value? If so what is it.

But I can’t really tell if this is legal or not — attributes are handled kind of inconsistently by various XML validators. Maybe I’d need to namespace the attribute with a custom namespace too:

... xmlns:mine=http://example.org/vocab ...

<dcterms:type mine:vocab="schema.org">http://schema.org/ScholarlyArticle</dcterms:type>

But namespaces on attributes are handled very inconsistently and buggily by various standard XML parsing libraries I’ve used, so I don’t really want to do that, it’s going to make things too hard on the client to use namespaced attributes.

But I kind of like the elegancy of that ‘add attributes to dcterms:type’ approach. I suppose you could even use full URIs instead of random terms to identify the vocab, for the elegance of it:

<dcterms:type vocab="http://schema.org">http://schema.org/ScholarlyArticle</dcterms:type> 
<dcterms:type vocab="http://dublincore.org/documents/dcmi-terms">http://purl.org/dc/dcmitype/Text</dcterms:type>

Another option, especially if that isn’t legal, is to give up dcterms entirely and use only my own custom namespace/vocab for ‘type’ elements:

<mine:schema-type>http://schema.org/ScholarlyArticle</dcterms:type> 
<mine:dcterms-type>http://purl.org/dc/dcmitype/Text</dcterms:type>
<mine:uncontrolled-type>Scholarly Book Review</dcterms:type>

Which is kind of ‘inelegant’, but would probably work fine too. Realistically, any consumer of my response is going to be custom written for my response, it can be written to deal with mine:schema-type just as well as dcterms:type with attribute vocab=something. Standardizing here isn’t really necessary at all for primary use cases, although there are a variety of ancillary hypothetical benefits.

Or maybe there’s some other solution entirely I’m not thinking of.

So, any feedback? What solution makes sense, balancing standards, clarity, parsimony, ease of development, ease of client development, etc.?

The ‘type’ example is a good example, but this comes up in some other places too. Another example is for ‘language’, I may have either or both of an ISO code (two letter or three letter variety, such as “en” or “eng”), and an English-language free text description of the language “English”, and want to provide either one or both, unambiguously and easy to consume for the client.

This entry was posted in General. Bookmark the permalink.

8 Responses to Metadata vocab re-use question

  1. Ben says:

    You probably know about MODS?
    I recognise the need for both controlled and uncontrolled vocabularies and the MODS solution seems reasonable to me. MODS refers to all kinds of controlled vocabularies for genre, type, form et cetera (mostly LOC lists) and the schema defines attributes to be used for indicating the type of value (i.e. “code” or ~”(free) text”) and vocabulary the term or code was taken from.

    Using dcterms:type and a vocabulary attribute is most similar to the MODS approach. And the vocab attributes (I assume they are (to be) defined in a schema) can be ignored if the client isn’t interested.
    Are you sure you are using the right XML parsers if they can’t handle attributes properly? ;)

  2. jrochkind says:

    I think MODS is serious overkill. The learning curve (both for me and for clients) and technical overhead of wrapping my HTTP response in MODS is such that I’m unlikely to do that.

    Using an Atom feed with some external XML namespaced vocabularies, like PRISM, is pretty much _just as easy_ (technically) as writing my own custom response format from scratch, and the resulting XML documents are just as easy to deal with. While I get some ancillary benefits of standardization.

    Wrapping everything in MODS, on the other hand, complicates things quite a bit.Not gonna lie, ‘standards for the sake of standards’ aren’t worth that level of added complexity to me.

    On Tue, Jan 29, 2013 at 4:37 PM, Bibliographic Wilderness

  3. jrochkind says:

    I did not say XML parsers I’ve used can’t handle attributes. I said they can not handle _namespaced_ attributes very well.

    On Tue, Jan 29, 2013 at 4:37 PM, Bibliographic Wilderness

  4. Ben says:

    Okay, I wasn’t trying to suggest to use MODS, but that you could think of using similar “value type” and “vocabulary source” attributes accompanying the value (without changing the element name).
    And excuse me for misreading the sentence about the XML parsers, but I still think there must be a parser around that handles namespaces on attributes correctly.

  5. danc86 says:

    Use RDF? It was invented to solve exactly this kind of problem.

  6. jrochkind says:

    dan86: And is again a heck of a lot more work and conceptual load both to write the provider and to write a client. For unclear gain, theoretic gain to interoperability that I am dubious will never come to be.

    What I ended up doing for now is add a custom invented-by-me non-namespaced ‘vocabulary’ attribute to otherwise more generic elements. Tried to use a suitable URI as the value of ‘vocabulary’ where available.

    en http://schema.org/Article

    On Wed, Jan 30, 2013 at 5:26 PM, Bibliographic Wilderness

  7. Ben says:

    Sorry for spamming, but did you see the Atom based Open Publication Distribution System? Its goals are somewhat similar to yours. http://opds-spec.org/

  8. jrochkind says:

    I am aware of, yes, thanks!

    On Wed, Jan 30, 2013 at 7:36 PM, Bibliographic Wilderness

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s