Library data part three-and-a-bit: sharing customer data

Having had a quick scamper through the worry list, what customer data could be shared openly?

Let's start with what can't be shared:

• Name
• Full address
• Unique identifier for the data record
• Nearly all combinations of data elements within the record

The first two are obvious Data Protection precautions; the last two are less obvious precautions for the same reason: they make it possible to identify the individual data subject.

Any data extraction for release as open data must specify the required data elements. Required fields need to be selected for extraction rather than having fields not required filtered out post-extraction. This prevents any accidents. Once data's openly out in the wild it's out in the wild.

"Registration location" and "Library/libraries used" (if available) are both safe in themselves as they aren't personal data and will have data sets broad enough not to be able to identify individual data subjects. They could be combined with each other and any one of the following:

• Category (e.g. type of borrower)
• Ethnicity
• Disability
• Gender
• Year of birth/age in years (if only date of birth can be extracted then this data shouldn't be used)

The data extract could be:

       Bedlam Library     Child
       Bedlam Library     Child
       Bedlam Library     Adult
       Bedlam Library     Adult

But not:

       Bedlam Library     Child     Male
       Bedlam Library     Child     Female
       Bedlam Library     Adult     Female
       Bedlam Library     Adult     Male

Any two of these could be combined:

• Category (e.g. type of borrower)
• Ethnicity
• Disability
• Gender
• Year of birth/age in years (if only date of birth can be extracted then this data shouldn't be used)

A postcode dump for the whole library authority could be made available but not combined with any other data because of its very specific nature for identification purposes.

I think that's pretty much it. And I'd still want to run it by an Information Governance expert before going ahead (and for them to check my Privacy Impact Assessment).

Library data part three: dangerous demographics

The data about the people registered with a library is at one and the same time the most potentially useful and the most potentially dangerous. So dangerous, in fact, that when it comes to making this data openly-available the default position must be: Don't. Do. It.

That position will be strongly challenged by many so I'll devote the rest of this post to explaining the dangers and the next one will have a look at the data that might be openly-shareable so long as all the necessary precautions are taken.

Demographic information is immensely useful to a library service. Operationally it's important to see that the service is meeting the needs of all its communities and not just providing a service "for people like us by people like us." It's important to be able to make sure that particular services are reaching their target audiences and that you're not doing anything to put sections of the community off using your services. And it's essential that you have this data for Equality Impact Assessment of policy decisions. So why would you not want to share the data to get a bigger picture?

Generally speaking there are three main concerns:

• Privacy. The library is one of the few safe public places left for the individual. Removing the right to privacy is an information governance issue just as much as an ethical one and both need to be taken very seriously (both are generally given too much lip service and too little analysis and action).
  
  It also compromises the quality of the service being provided: if library customers know that how they as individuals use the library will be made public a good many of them will modify their behaviour and not use the library the way they want or need to. If they don't know this the library will have committed a significant breach of trust. 

• Legality. Does the library have the legal right to share the data? If it is possible to identify individual data subjects then the answer is categorically: No, unless the data subject has explicitly said that their data may be shared.
  
  Anonymising the data so that it is no longer personal data is easier said than done. It isn't a matter of just removing all the names. We'll have a look at this later on.
  
  The agreement has to be an opt-in and the purpose of this data sharing has to be clearly stated. "We want to make your data open so that other, as yet unknown, people can manipulate it to get as yet unknown information and outcomes" would be an open invitation to the Information Commissioners' Office to come and investigate your organisation.

• Safeguarding. This is the most problematic and under-appreciated concern. Anybody knowing whether or not a person even visits a library, let alone uses it, may put that person in actual physical danger. In some controlling relationships a partner may only be allowed out to go to the shops and heaven help them if they do anything else. They may be allowed to take a child to library activities such as story times but not for themselves. An abusive partner discovering that somebody was somewhere they shouldn't be — the wrong end of town or even the wrong town — could be a trigger for violence. The test here isn't: "What is reasonable?" because this isn't about safeguarding people against reasonable action. It's about safeguarding them from action that may be anything but reasonable.
  
  In my head I can hear somebody saying: "If they let us know that they're in an abusive relationship we could put a flag in their record to say their data's not to be shared."
  
  
• This requires the data subject to actively opt out of data sharing.

• Identifying yourself as a person in an abusive relationship is a brave thing to do and not something that should be required to be done at a public service point in a library.

• The library suddenly becomes a less safe place.

• Someone's got to remember to filter out the flagged records before sharing the data.


I don't think any of that is acceptable. (And said so when it was said to me that time).

Anonymising the data requires more than stripping out all the names. The Information Commissioner's Office has a useful checklist (pdf).

In public libraries the combination of nearly any two data elements may be enough to make that data subject identifiable, or at least narrow the number of possibilities down enough to make it statistically probable they could be identified. The combination of "library where registered" and "library used" plus one other datum is usually OK but this needs to be tested with the particular data set, in case of nasty surprises. Other combinations very quickly narrow down to the individual.

A lot depends on the data itself: if the categories used are very general it might be safe to combine it, though it may be so general as to be pretty useless. I really did once work with a library service that thought it was OK to have two ethnic identifiers in the system: blank for "people like us" and "ethnic" for anyone who looked or sounded a bit foreign; I put a block on that the first chance I got; even so it wasn't until we got all the libraries onto the library management system that we finally got right of the last of the old Browne Issue tickets with a red E on them (disturbing symbologies like that make me wonder what librarians were thinking about in the eighties).

• I'd imagine my local library authority will have thousands of white adult males in their database. How many — or few — teenage Bangladeshi females would there be?

• Postcode data very quickly narrows down. There are perhaps ninety people in my postcode area. Twenty-odd adult white males. About four males in their fifties. One white male in his fifties.

• Age data gets very specific very quickly. "Adult" and "Child" is pretty safe but as soon as you start refining that down it becomes problematic. Full date of birth is so specific it 's a red flag. 

So we would need to be very careful about what data — and what combination of data — is made available. In a library consortium setting this should be governed by formal data sharing protocols that had been passed by each authority's information governance experts and given the OK by whoever is responsible for the authority's information risk so all the data of all the people who have actively agreed to their data's being shared can be made available to the appropriate staff for the appropriate purpose within the consortium. That's a very specific remit for a very specific purpose for the use of a very specific group of people, with checks and balances and sanctions for abuse.

Which is exactly not the case with the open release of data, so different rules need to apply and need to be applied proactively (the genie doesn't go back into the bottle if you find you've made a mistake). Hence the greater need for precaution.

Library data part two: what do we know about the stock?

In principle stock data is much the least problematic data set held by libraries when it comes to trying to map it and potentially share it across local authority boundaries or make the data openly-available. There are good reasons for this:

• Every English public library service has a catalogue of resources
• There has been decades' worth of data-sharing for the purposes of interlibrary loans including, but not limited to, the UnityUK database
• There are long-established standards for title-level bibliographic data
• The outsourcing of most bibliographic metadata, limits the number of original sources of data and so imposes some consistency

Added to this can be the data mapping work involved in setting up an interface with the evidence-based stock management system CollectionHQ and the increased use of library management systems in consortium settings. Both of these get library systems people thinking about the way their data maps against external frameworks,

Technically, data about virtual stock holdings can be treated the same way as physical stock holdings. Culturally, there is some variation in approach between library services.

For the purposes of this post we'll assume that all stock has been catalogued and the records held in the library management system. In reality this will be true of most, if not all, lending library stock and a high proportion of whatever reference library stock there is these days. Many local studies collections and special collections are still playing catch-up

Title-level bibliographic data

All the bibliographic records come from the same place so this is standard data and would be easy to share and compare, right? Well… up to a point, Lord Copper.

• Not all library authorities are buying in MARC records.
• Of those that do, not all of them are retrospectively updating their old records so they'll have a mix of bought-in MARC records and locally-sourced records which may or may not be good MARC records in the first place and which certainly have variations in the mapping details.
• Those that did do a retrospective update may have hit a few glitches. Like the library authority that had an LMS that had ISBN as a required field and so had to put dummy data in this field which turned out to be the valid ISBNs of extremely different titles to the ones they actually had. (This wasn't Rochdale, though it did cause us some collateral damage.)
• There may be local additions to commercial MARC records, for instance local context-specific subject headings and notes.
• Commercial MARC records may not be available for some very local or special collection materials so these will need to be locally-sourced.

Taking these into factors into consideration this would be much the most the most reliably uniform component of a national core data set for libraries if any such were ever developed. The data available would be either:

• A full MARC record + the unique identifier for this bib record in this LMS (this is required to act as a link between the title-level data and the item-level data); or
• A non-MARC record including:
  • Title
  • Author
  • Publisher
  • Publication date
  • ISBN/ISSN or other appropriate control number, if available
  • Class number
  • Unique identifier for this bib record
  (I think there's a limit to the amount of non-MARC data that should be admissible.)

For the purposes of this game RDA-compliant records can be assumed to be ordinary MARC21 records (there's a heap of potential MARC mapping issues involved in any national sharing exercise which we won't go into here). I can see the need for the use of FRBR by public libraries but I don't see it happening any time soon so it's not considered here.

Item-level holdings data

The library catalogue includes holdings data as well as bibliographic data so that, too, could be part of a national data set. The detail and format of this data can vary between LMSs and from one library authority to another:

• Some, but not all, item records may have at least some of their data held in MARC 876 — 878 tag format
• The traditional concept of a "collection" may be described in different fields according to the LMS or the local policy. Usually it would be labelled as one or other of item type, item category or collection.

Which data to include? Or rather, which would be most likely to be consistently-recorded? My guess:

• Unique identifier (usually a barcode)
• Location
• Key linking to the appropriate bibliographic record
• Item type/item category/collection label best approximating to the traditional concept of "collection"
• Cost/value
• Use, which would generally mean the number of issues
• Current status of the item

After that the variations start to kick in big time.

There are a few devils in the detail, for instance:

• There is no standard set of "collections," though there is a de facto standard set of higher-level item types:
  • Adult Fiction
  • Adult Non-Fiction
  • Children's Fiction
  • Children's Non-Fiction
  • Reference
  • Audiovisual
  • Everything else
  The item type/item category/ collection for each library authority would need to be mapped against a standard schedule of “Item types.” For instance, when I used to pull out stock data for CIPFA returns I didn't have the appropriate categories available in fields in the item records; so in Dynix I had a dictionary item set up to do the necessary in Recall and with Spydus I set up a formula field in a Crystal Report, in both cases it involved a formula including sixty-odd "If… Then… Else…" statements.
• Are those already used for CIPFA adequate or would a new suite need to be developed and agreed?
• Would this translation be done at the library output stage or the data aggregation stage?
  For CIPFA our translation was done at output, for CollectionHQ it was done at data aggregation stage according to previously-defined mapping.
• Cost could be the actual acquired cost including discount; the supplier's list price at time of purchase, without discount; or the default replacement cost for that type of item applied by the LMS.
• Use count data may be tricky:
• It could be for the lifetime of the item or just from the time that data was added to this particular LMS if the legacy data was lost during the migration from one system to another. 
• Some LMSs record both "current use" (e.g. reset at the beginning of the financial year) and total use. You need to be able to identify one from the other.
• The use of loanable e-books/e-audiobooks may not be available as this depends on the integration of the LMS with the supplier’s management system.
• Curated web pages would be treated as reference stock and not have a use count.
• Some LMSs allow the recording of reference use as in-house use.
• Item status is always interesting:
• Does this status mean the item is actually in stock?
• Is the item available?
• Has the item gone walkies/been withdrawn?
• Again, this would have to be a mapping exercise, similar to the one we did for CollectionHQ

So what have we got?

Overall, then, we could say that every public library could put their hand to a fair bit of title-level data that's reasonably consistent in both structure and content; and some item-level data that wouldn't be difficult to be structurally consistent but would need a bit of work to map the content to a consistent level.

Library data part one: variations on a theme

Over the Summer I've been doing a bit of work for the Public Libraries Taskforce and that set me thinking about the data that public library services hold. Each one holds a shedload of data about its resources, its customers and its performance, but each one holds a slightly different shedload to its neighbours. Why would that be?

Technical reasons

• There are surprisingly few standard data structures in play in public libraries
• Different management systems hold data in different ways
• Even if the data has the same structure a different suite of descriptive labels may be in use

Human reasons

• An organisation might not feel the need to record the data at all
• The quality — or not — of the data may not be a priority so elements may be missing
• Naming conventions, etc. may change over time without retroactive conversion, leading to internal inconsistency
• The data may still be on bits of paper

Having said that there are some key data that are generally common to all, though variable in detail. I'll have a look at those over the next few posts.