Needles in haystacks: explorations in knowledge management and user research libraries
I started writing my weeknotes this week, but then this popped out. Only there’s nothing note like about this— except insofar as it has marinated in my brain throughout my week, and all the weeks really, for almost a decade. Work with me, eh?
Caveat: I’m not qualified as a knowledge manager, librarian or anything like that, just have worked in what I would characterise broadly as the democratisation of knowledge since about 2012, maybe earlier if you count my time as a trainer. Expect errors. Librarians, expect that you might shout at the screen a few times. Please do let me know, if you like, about the bits that make you shout. Strong opinions, loosely held and all that :)
Setting the scene
- Late in the week, I met up with a colleague from another government department, one of those ‘central agencies’, wanting to talk about research libraries. Specifically, they wanted to talk about the practice of making what they already know discoverable, findable*, searchable, consumable and importantly, connectable (i.e. building databases, connecting datasets via common data spines etc).
- Mid-week, I met up with a friend working on making scientific research more discoverable, findable, searchable, reusable, impactful.
- Last week, I met up with some folks from the UK wanting to make better use of their research data — their expensive but valuable ‘thick’ or ‘deep’ data — to make it more discoverable, findable, searchable, consumable, reusable.
- The week before, I met up with some folks from a state government department wanting to make their research outputs more discoverable, findable, searchable, to improve use and uptake.
This stuff seems to happen almost every week.
- My first brush with this was back in 2012. I worked on my first knowledge repository — an effort to capture the combined knowledge of 200 people, most with several decades of accumulated experience in an attempt to not to lose the value held in their heads as they retired or moved on, one by one. I wanted to store their accumulated and informal knowledge, others insisted only formal information mattered…from where did our combined ability to do our work come? What needed to be held onto, what let go? I asked myself, how do I honour that grace in handing me their informal, hard won experiences? How do I make it possible to consume all that?
….you can see where this is going, can’t you?
The thing is, these problems are everywhere. Not just in research, not just at work. Once you see it, you can’t un-see it, and that’s why I feel the need to just say all the things. You’ll be in a meeting, and someone will present their work, and they’ll email it, and then in 6 months, you see someone talk about something similar, and you think, I’ve seen that, where was it? Who said it? Where did I save it? Everywhere, every day, people are creating new knowledge, and even though the most privileged among us have access to all the world’s data, we still find it hard to learn anything. We still can’t find anything. We still wrestle with managing just the things we learn every day (I know I do!), let alone the avalanche that’s out there, in the web.
Why is knowledge management so important?
I’m just going to lift these points straight from a 2020 article in Starmind which brings together several good sources:
- Fortune 500 companies lose at least $31.5 billion a year by failing to share knowledge, according to International Data Corp.5
- In the Critical Knowledge Transfer book, a survey highlighted that 53% of respondents claimed the knowledge-related costs of losing key employees could reach almost $300,000. In another estimate from the same source, 11% believed the costs associated with losing knowledge through employees leaving could be close to $1,000,000, while others said the cost was ‘priceless.’6
- Research also shows that employees spend around 26 days each year searching for information, which is valuable time that could be undoubtedly better invested.
What about the role of special libraries?
In 2014, the Australian Library and Information Association published a report on the return on investment of ‘special libraries’. They found (conservatively) that for every AUD$1 spent on special libraries, they returned AUD$5.41.
How we know (imo)
When you start a project (of any kind, honestly) to me, the basis of what you can achieve is limited to three things: what you already know, what you can find out, and your imagination. This shapes how you frame your approach, the questions you ask yourself and others, and what you can conceive of as possible.
The before times and the after times: the influence of the world-wide web
I’m old enough to have been a university student in two different eras: before the worldwide web, and after the web began. When I researched a topic prior to the web, the extent to which I could meet those first two aspects (what I know, and what I can find out) was limited to the resources available to me — the library, people I knew, and their book collections and resources. Knowledge was an entirely undemocratic thing. It was also manageable. It was possible then, to have a ‘new idea’ (only because we couldn’t know who else had the same idea!). Some of my earliest memories are of libraries. The library at my primary school where I found a floorboard that lifted up and underneath there were someone’s lost treasures from decades before. The state library with the little museum and the floors and floors of books. My university library with the abandoned house next door with the photos on the mantelpiece and the chairs that looked like someone just popped out to get tea, some hundred years earlier.
Now, knowledge is seemingly democratic, and available anywhere. The number of un-had thoughts seems small (I hope it isn’t). I really want to run away on a worthy tangent here about the injustices of the current state of knowledge management. How very undemocratic a world we live in, how the web has shown us that the fact of knowledge being available doesn’t make it democratic. There’s lot to prosecute on that. Here’s a small part, from a previous post on knowledge management as a feminist act. Sadly, it is almost the lesser of all the things, such is the state of our unequal access to data, information and knowledge. Let’s not even talk about the contents, who gets to make it, how, and what gets left out.
The expectations on me now as a student, and those in the before times are totally different. Every sentence I write has to have been based in someone else’s thinking, and I have to be able to cite it. Because I can. It is much harder to have a new thought now than it was then. It is impossible to finish a literature review, because every day, thousands, millions of new thoughts are written and made to be consumed by my tiny brain. The overwhelm is real. And guess what? It is like this for everyone — at work, at home, on social media.
Enter the knowledge manager: exit the library and the librarians (& the tragedy of the commons)
You’d think all this overwhelm would be sending us towards our librarians and our libraries. But no. The demise of the public library is well documented, a tragedy unfolding in such slow motion that we won’t notice what we’ve lost because our memories will fade alongside our fading libraries. Libraries both create and sustain democracy, but the influence of new public management has gutted funding and eroded our understanding of knowledge as a public good.
But there is a light, and it can be found in the most interesting of places — the mushrooming of access to information has placed knowledge management as a critical problem for companies wishing to find an edge, wanting to make the best decisions possible, as quickly as possible, as cheaply as possible. I look forward to the day that throwing away hard won knowledge by sending it into some kind of file ether is a thing of the past.
Consuming data and information and turning it into knowledge and wisdom
The DIKW (Data -> Information -> Knowledge -> Wisdom) Pyramid is a well known framework for how we use data and turn it into what we need in order to make decisions.
Ontotext state:
the pyramid represents the relationships between data, information, knowledge and wisdom…Each step answers different questions about the initial data and adds value to it. The more we enrich our data with meaning and context, the more knowledge and insights we get out of it so we can take better, informed and data-based decisions.
As we add value to data, by structuring it, adding context, meaning, and undertaking ‘data cleaning’, we can move through the pyramid. As a mixed methods researcher, I also notice that getting to wisdom (why, what now?) is a lot easier with qualitative data. As a knowledge manager, I can see that while this process happens within a piece of research, it also happens, on a meta level, in the process of knowledge management. If one flipped the pyramid on its head so as to add gravity into the mix, gravity would be the knowledge management — how we take lots of data and make it firstly findable and discoverable, then understandable, then consumable, then actionable, how we direct it through the (now, because we turned it on it’s head) funnel. This could also be seen as the act of knowledge transfer (here’s a long list of reading on transferring knowledge I wrote a little while back).
Why having knowledge isn’t something we do as a project (or: knowledge as expression of living culture)
One of the things I hear a lot from folks is ‘I’ve hired a contractor to put in a library, how do we get it done?’. My heart sinks at the idea of done. The first rule of knowledge management and of making a library, is that it is never done. Done is a document graveyard. Done is a point in time snapshot of who you were. Your knowledge and your organisation’s knowledge grows every day. So should your library.
Libraries are gardens. Hire a garden designer, but fund a gardener ongoing, or expect you’ll only grow weeds.
How we begin
I’ve already said I’m not a librarian, or even an academically trained knowledge manager. I won’t go into the technical aspects of building a library or the doing of knowledge management for that reason, and also because there are loads and loads of articles and books about the technicalities of it all.
As a practitioner, I can add some value, by going over the less obvious aspects of perhaps building a repository or a library. Specifically, these are my top 9 things to consider when you are thinking about stepping into managing user research data and research outputs. As noted above though, these problems are everywhere.
- Staff appropriately — if you are going to build a platform for a digital library, you are effectively spinning up a very specialised product team. You’ll need to research your users, develop information architecture, a user interface. You’ll also need to take on the governance of research data and outputs. You’ll need to design and maintain a taxonomy and do a heap of metadata management. If you’re doing the build using agile methodologies (I hope you are!), you’ll find the process of looping back around to keep on developing it will happen. The same with the stuff you put in it — you’ll need to garden it, or your library will die. Simple as that.
2. Persistent identifiers rather than unmanaged terms — if you can, and if you want to scale, you want to use a controlled vocabulary. If you can also use persistent identifiers for the terms within that controlled vocabulary, all the better. You can have a folksonomy (more on that below), but you will need the capacity to a) have alternate labels for your terms used in the metadata (or tags, depending on your approach) and b) to refer the system to a term using a link that is persistent. You can then change the definition of things over time, but only have to update the termset.
3. Pace layers approaches to taxonomy development — as noted above, there are positives to controlled vocabularies, but there are also a lot of positives for folksonomies. If you are developing a research repository where the data itself is tagged or has metadata added, you will have a loose set of terms that might be only relevant to that project, or they might be a tag for a new finding that has never occurred before. I’m a huge fan of identifying a set of terms that don’t change (the metadata about the thing you are storing for example — the date the data was collected — our concept of what a date is doesn’t often change), identifying terms that only change a little bit (things like your company services or demographic tags for example), developing a set of terms that might evolve slowly over time, such a behavioural or experiential terms, and then allowing somewhere for a small collection of random terms that the researcher thinks are relevant.
4. People — the first thing you will deliver when you start to build a library or repository, is a real live human whose job it is, is to know what research they have in the thing they’re building. They are always your most effective tool in dealing with the knowledge overwhelm, and will be your primary advocate of your research. Never underestimate the value of having a person like this in your organisation.
5. Skills — I listed a lot of jobs in point one. That’s a lot of different skills you will need to find, often in one or two, maybe three (hopefully more!) people. 5 people is a great number. You effectively need: a developer or information architect, a curator, a library assistant (for your users), a researcher (to both do user research for library itself and to help with knowledge transfer and consumability of research) and someone to do all the stakeholder engagement. Libraries are social things.
6. Platform — I’ve written here entirely assuming you are building a digital library or repository. You will need somewhere to store all the stuff, some way of governing and maintaining all the stuff, some way of reporting on all your stuff, some way of serving all the stuff to your users. It gets messy when you add in that I’m largely referring to user research libraries, which have added complexities of managing human research according to guidelines (usually set by medical research bodies in your country — check out this database by the ResearchOps Community to see yours). That’s a platform with really flexible approaches to how you provide different people with access to data and outputs. I won’t tell you which platform to buy. There are loads. I think my only suggestion is to remember to build your solution where people already are. If you buy a platform solution, make sure your users can get to it via their normal ways of working.
7. Policies and procedures — guess what’s hard about writing policies and procedures for a living thing? That’s right, it doesn’t stay still. Again, think about it like pace layers — the stuff that doesn’t change right through to the stuff that changes every day. I made a cartoon video how-to in my last job, and I was really proud of it, it was nice and easy to understand I think, but it remained current for about a week. Shoulda seen that one coming! Importantly though, having effective governance is the very thing that sets your research free. If you’re reading this and you have anything to do with user research, when was the last time you didn’t share some research you had from someone else, because you didn’t know if you were allowed to? That right there is the pressing need for research governance and for user research libraries.
8. Barriers, effort, user experience — think about your users. Research your users. Understand the things that act as a barrier to them both accessing and using your library, and also what acts as a barrier to making sense of the research you’re giving them? How much effort is it to find your library? How intuitive is the interface? Understand why, when, how, what, where and who. Then keep doing that over and over again over time.
9. Behaviours — what are you doing here? Trying to support your staff and your organisation to make the right decision at the right time with the right data, information, knowledge and wisdom. In user research, you’re also making change happen. You’re changing your research from a thing you use once to something that persists. That changes how researchers think about their work. It changes how research consumers think about the research. It changes how your organisation perceives the value of this thing called user research and human centred design. Whatever you build and maintain must support this. You have to be the change you want to see.
*here is a great article from the NN Group covering the difference between discoverability and findability
