Developing Research Excellence and Methods

The concluding DREaM conference (Monday 9^th July at the British Library, London) was an excellent, enjoyable and thought-provoking day and I have learned a great deal from the conference keynotes, papers, panel sessions and networking and conversations with other participants. 

Professor Hazel Hall reiterated the aims of the DREaM project in her introduction – the project aimed to develop a UK-wide network of LIS researchers, to build a secure foundation for long-term research capacity, capability and quality and to embed notions of “value”, “impact” and “influence” among practitioners and researchers. 

Traditionally LIS researchers and practitioners have been good at the survey, focus group and interview data collection techniques but other methodologies and techniques have been less well-used.  DREaM has provided an accessible and comprehensive introduction to these lesser-well known methodologies. 

All sessions were extremely valuable but the ones which were most directly relevant to my practice as a health librarian were the two keynotes by Professor Carol Tenopir and Dr Ben Goldacre on “Building evidence of the value and impact of library and information services: methods, metrics and ROI” and “Research, evidence bases, decision making and policy” respectively.

The subject of Professor Carol Tenopir’s talk – articulating the benefits, value and impact of library and information services – is central to my work in the health sector and the topic as a whole is of major interest to both LIS researchers and practitioners. 

Carol made a distinction between two fundamentally different kinds of “impact” – 1) impact in terms of information as value in terms of purchase or exchange, making savings or saving time in that “time is money” 2) the “use value” of information, i.e. the valuable (or not!) consequences derived from reading and using the information.  NHS libraries have made a lot of use of the second kind of “value” (see for example the impact studies on the NHS SHA Library Leads Website at: http://www.libraryservices.nhs.uk/forlibrarystaff/impactassessment/) but it is important that they and other kinds of libraries/information services do not neglect the first kind.  

More practically, Carol’s talk highlighted 3 approaches to measuring value – 1) measurements of implied value (i.e. producing usage statistics and theorising as to the probable value of the usage “outputs” to LIS users), 2) explicit value (i.e. trying to define outcomes more specifically, including use of the critical incident technique) 3) derived values (i.e. contingent “economic” evaluation and return on investment (ROI). 

The critical incident technique highlighted in the second of these approaches is, I feel, a particularly important technique in LIS practice and research.  The critical incident technique asks information users to articulate the value of an information product or service in a specific instance (rather than provide more vague general expressions of the value and benefits derived).  This is a very practical technique which can produce potent examples of “impact” in terms of rich qualitative data which can be appealing to the organisations in which the information service is located.

As a health library practitioner, Dr Ben Goldacre’s talk was also vital and touches on many themes I encounter in my work.  Ben highlighted how the age-old problem of publication bias (the fact that health studies are more likely to be published if they produce “positive” results or results which are favourable to commercial companies) can ultimately harm patients because research findings which might affect patient care are sometimes not being made available to healthcare professionals.  More insidious practices can, and do, go on such as researchers reporting their final results by defining outcomes differently from that laid out in their protocols (i.e. statement of intention to conduct research and details of how this research will be carried out) or using a multitude of statistical tests (some of which may not be appropriate for the data) in order to manipulate a probability value at the level generally considered significant. 

Publication bias, selective reporting of studies and withholding research data can, as well as impeding health professionals’ access to important information, harm the work of the healthcare librarian. 

The concept of a clinical librarian working with health professionals has been around for 30 years.  Clinical librarians work in a variety of ways but central to the job is to facilitate access to the health evidence base for clinicians and managers.  (Sometimes this is done by the librarian producing “evidence summaries”; other clinical librarians act more as trainers/facilitators). 

There are a plethora of examples from the health literature of the benefits of clinical librarians to patient care and other clinical/professional activity in healthcare organisations.  However, in order to harness the potential of externally published research for the benefit of patients and healthcare organisations, clinical librarians are dependent on robust and transparent reporting mechanisms of this externally published research.

The continuing improvement of information architecture and the continuing articulation of the value and benefit of information/knowledge to healthcare are central to the healthcare librarians’ remit and the keynote by Dr Ben Goldacre showed how important it is that these issues are given proper consideration.

As a new professional I very much appreciate having the opportunity to attend this conference and I would like to thank the DREaM team for the travel bursary which was kindly provided to me to support my attendance.

I was delighted to receive a travel bursary for the DReAM Conference in London.  The conference offered the opportunity to catch up with people I had met previously at DReAM events, to meet other researchers and practitioners and to share my research interests.   The conference venue at the British Library was excellent and a good lunch was provided. 

Professor Carol Tenopir’s excellent presentation on measuring value and the impact of libraries enabled me to think about why we must measure value and how we can measure value.  Although focusing on academic libraries the content of the talk was very relevant to my research into information provision in public libraries.  Professor Carol Tenopir discussed the Lib-Value project which she was involved in and the methods used to measure value within the academic library context.  The presentation highlighted the challenges which libraries face in time of economic downturn, how we must remain accountable and be able to show the impact we make.  Professor Tenopir noted the importance of measuring value not only now, but also in the future.  Measuring value can enable us to decide which services we should keep in the future.  She explained that when measuring value we should be looking at our purpose for existing – measuring what the institution says it does, looking for what changes we can make and are making.  We should be looking at our competitors – think about what we are offering and how we can develop the services we provide.  An interesting comment by Professor Tenopir was ‘what if the library disappeared?’ which suggested the importance of reaffirming why we exist and what we actually do.   Professor Tenopir highlighted the need for collaboration across sectors.  As the library is constantly evolving and services we provide are changing and adapting to meet user needs and expectations there are important choices to make about services to keep and develop in the future. 

I enjoyed the adrenalin rush fuelled by the One minute madness sessions which provided the chance to discuss my research - ‘very’ briefly to a wider audience!  The feedback I received after the one-minute madness was very encouraging.  At lunch, the chance to discuss my research and get insightful ideas from more experienced researchers and practitioners was rewarding.    I got the chance to talk to John Dolan (CILIP) who felt that my research into Information and Communication Poverty was a great topic and suggested that I take a look at Barbara Buckley Owen’s (2010) PhD on “The development of UK government policy on citizens’ access to public sector information”.  Having previously met Yvonne Morris (CILIP) at other DReAM events it was good to chat to her again about how my research was progressing.  It was also great to meet Paul McCloskey from the Edinburgh library service who shares similar research interests into public library services as me. 

After lunch Dr Louise Cooke’s presentation on social network analysis illustrated how DReAM participants expanded both their knowledge of research methods and also their social networks with other DReAM participants.  The presentation highlighted how successful the DReAM workshops had been in building awareness of LIS Research methods and in developing a network of participants.

The panel session in the afternoon involving Jo Alcock from Birmingham City University , Dr Louise Cooke, Loughborough University, Professr Charles Oppenheim and John Dolan (CILIP) was excellent.  As with Professor Carol Tenopir’s earlier presentation, the importance of collaboration between library sectors and collaboration between academics and practitioners was emphasised.  During the panel sessions some concerns facing libraries at present such as cuts in spending on staff training were highlighted.  Dr Ben Goldacre’s concluding key note speech on the nature of published and unpublished information from drugs trials concluded an eventful day.

Unfortunately I missed the networking drinks reception afterwards as I had to rush to catch my plane back to Belfast.  The three DReAM workshops and the DReAM conference has been a very rewarding experience for me. I have met new contacts, promoted the research that I am doing and have expanded my awareness of the broader library and information landscape.   

I was delighted to be asked to apply for a travel to bursary to attend the final DREaM event at the British Library, having heard about the conference from Professor Hazel Hall whilst presenting at and helping to run the iDocQ information science colloquium held at Edinburgh University in June. It was impressive to see such an international turnout for the conference, with delegates attending from various European nooks and crannies, the United States, Australia as well as my slightly less impressive trip down the west coast mainline from Glasgow.

The day began with Professor Hall’s introduction to the conference and the goals of the DREaM project more widely, where the notion of building a solid foundation in both qualitative and quantitative research methods around a “cadre” of core information science researchers was introduced. The deep and broad infrastructure was impressive with project events previewed, amplified, reviewed and archived through multiple digital and physical conduits.

The numbers stood out as well with 33 official contributors and 213 onsite delegates representing 12 countries from 4 continents (or 5 depending on your geo-political stance!) and an online footprint of over 80 blog posts and over 800 tweets, which is now surely an underestimation if the regular updates on the conference ‘Tweet Wall’ were anything  to go by. However, and perhaps most importantly, a research skills audit carried out between the first and third workshops of the DREaM project indicated a growth in theoretical knowledge amongst cadre members, as well as a greater appreciation of various methodological approaches and the resources required to carry these approaches out.

The main themes of the day, examining the value and impact of LIS services, were also introduced by Professor Hall and then picked up and discussed in the opening keynote speech by Professor Carol Tenopir, who examined the issues of service value and impact within the context of the LIB-VALUE project being carried out at the University of Tennessee, of which Professor Tenopir leads the reading and scholarship and journal collections specialisms. She began by discussing how the LIB-VALUE project has tackled the thorny issue of defining value through reference to models that break the term down first as an economic concept that views value variously in terms of an exchange-use value dichotomy (Machlup), second as a three-part concept with economic (private), social (public) and environmental aspects (Kingma) and third as a distinct three-part concept with implied, explicit and derived components. The trick is to then ask the correct sort of questions that allow one to probe these various aspects of value as defined in theory.

Throughout her excellent talk Professor Tenopir examined the use of tools for estimating derived value such as Return on Investment (ROI) and Contingent Valuation (CV), whilst emphasising that their utilisation does not preclude the use of qualitative elements that can help provide a contextual human element to service quantification. Professor Tenopir made one further point in particular that resonated with me; that a reading study cannot simply focus on readers but must also examine readings as well. Not all readings are carry an equal weight in the considerations of the reader you might say and the purpose of individual articles and their perceived utility will vary considerably. A simple point you might think, but one worth re-emphasising when one is tempted to reach only for the relatively simple implied values of usage statistics and download numbers.

Professor Tenopir’s talk was followed by a one minute madness session, an always concise and entertaining (for the audience at least) way to present on a variety of topics. This session was no different, and we were treated to a variety of topics ranging from research into augmented reality in special collections, to a study examining non-users’ perceptions of a public library service and an open invitation to engage in collaborative research with fellow information scientists down in sunny Australia. It was testimony to the hard work and practice that had gone into preparing for the session that many of the participants, in line with many a good action movie, finished rather impressively with that one important second to spare.

Following a lovely lunch Dr Louise Cooke introduced the audience to social network analysis and the importance of understanding networks and their development. Dr Cooke had carried out a social network analysis of the DREaM cadre network and her research highlighted how, as the project progressed, the cadre developed from a vulnerable network to a stronger one showing increases in both network density and research awareness and interaction between cadre participants.

This was followed by a panel discussion that explored some aspects of what the future holds for the DREaM project. The four participants covered topics ranging from accreditation standards, the importance of face to face meetings to the development of the DREaM cadre network, the exploration of longitudinal research that transcends political and research funding ‘short termism’ all the way through to what constitutes the LIS discipline and finding a balance between creating academic silos and encouraging interdisciplinary research.

Dr Ben Goldacre was then called upon to present the LIS Research Practitioner award to the North West Clinical Librarian Systematic Review and Evaluation Group.  Dr Goldacre, well known author of Bad Science and all round geek aficionado, gave a passionate talk delving into issues such as academic publication bias, the situation concerning drug companies withholding vital information about their products and how one would go about teaching a group of primary school children to conduct a systematic reviews using a jar of balls.

Some of the main points that Dr Goldacre made concerned the nature of published and unpublished information from drugs trials, the fact that this potentially conflicting information was to be found scattered all over the place and that trials registers have been a bit rubbish in sorting the situation out. His idea is to create Alltrials, essentially a website that pulls in data on trials from every source that can be found, puts that information side by side so that if there are inconsistencies then you can say “hey, that’s a bit weird isn’t it?” But there is still quite a bit of work to go, so if you want to help him or have any ideas then let him know, his email is ben@badscience.net.

Having dropped in on the final DREaM event I found it to be an excellent opportunity to meet new people as well as meet up with some (comparatively) old faces and learn a lot from those presenting on the day. After saying my goodbyes it was back on the train to Glasgow, however I seem to have overshot my office and find myself now in a warm, sunny and remote corner of Ardnamurchan in the Scottish highlands. So I think I will finish up there, turn off my laptop and enjoy the views.