Synthesis of feedback to Fellows’ presentation of themes emerging from Qualitative Profiles.

Collaborative Collection Development
• Collect measures that help us satisfy accountability to legislators, administrators, etc.
Development and Fundraising
• A quantitative measure would be of interest
• Attention to how funds are used.
• Human capital as a measure*
o How development staff are hired, paid, how their time is spent

Digitization/Digital publishing
• Differentiate roles of library as author/host.
• Expressed frustration with supplementary statistics
o When methods at home change but supplementary statistics does not accommodate
o Need to capture how resources are being used
o Definitions need to be refined, not actually getting at the heart of the matter.

E-Science
• Trend worth exploring in light of NSF declaration assertion

Instruction:
• Expand collection of data to include asynchronous activity.
• Consider a way to capture the effort expended on development of materials.
o Mention of Lib Guides, a hosted service.
• Develop a metric that illustrates impact on the audience.

Staffing changes and changing roles*
• If buying aggregated packages what is role of subject specialist?
o In general, changing roles in libraries
• Organizational structures how are they changing?

General
• Impact should be a guiding principal in collection of statistics.
• Must justify time spent; don’t put more time and effort into gathering data than value of data merits.
• Some topics may be more appropriate for Spec Kit surveys.
• Need different stats that measure trends
• Link measures to ARL strategic plan.
• Quantitative measures should be comparable over time and contribute to a national context.
• Must understand why people are using the measures in order to take a good measurement

Summary of comments in response to Carole Pilkinton’s question about counting e-books:

An analogy was drawn between CRL and Google Books/Hathi Trust. Libraries jointly own the CRL collection but do not count them as part of local collections. Further, there are so many pathways to Hathi that it cannot be known how users access those titles and whether it is appropriate to consider a use of a Hathi title as a use of the local library’s collection; the term “artificial count” was applied. Issues of ownership and access were raised. The future of the catalog was raised. EEBO as a purchased resource may be counted differently than a licensed resource to which access is provided.
“Are we measuring the right things?” Participants were reminded to consider how the numbers will be used, which audience is being reached when making use of the data; are you petitioning for more space or more IT support? Use as a measure of value to the customer is a more appropriate metric. Are we capturing that metric?

(by Ann Snowman for RLLF Fellows)

More than 250 people packed Washington Convention Center 145A on Monday morning to listen to presenters from ten libraries describe informal assessment techniques they used to quickly evaluate and improve services throughout their libraries. Here are summaries of the ten presentations. The PowerPoint from the session and most of the speakers’ notes are available in the LLAMA MAES group area on ALA Connect and on Slideshare. The session was co-moderated by Jeanne Brown, University of Nevada Las Vegas and Jan Lewis, East Carolina University.

Karen Neurohr and Jennifer Paustenbaugh, Oklahoma State University presented a combination survey and focus-group-lunch technique aimed at student scholars. Scholars were identified by the university. Since the focus groups were informal they were termed “listening sessions.” Together the survey and listening sessions served to uncover the most important topics for the library to address related to this population. The activity will be repeated next year.

Louise Lowe and Judith Brook, Mercer University presented on their use of product demonstrations involving students, and the impact of the student feedback on purchasing. One trial was for a coffee vendor, with taste tests being quite popular with the students (a log was kept next to the coffee for students to weigh in on their opinions). The pattern with products was that students preferred value over bells and whistles. Decisions were based on the feedback, and a To Do list was put on a poster in the library, with items checked off when completed.

Kornelia Tancheva, Cornell University, discussed their use of unobtrusive user observations. They performed both day and night observations, simply walking through and noting student activities. Since security walks through at night as well, they felt the students did not perceive they were being studied. She cautioned against over-reliance on observations. Their follow up will be to find out why students were doing what the staff observed them doing.

Sharon Naylor and Bruce Stoffel, Illinois State University, reported on their investigation of chat reference through focus groups. They identified a campus faculty member who became their advisor. He suggested that they continue doing focus groups until they heard nothing new, which is what they did. They found that their students saw web 2.0 modalities to be social, not for academic purposes. They also found that the students prefer a personal approach.

Jeff Gatten, California Institute of the Arts spoke on the use and value of poster surveys for a distinct population, one that is right-brained and less linear. The poster approach was interactive — students grouped around the poster and answered three questions together. He also felt that the library got feedback they would not have in a traditional survey, since the students could report their immediate frustrations, which might fade. In fact they did try an online survey and got no participants.

Kirsten Kingsley and Rachel Besara, Florida State University, discussed what they learned from interviewing students in their natural environments on campus. As part of a project to renovate student study spaces in the library, librarians and staff asked students about their study habits. Most of the interviews were done by the library’s undergraduate student services staff, many of whom were only slightly older than the students. Interviews were recorded using a digital voice recorder, and were later transcribed. The results challenged many of their assumptions about students, for example: what “quiet” means to students, the nuances of group study, and when students like to study and for how long. The results identified a need for more night-time services, more software for curriculum support, and more technical support.

Wanda Dole and J.B. Hill, University of Arkansas at Little Rock (UALR), presented the results of a combined method assessment of use of UALR by community users. Quantitative measures included an examination of data from the library’s integrated library system, print management system, and donor list. Qualitative measures included data collected from a survey of community user needs and expectations. They found that community users broke out into two groups: borrowers and computer users. Borrowers were more likely to be UALR graduates, use the library for school work, and encourage others to use the library. Computer users were more likely to use the library for Internet access and to live nearby. While there have been few if any financial gifts from community users, the cost of providing access is also low. Despite having no borrower’s fees and no overdue fines, the library has lost only $4,000 in non-returned materials over a two-year period. Dole and Hill concluded that offering library services to unaffiliated users has been an important contribution to the local community, even though “good will” benefits are hard to quantify.

Ameet Doshi, Georgia Tech, described the “Flip the Library” assessment project. Georgia Tech Library’s 20-person student library advisory group was tasked with looking at four areas of the library (entrance, signage, study areas, and website) and suggesting improvements. The four groups took 15-30 minutes to record each area with a flip camera, and then met as a group to debrief. “Flip the Library” allowed library staff to view the library from the student perspective to help assess completed renovations, inform new signage and way-finding efforts, and identify a graffiti problem.

Lisa Horowitz, MIT, described a five-question Zoomerang survey used to determine if the benefits of the Humanities Library’s bookmobile service – used to promote recreational reading, DVDs and music CDs before long weekends and breaks – outweighed its costs. Staff members involved with the bookmobile were surveyed to understand better the impact of staffing the bookmobile on their regular workload. She concluded that the increased visibility for the library was worth the staff time invested and that the bookmobile was valuable to those who used it, as well as to staff. The informal assessment of 51 users provided the information needed to move forward with a decision to continue the service, but with fewer outings.

Kathy Ray, American University of Sharjah, observed undergraduate students in the library during the busiest hours to determine why students chose to use particular spaces in the library. Fifteen observations were conducted over a period of five weeks. She marked on a photocopy of the floor map where people were sitting and what they were doing. The study helped pinpoint one particularly noisy area where people liked to socialize in an area that was intended for quiet study. Furnishings in the area were reconfigured and a browsing area expanded into the space, resulting in a decrease in the noise level.

Background Points:
It would be a brief presentation on how a library run, night-time peer tutoring program began (and gained outside funding) using supporting data.
–Tutoring subject areas were chosen based on high enrollment rate/high failure courses.
–Night time tutoring filled a niche that no other campus tutoring program provided on campus. The choice to offer night-time tutoring was due to an large ethnographic study conducting in 2008 of 1300 students which discovered that a predominant number of FSU students study at night).
–Student Government Association (SGA) provided funding and supported this library initiative in part because of the data the library collected about their students. As a result of tremendous growth and success of the tutoring program since its inception in October 2009, SGA has decided to once again reward the program money.
– Students received free tutoring from hiring well-trained tutors from other campus tutoring programs, that were previous limited to daytime hours around campus.

> Perhaps the following two resources from the public library world would help and/or be adaptable:
>
> A “library use value calculator” that looks at the costs of use vs value (in monetary terms) for various transactions such as reference questions asked, and CDs borrowed: http://www.chelmsfordlibrary.org/library_info/calculator.html
>
> A manual from the Southern Ontario Library Service The library’s contribution to your community revised and re-issued in 2007 about how to demonstrate the value of one’s library services in terms “key funders” can understand.
>
Best of Luck,

Kimberlee

On 2010-03-10, at 6:37 PM, Sara Tompson wrote:

> Hi Alice:
>
> Are you a member of the Special Libraries Association/SLA or are any of your colleagues?
>
> SLA members have been working the value proposition for years.
>
> Part of our recent alignment (w/parent org values, etc.) project has some good information — much of it is available to non members, it seems, I just accessed w/out logging in.
>
> Take a look here: http://www.sla.org/content/SLA/alignment/portal/index.html
>
> and here:
> http://www.sla.org/content/learn/members/competencies/index.cfm
>
> Hope this helps!
>
> Sara
> SLA-SCC Professional Development Chair
>
>
> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
> Sara R. Tompson
> Associate Dean for Public Service
> USC Libraries
> sarat@usc.edu
> http://isd.usc.edu/~sarat/
> * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
>
> —– Original Message —–
> From: Alice Daugherty
> Date: Wednesday, March 10, 2010 1:27 pm
> Subject: ROI and value of library?
> To: uls-l@ala.org
>
>>
>>
>> Hello all,
>
>>
>>
>>
>> I am requesting guidance and wisdom regarding an issue that seems
>
>> murkyto me at the moment; and that is finding “what value our unit
>
>> (which is
>
>> the library) brings to the table” and also “how to demonstrate the
>
>> impacts and benefits (the library) has on our community.”
>
>>
>>
>>
>> I am also limited because I can only discuss Reference, GovDocs and
>
>> CD ;
>
>> well, and perhaps a smattering of circ statistics.
>
>>
>>
>>
>> The above charge has been given to all units on campus.
>
>>
>>
>>
>> I am aware that we can find out who the stakeholders are and we can
>
>> do a
>
>> needs analysis of our patrons. We can also gather typical statistics
>
>> such as Reference Desk stats, VR stats, gate counts, etc. but then
>
>> what?
>
>>
>>
>>
>> I am familiar with the Illinois study and I have been to a few ROI
>
>> colloquiums, but I can’t help but wonder for those of you that are
>
>> administrators out there – how are you portraying the worth and
>
>> value of
>
>> your libraries?
>
>>
>>
>>
>> You may reply off list if you would like.
>
>>
>>
>>
>> Thank you.
>
>>
>>
>>
>> Alice
>
>>
>>
>>
>> Alice Daugherty
>
>>
>> Information Literacy Librarian
>
>>
>> Liaison to Linguistics, Communication Sciences &
>
>>
>> Disorders, and Communication Studies
>
>>
>> Louisiana State University
>
>>
>> Baton Rouge, LA 70803
>
>>
>> 225.578.7652
>
>>
>>
>

The Library Assessment Working Group at UIUC has started discussing this very issue recently. I certainly don’t think it’s old-hat and I would definitely be interested in any findings you come across or any replies to this post.
I know a few of our preliminary issues are A) requiring a suitable amount of background info, which provides transparency into how the data was collected but without being so complex as to discourage deposits, B) the relation between our IRB and what data could be published, the possibility of limiting access (i.e. to administrators and librarians only, as you allude to), and C) linking this to a larger culture of assessment, how to incentivize deposits and collaboration between distinct units. At least those are the issues that come to mind right now. If you come across something substantive, please do think of contacting me.
Best,
Eric Phetteplace
Graduate Assistant with UIUC LAWG