Image
Abstract: Defining Success: A Multi-Dimensional Framework for Evaluation in Public Libraries
Evaluation assesses the quality, success, or value of a library program or service to identify improvements, fulfill accountability requirements, or deepen understanding of the library’s offerings and community (Hernon & Schwartz, 2012; Markless & Streatfield, 2013). A variety of library-specific resources have emerged to address the growing demand for evaluation in public libraries, including books (e.g., Gross, et al., 2016; Markless & Streatfield, 2013), training initiatives (e.g., Research Institute for Public Libraries), and evaluation tools (e.g., Public Library Asssociation’s Benchmark and Project Outcome). These resources seek to help libraries tell their stories of the value they provide for their communities.
Many resources advance an outcomes-based evaluation or outcome measurement approach in which “success” is defined as a program or service achieving the objectives established by library leaders (Institute of Museum and Library Services, n.d.). Yet, an exclusive focus on library objectives can overlook important aspects of programs and services, such as alignment with community needs (Davidson, 2005), cost-effectiveness (Cellini & Kee, 2015), sustainability (Julnes, 2019), and issues of equity (AUTHOR, 2022).
To address these gaps, this paper will introduce an expanded view of library “success” that can help public libraries document and tell a multi-dimensional story of their value—and identify opportunities for improvement. We will begin by presenting our interdisciplinary research to develop a framework for defining and evaluating program “success” (AUTHOR, 2021, 2023). Our multi-phase, multi-method study included synthesis of conceptual literature in the discipline of program evaluation, analysis of evaluation reports, analysis of published evaluation studies, and interviews with evaluators. The research yielded 11 dimensions of success relevant to a broad range of programs and services.
Next, we will apply the framework to programs and services in public libraries. We will demonstrate how the 11 dimensions of success can be adapted and operationalized to align with specific libraries, contexts, and communities. In doing so, we will illustrate how the framework is not prescriptive but rather serves as a map of possibilities and a guide for libraries in planning research and evaluation activities and telling library stories that address local priorities.
Finally, we will share our use of the framework in professional development activities with public library staff nationwide. Through examples ranging from large-scale research methods training to small-group evaluation coaching, we will highlight how we have guided public libraries in applying the framework to investigate their individual programs and services. We will conclude with resources that library practitioners, library and information science educators, and library researchers can use to strengthen evaluation and research in public libraries and use the findings to better tell library stories.
Many resources advance an outcomes-based evaluation or outcome measurement approach in which “success” is defined as a program or service achieving the objectives established by library leaders (Institute of Museum and Library Services, n.d.). Yet, an exclusive focus on library objectives can overlook important aspects of programs and services, such as alignment with community needs (Davidson, 2005), cost-effectiveness (Cellini & Kee, 2015), sustainability (Julnes, 2019), and issues of equity (AUTHOR, 2022).
To address these gaps, this paper will introduce an expanded view of library “success” that can help public libraries document and tell a multi-dimensional story of their value—and identify opportunities for improvement. We will begin by presenting our interdisciplinary research to develop a framework for defining and evaluating program “success” (AUTHOR, 2021, 2023). Our multi-phase, multi-method study included synthesis of conceptual literature in the discipline of program evaluation, analysis of evaluation reports, analysis of published evaluation studies, and interviews with evaluators. The research yielded 11 dimensions of success relevant to a broad range of programs and services.
Next, we will apply the framework to programs and services in public libraries. We will demonstrate how the 11 dimensions of success can be adapted and operationalized to align with specific libraries, contexts, and communities. In doing so, we will illustrate how the framework is not prescriptive but rather serves as a map of possibilities and a guide for libraries in planning research and evaluation activities and telling library stories that address local priorities.
Finally, we will share our use of the framework in professional development activities with public library staff nationwide. Through examples ranging from large-scale research methods training to small-group evaluation coaching, we will highlight how we have guided public libraries in applying the framework to investigate their individual programs and services. We will conclude with resources that library practitioners, library and information science educators, and library researchers can use to strengthen evaluation and research in public libraries and use the findings to better tell library stories.