Building program theory for evaluation
Robertson, Mary Kathryn
MetadataShow full item record
HRD professionals have an ethical and pragmatic obligation to evaluate training programs (Russ-Eft & Preskill, 2001). However, outcome focused evaluation models have dominated the organizational literature. Theory-driven evaluation has been suggested as an alternative. Theory-driven evaluation uses a program theory to make “explicit the underlying assumptions about how programs are expected to work…and then using this theory to guide the evaluation” (Rodgers, Petrosino, Huebner, & Hacsi, 2000). However, the program theory building process has not been explored. The purpose of this research was to explore the ways that multiple stakeholders’ interests are represented in the program theory building process when conducting a theory-driven evaluation. Three research questions guided this study: 1) what are the program theories of stakeholders?, 2) what similarities exist among the stakeholders’ program theories?, and 3) what aspects of the program theories are unique to each stakeholder? A case study methodology was used, focusing on a new employee orientation program at a 196 bed hospital. Findings included a description of the orientation program and a program theory for each of the nine stakeholders interviewed. The program theories were compared for similarities and differences. This study had three major conclusions: 1) similarities exist between stakeholders program theories, 2) individual stakeholders also have differing, and conflicting, components of their program theories, with many of the differences accounted for by the role of the stakeholder within the organization, and 3) in developing a program theory for evaluation, a broad range of outcomes and processes were specified by program stakeholders. The major assumption in the theory-driven evaluation literature is that there is a single program theory underlying a program. This research confirmed that there is some agreement among stakeholders, but also showed that stakeholders have unique understandings about how and why programs function. Finally, the research showed many of the outcomes and processes identified as important by stakeholders would not be included in an outcome focused evaluation. Recommendations for practice included the need for evaluators to negotiate a single program theory from stakeholders’ multiple theories, the role of the evaluator, and when to use theory-driven evaluation.
Showing items related by title, author, creator and subject.
An economic evaluation alongside a randomized controlled trial evaluating an individually tailored lifestyle intervention compared with usual care in people with Familial Hypercholesterolemia Broekhuizen, Karen; van Wier, Marieke F; Koppes, Lando L J; Brug, Johannes; van Mechelen, Willem; Bosmans, Judith E; van Poppel, Mireille N M (2015-07-29)Abstract Background Cost-effectiveness analyses provide insight in the use of lifestyle interventions. To evaluate the cost-effectiveness of a lifestyle intervention compared ...
University of Georgia. College of Education. School of Teacher Education. Dept. of Elementary Education. (uga, 1999)
Expert evaluators’ naturalistic decision making during evaluations of HIV/AIDS health education programs Zgambo, Anita Frances (uga, 2012-08)The goal of this research was to understand expert evaluators’ naturalistic decision making during evaluations of HIV/AIDS health education programs. The focus was to understand the following aspects: (1) the critical ...