The Evaluation Capacity of Czech Nonprofit Social Services Organisations

• In Event: Third sector performance

Abstract

This study explores the capacity of Czech nonprofit social services organizations to have their services evaluated. While these organizations are often required to evaluate their services by their national and regional governments and the EU Social Investment Fund, they are challenged with evaluation capacity building (ECB). This article organizes the ECB literature around six dimensions, and the research examines this exploratory question of capacity through a national survey of, and semi-structured interviews with, directors of these organisations. Results demonstrate that the challenges for evaluation capacity are similar to previous ECB studies in other countries. However, in this study they are somewhat driven by directors’ social constructions of the concept and process of evaluation influenced by the frameworks, guidelines and regulations from the same external stakeholders requiring it. Recommended practice is provided to correspond with conclusions.

Keywords: Evaluation capacity building; Evaluation; Czech Republic; Nonprofit management; Social services

References

Argyris, C., & Schon, D.A. (1996). Organizational Learning II: Theory, Method, and Practice (2nd Edition). Reading, MA: Addison-Wesley Publishing Co.

Alaimo, S. P. (2008). Nonprofits and evaluation: Managing expectations from the leader’s perspective. In J. G. Carman & K. A. Fredericks (Eds.), Nonprofits and evaluation. New Directions for Evaluation, 119, 73–92.

Bamberger, M. & Mabry, L. (2020a). Real World Evaluation: Working Under Budget, Time, Data, and Political Constraints 3rd edition. Thousand Oaks, CA: Sage Publications, Inc.

Bamberger, M. & Mabry, L. (2020b). APPENDIX 19.6 EVALUATION CAPACITY DEVELOPMENT. Real World Evaluation: Working Under Budget, Time, Data, and Political Constraints 3rd edition. Thousand Oaks, CA: Sage Publications, Inc.
https://edge.sagepub.com/system/files/bamberger_realworld_3e_online_appendix_proof_final_0.pdf accessed July 10, 2022, 266-273.

Bourgeois, I. and Cousins, J.B. (2013). Understanding Dimensions of Organizational Evaluation Capacity. American Journal of Evaluation 34(3), 299-319.

Boyle, R., Lemaire, D., & Rist, R. C. (1999). Introduction: Building evaluation capacity.
In R. Boyle & D. Lemaire (Eds.), Building effective evaluation capacity: Lessons from
practice. New Brunswick, NJ: Transaction, 1-19.

Bryson, J.M. and Patton, M.Q. (2015). Analyzing and Engaging Stakeholders. In Newcomer, Kathryn E., Hatry, Harry P., and Wholey, Joseph S. eds., Handbook of Practical Program Evaluation, 4th edition. Hoboken, NJ: John Wiley & Sons, 36-61.

Compton, D.W., Glover-Kudon, R., Smith, I.E., and Avery, M.E. (2002). Ongoing Capacity Building in the American Cancer Society 1995-2001. New Directions for Evaluation, 93, 47- 61.

Cousins, J.B., Goh, S.C., Elliott, C.J., and Bourgeois, I. (2014). Framing the Capacity to Do and Use Evaluation. New Directions for Evaluation 141, 7-23.

Czech Evaluation Society. (2011). Evaluator´s Code of Ethics.
https://czecheval.cz/Dokumenty%20-%20archiv/ces_eticky_kodex_en.pdf . Accessed July 21, 2022.

Czech Evaluation Society. (2013). Formal Standards of Conducting Evaluations.
https://czecheval.cz/Dokumenty%20%20archiv/ces_formalni_standardy_evaluaci_short_en.pdf
Accessed July 21, 2022.

Czech Evaluation Society. (2022). About us. https://czecheval.cz/en/About-us . Accessed August
7, 2022.

Czech Statistical Office. (2022a). Public Database: Organizational structure by group of
institutional sectors.
https://vdb.czso.cz/vdbvo2/faces/en/index.jsf?page=vystup-objekt&pvo=ORG07&z=T&f=TABULKA&skupId=3775&katalog=33695&pvo=ORG07&str=v386&v=v7__null__null__null Accessed December 13, 2022.

Czech Statistical Office. (2022b). Satellite Account of Non-profit Institutions.
https://apl.czso.cz/pll/rocenka/rocenka.indexnu_sat Accessed April 21, 2023.

Dillman, Don A., Smyth, Jolene D., Christian, Leah Melani. (2014). Internet, Phone, Mail and
Mixed-Mode Surveys: The Tailored Design Method, 4th edition. Hoboken, NJ: John Wiley.

Duigan, P. (2003). Mainstreaming Evaluation or Building Evaluation Capability? Three Key Elements. New Directions for Evaluation 99, 7-21.

Evaluation Theory and Practice. (2022). About the Journal. https://en.evaltep.cz/ . Accessed
August 7, 2022.

Festen, M. & Philbin, M. (2007). Level Best: How Small and Grassroots Nonprofits can Tackle
Evaluation and Talk Results. San Francisco: Jossey-Bass.

Fitzpatrick, J.L., Sanders, J.R. & Worthen, B.R. (2011). Program Evaluation: Alternative
Approaches and Practical Guides, 4th edition. Boston: Pearson Education.

Green, A.T. (1999). Nonprofits and Democratic Development: Lessons from the Czech Republic. VOLUNTAS: International Journal of Voluntary and Nonprofit Organisations, 10(3), 217-235.

Grudens-Schuck, N. (2003). The Rigidity and Comfort of Habits: A Cultural and Philosophical
Analysis of the Ups and Downs of Mainstreaming Evaluation. New Directions for Evaluation 99, 23-32.

Imas, Linda G. Morra and Rist, Ray C. (2009). The Road to Results: Designing and Conducting
Effective Development Evaluations. Washington, DC: The World Bank.

King, J.A. (2007). Developing Evaluation Capacity Through Process Use. New Directions for Evaluation 116, 45-59.

Knowlton, L.W. and Phillips, C.C. (2013). The Logic Model Guidebook: Better Strategies for Great Results, Second Edition. Thousand Oaks, CA: Sage Publications.

Ministry of Labour and Social Affairs (MoLSA). (2016). Standards for Quality in Social Services. https://www.mpsv.cz/documents/20142/225517/Vyhlaska_c._505_2006.pdf/541be239-51fb-5924-bf4e-aecf544f318b Accessed April 14, 2023.

Ministry of Labour and Social Affairs (MoLSA). (2022a). Social Services Act No. 108/2006 Coll. https://www.mpsv.cz/documents/20142/372809/Annex_3_social_services_act.pdf Accessed July 21, 2022.

Ministry of Labour and Social Affairs (MoLSA). (2022b). Register of Social Services Providers.
http://iregistr.mpsv.cz/socreg/hledani_sluzby.do?SUBSESSION_ID=1659701652781_2 Accessed August 5, 2022.

Ministry of Labour and Social Affairs (MoLSA). (2022c). The Structure of Social Services –
current status in the Czech Republic. https://www.mpsv.cz/documents/625317/625935/2646.pdf/ Accessed July 22, 2022.

Ministry of Labour and Social Affairs (MoLSA). (2022d). Social Services.
https://www.mpsv.cz/web/en/social-services Accessed August 6, 2022.

Ministry of the Interior. (2012). Collection of Laws and Collection of International Treaties.
https://aplikace.mvcr.cz/sbirkazakonu/SearchResult.aspxq=89/2012&typeLaw=zakon&what=Cislo_zakona_smlouvy. Accessed July 12, 2023.

Monroe, M.C. et al. (2005). Evaluators as Educators: Articulating Program Theory and Building Evaluation Capacity. New Directions for Evaluation 108, 57-71.

Newcomer, Kathryn E., Hatry, Harry P., and Wholey, Joseph S. (2015). Planning and Designing Useful Evaluations. In Newcomer, Kathryn E., Hatry, Harry P., and Wholey, Joseph S. eds., Handbook of Practical Program Evaluation, 4th edition. Hoboken, NJ: John Wiley & Sons, 7-35.

Office of the Government of the Czech Republic, Department of Sustainable Development. (2017). Strategic Framework Czech Republic 2030. https://www.vlada.cz/assets/ppov/udrzitelny-rozvoj/projektOPZ/Strategic_Framework_CZ2030.pdf Accessed December 13, 2022.

Office of the Government of the Czech Republic. (2021). Strategy for Cooperation Between Public Administration and Non-governmental Non-profit Organisations 2021-2030.

Parsons, Beverly L., Lovato, Chris Y., Hutchinson, Kylie and Wilson, Derek. (2016). Building
Evaluation Capacity Through CLIPs: Communities of Learning, Inquiry, and Practice. New Directions for Evaluation 151, 21-36.

Patton, M.Q. (2022). Utilization-Focused Evaluation, 5 th Edition. Thousand Oaks, CA: Sage
Publications.

Pospisil, M., Navratil, J. and Pejcal, J. (2015). Civil Society in the Czech Republic. In Christian
Schreier. 25 Years After: Mapping Civil Society in the Visegrad Countries. Stuttgart: Lucius & Lucius Verlag, 143-188.

Potucek, M. (2000). The Uneasy Birth of Czech Civil Society. VOLUNTAS: International Journal of Voluntary and Nonprofit Organisations, 11(2), 107-121.

Preskill, H. & Boyle, S. (2008). A Multidisciplinary Model of Evaluation Capacity Building. American Journal of Evaluation 29 (4), pp. 443-459.

Remr, Jiří and Potluka, Oto. (2020). Czechia. In R. Stockmann et al. (eds.), The Institutionalisation of Evaluation in Europe, 351-375. https://doi.org/10.1007/978-3-030-32284-7_14 .

Runnels, Vivien, Andrew, Caroline and Rae, Jennifer. (2017). Building Evaluation Culture and
Capacity in a Community-Level Program: Lessons Learned from Evaluating Youth Futures. Canadian Journal of Program Evaluation 32(1), 122-130.

Sanders, J.R. (2003). Mainstreaming Evaluation. New Directions for Evaluation 99, 3-6.

Schein, E. (1992). Organizational Culture and Leadership. San Francisco: Jossey-Bass Publishers.

Skacel, Jiri. (2023). Ukraine war: A year on, refugees in Czech Republic strive to integrate but
miss home. Euronews. Accessed July 5, 2023 from https://www.euronews.com/2023/02/22/ukrainians-find-refuge-in-the-czech-republic

Sonnichsen, Richard C. (1999). Building Evaluation Capacity Within Organisations. In R. Boyle
& D. Lemaire (Eds.), Building effective evaluation capacity: Lessons from practice. New Brunswick, NJ: Transaction, 53-73.

Stejná odměna (2023). What do we recommend? Retrieved from
https://www.stejnaodmena.cz/co-doporucujeme/ Accessed June 29, 2023.

Stockdill, S.H., Baizerman, M. & Compton, D.W. (2002). Toward a Definition of the ECB Process: A Conversation with the ECB Literature. New Directions for Evaluation, 93, 7- 26.

Stufflebeam, D. L. (2002). Institutionalizing evaluation checklist. Retrieved from
https://wmich.edu/evaluation/checklists.

Taylor, David and Balloch, Susan. (2022). Introduction - The Politics of Evaluation: An
Overview. In Taylor, David and Balloch, Susan, eds., The Politics of Evaluation: Participation and Policy Implementation , pp. 1 – 18. Bristol University Press DOI: https://doi.org/10.46692/9781847421210.002

Torres, R.T. and Preskill, H. (2001). Evaluation and Organizational Learning: Past, Present and
Future. American Journal of Evaluation, 22 (3), 387-395.

Toulemonde, Jacques. Incentives, Constraints and Culture-building as Instruments for Development of Evaluation Demand. In R. Boyle & D. Lemaire (Eds.), Building effective evaluation capacity: Lessons from practice. New Brunswick, NJ: Transaction, 153-168.

Trevisan, M.S. (2007). Evaluability Assessment From 1986 to 2006. American Journal of Evaluation 28(3), 290-303.

Volkov, B & King, J. (2007). A Checklist for Building Organizational Evaluation Capacity. The
Evaluation Center: Western Michigan University. Accessed June 4, 2002 from https://wmich.edu/sites/default/files/attachments/u350/2018/eval-cap-bldg-volkov%26king.pdf

Wade, Jay and Kallemeyn, Leanne. (2019). Evaluation capacity building (ECB) interventions
and the development of sustainable evaluation practice: An exploratory study.
Evaluation and Program Planning 79, 1-9.

Wholey, Joseph S. (2015). Exploratory Evaluation. In Newcomer, Kathryn E., Hatry, Harry P.,
and Wholey, Joseph S. eds., Handbook of Practical Program Evaluation, 4th edition. Hoboken, NJ: John Wiley & Sons, 88-107.

Williams, D.D. & Hawkes, M.L. (2003). Issues and Practices Related to Mainstreaming
Evaluation: Where Do We Flow from Here? New Directions for Evaluation, 99, 63-83.

Authors

• Salvatore P Alaimo, Grand Valley State University

• Jakub Pejcal, Masaryk University

• Simona Smrčková, Charles University - Prague, Czech Republic