Measuring the intangible: lessons from USAID partners on how to measure the impact of organizational learning and adaptive management

Abstract

This paper explores lessons learned from a USAID-funded learning network of implementing partners, known as CLAIM, which developed innovative methods to measure the seemingly intangible contributions of collaborating, learning, and adapting (CLA) to organizational effectiveness and development outcomes. CLA is a USAID Framework and a set of practices that can be used to strengthen organizational learning and the conditions that enable it. CLA can be applied to USAID?s programming to improve organizational effectiveness and development outcomes. Learning network members developed and tested a range of methods and tools to measure the extent to which organizations and projects integrated CLA and whether that integration contributed to organizational or development outcomes. Approaches analyzed in this paper include: developing clear theories of change to determine what researchers would ?expect to see?? were CLA to contribute to outcomes; CLA self-assessment processes to measure the extent of CLA integration; and the use of pivot or change logs to document both CLA integration and its contributions. Learning network members also found the learning network model to be a useful approach for sharing and pooling learning across members, despite some structural challenges identified in this paper during the life of the network. The approach and early findings may be useful for a broad audience, including knowledge management professionals who regularly facilitate peer learning, and researchers who study difficult-to-measure outcomes in various technical sectors. Specifically, it may useful for those interested in measuring the contribution of organizational learning, knowledge management, and adaptive management to organizational change and development results. Challenges and limitations around time, samples, and resources impacted the ability of the learning network members to produce categorical evidence about the impact of CLA on organizational effectiveness and development outcomes. However, the initiative produced interesting findings, such as how pivot logs proved most helpful for capturing development outcomes but are subject to biases.