Overview
Organizations are increasingly being required to provide concrete evidence of not only their outputs, but the impact they are having on people, planet and economics. In this context, a Monitoring, Evaluation, and Learning (MEL) system is a crucial element which enables organizations to better track, analyse, report and communicate their impact. More broadly, MEL systems can strengthen organizational accountability, improve decision-making and performance, and facilitate an ongoing cycle of learning.
Our approach to M&E – going beyond the basics
At NewForesight, we can help you to develop a robust MEL system that is insightful, actionable, and efficient.
Critical success factors of a strong MEL system
Insights clearly linked to the Theory of Change
Increasingly there is a need to report on impact and outcomes rather than solely on activities and outputs. We believe that strong impact measurement is key to informing strategic considerations. An organizational Theory of Change should be well-developed and understood to align the MEL system to its objectives.
Actionable towards the learning agendas
The whole system should be structured in a way that serves the learning agenda. Every indicator should be assessed on its action perspective and feasibility. Additionally, it should be clear to the recipients what data they receive and more importantly, how they can use it.
Balancing efficient use of resources
We understand the limited resources and capacity for extensive data collection and balancing between the need and capacity for more extensive M&E systems. This also includes testing the feasibility with end users and partners.
From experience we have learned that the following five topics bring the most success:
1. From Collecting and storing data to data as a value proposition
We don’t just assist organizations to collect data for compliance sake – we make sure it’s useful for them, and other stakeholders more broadly. We help organizations organize data in a coherent way, offer intermediate insights and benchmark performance.
2. From Reporting on outputs to reporting on impact and systemic change
Many organizations see M&E as a cumbersome and expensive process that fails to deliver deeper insights. The root cause of this is often that organizations’ hyperfocus on output indicators, which are easy to collect – but often not very insightful. At NewForesight, we focus on the bigger picture outcomes and impact indicators to help organizations more deeply understand their trajectory, while ensuring the MEL work is financially sustainable.
3. From Project-based impact reporting to tracking long-term progress
We don’t see MEL activities as having a start and end point (i.e. driven by donor requirements). We work with organizations to think holistically about how performance measurement can help them achieve their long-term vision.
4. From Annual impact evaluations to ongoing data capture
While no doubt there is value in conducting annual impact evaluations, we move beyond static approaches and support our clients to implement systems which enable them to access real-time, streamlined data that demonstrates progress, not perfection. In turn – this supports timelier decision making and enhances organizational agility.
5. From Strong attribution to causal logic to prove impact
We are passionate about supporting organizations improve the attribution between the activities they deliver and the outcomes they seek to achieve. For us, rather than seeking the holy grail of perfect attribution, it is more important that organizations acknowledge that change takes time – and take an incremental approach to routinely test and refine their causal logic (i.e. as set out in their Theory of Change).
Services
From experience, we have learned that to get the best result MEL development should go through four phases. Below is a high-level overview of the services we support our clients with.
Our analytical framework for establishing MEL systems
A robust MEL system needs to have the right building blocks and infrastructure in place. Building a MEL system consists of a series of steps with key elements that need to be considered and reflected upon when developing the system and operationalizing it. It also needs to be accompanied by continual capacity development, for instance on data collection, analytics reporting, and quality, etc.
Below we present our overarching approach to developing a MEL system.
We have the expertise in the whole lifecycle of M&E systems to enable adoption after implementation
Projects
Hear The World Foundation
Moving from output to outcome and impact measurement
The request: The Hear the World Foundation (HTWF) supports projects around the world that enable children with hearing loss to develop appropriate to their age, and ultimately improve their quality of life. HTWF wanted to evaluate and extend its impact measurement system to shift from out to outcomes and long-term impact reporting to deliver more strategic insights and compelling communications.
Activities: We worked closely with HTWF to identify goals, and reviewed and optimized existing materials, including the Theory of Change. To explore reporting on Outcomes and Impact we co-developed patient-level indicators which were merged into measurement strategies which were assessed on feasibility, value, and level of measurement – including with three project partners in South America and the Middle East. We formulated recommendations to solidify the current measurement approach, including a selective measurement on Outcome and Impact themes. Additionally, we made long-term recommendations about how HTWF could better systematically measure impact themes across its entire portfolio.
Our value:
“Working with NewForesight enabled us to thoroughly assess our options and have a good baseline to make decisions for the future. The team is very easy to work with and commits fully to the client topic. Crossing the world of impact management and Foundation sector is hard to find and valuable to have on the market for NPOs.”
Joelle Pianzola, Director Hear the World Foundation at Sonova Group.
Request: RA requested external support to assess and monitor the impact of SD/SI payments on producers’ ability to meet their sustainability commitments and to evaluate the extent to which SD/SI:
- helps to operationalize the principle of “shared responsibility in supply chains”,
- are effective policy instruments, and
- contribute to creating accountability to all stakeholders in terms of making a difference to producers, workers, and the environment.
Activities: NewForesight built an understanding of how SD/SI worked by reviewing RA’s Theory of Change and translating evaluations questions into hypotheses to be tested, and provided insights on the sector differences in application and roll-out of SD/SI. We also developed a robust evaluation design based upon the Theory of Change, evaluation questions and data availability. This included a risk analysis and assessing validity threats as well as determining the most suitable research methodology. This work culminated in the delivery of inception and baseline evaluation reports.
Rainforest Alliance
Evaluation of Sustainability Differential and Sustainability Investments (SD/SI)
TechnoServe
Developing a learning agenda, Measurement and evaluation framework, and supporting communications
The request: We were asked to help the trader establish the link between the company’s sustainability goals and their business operations for different divisions, and create ways to measure success.
Activities: NewForesight developed a Theory of Change and KPI framework, aligned with the client’s sustainability agenda which would help them to measure progress against goals and communicate (report) externally. We helped to operationalize the ToC into M&E tools such as a performance dashboard which tracked the operations across the client’s different origins. We also developed the M&E handbook and the learning agenda, while building the team’s capacity to further use the tools and implement M&E processes..
The request: The Syngenta Foundation required our support to develop a global performance measurement framework and strategy for continuous monitoring across its programs.
Activities: NewForesight supported SFSA in the development of their performance management system. NewForesight conducted a study to understand current activities’ contribution to SFSA’s desired impact and what indicators would increase understanding of impact. NewForesight tested the system to enable SFSA to understand risk & improvement areas and develop a learning strategy to enhance impact, based on the PSM information. Value was added through advanced expertise in measuring, analyzing, benchmarking farm-level input, and understanding how to optimize efficient data collection and impact assessment to drive innovation for agribusinesses and SMEs.
Syngenta Foundation
Global Performance Measurement Framework
ICCO
Global M&E system for the cocoa sector
The request: ICCO sought our support to develop a global monitoring, evaluation and learning framework for the global cocoa sector.
Activities: We developed the Global Cocoa Agenda Monitoring Framework, including the development of the program’s Theory of Change (economic performance, environmental & social sustainability, trade flows), for the ICCO and all its member countries and organizations. We also created a toolkit for (impact) data collection for numerous multi-stakeholder projects around the world driven by a range of regional implementation partner organizations, covering topics including quality, accountability, participation, standardization, reporting, credibility.
The request: NewForesight was tasked with consolidating NKG Bloom’s sustainable coffee sourcing strategy.
Activities: Together with the NKG team we defined the vision, the interventions needed, a system to track progress and supported the operationalization of the strategy. We developed the Theory of Change, the KPIs, Monitoring, Evaluation and Learning framework, and the implementation system. We developed protocols for the suppliers to implement sustainable practices and developed the internal verification system. Lastly, we developed capacity within the team and guided them to successfully develop the NKG BLOOM handbook and NKG BLOOM M&E operational manual.
NKG Bloom
Supporting the consolidation of NKG’s sustainable coffee program: NKG BLOOM
NKG Bloom
Evaluating the baseline impact of smallholders producers across 30 KPIs
Activities: For IDH and Neumann Kaffee Gruppe (NKG) we conducted a baseline impact evaluation for the NKG BLOOM coffee program, establishing the baseline in four African and Latin American countries, measuring performance across 30 KPIs related to living income, farm performance, social indicators (education, food security, etc.), and environmental sustainability. The study was based upon raw data collected for 2,000 farmers and resulted in a baseline report (the executive summary is available here) and strategic discussions about next steps.
The request: Rikolto sought an external assessor to evaluate the impact of their programmatic interventions and change strategies to develop the rice sector in DR Congo, Tanzania, Uganda, Burkina Faso and Benin.
Activities: We conducted an impact assessment of selected projects and considered the extent to which the projects:
- improved the livelihoods of smallholder farmers;
- built the business and organizational capacities of farmer organizations, and
- improved the institutional environment for inclusive, sustainable markets.
As part of this project, we also assessed the potential global impact of Rikolto’s change strategies and provided recommendations for improvement. We provided trainings to the team on data collection and assessments.
Our value: Rikolto gained a clearer picture of the impact of their programmatic interventions and built their internal knowledge and capacity to undertake data collection and analysis.
Rikolto
VECO Impact Assessment
SwissContact
Building a centralized M&E database for KPI reporting
The request: Develop an evaluation framework, learning agenda and conduct capacity building.
Activities: As part of this project, we reviewed the Theory of Change and impact indicators to develop a KPI framework to monitor both performance and impact. Next, we developed accompanying data collection methods and tools. Finally, we supported the operationalization of the M&E system by developing a tool to enhance portfolio management and strategic decision making and provided training to the SwissContact team and the local team in Nepal (Sahaj).
Our value: As a result, Swiss Contact has increased the capacity of its local team in Nepal to refine the on-the-ground activities to ensure impact in line with the overall ambition, to effectively collect data and learn and communicate about impact achieved.
Reports & Articles
Measuring performance to maximize impact: lessons learned at the Syngenta Foundation for Sustainable Agriculture
Looking ahead: 4 pathways leading to next level collaboration for sustainability in 2019
What gets measured gets done: Four success factors to effectively incorporate performance measurement:
NKG BLOOM Impact Baseline Evaluation Report
NKG BLOOM Impact Baseline Evaluation Executive Summary