The long journey of the EBRD’s Transition Impact

Evaluation Synthesis of the EBRD’s approach to Transition Impact (2017-23)

Background

Over the last year, IEvD and the Impact Team have substantially increased dialogue and exchange. Their common objective is to strengthen the Bank’s approach to TI. 

This realist evaluation synthesis aggregates relevant evaluation findings from a broad evidence-based body of evaluation knowledge covering 2017 — when the TI concept and operationalisation were last reviewed — to 2023. These findings were complemented by insights emerging from greater engagement and exchange with Management during the preparation of the synthesis. This has made it possible to contextualise past evaluative insights and weight them to enable a meta evaluation of such findings that speaks to current challenges of TI methodologies and results architecture.

The synthesis reaffirms the relevance of past evaluations and underscores the complexity of enhancing TI methodologies and results architectures. Strategic-level insights highlight EBRD’s pioneering focus on impact yet identify weaknesses in external communication and the transition results architecture. Operationallevel insights underscore the importance of the Expected Transition Impact methodology while pointing out its limitations in capturing the full scope of impact. 

Suggestions

IEvD suggestions include the following:

1. Focus on measuring systemic change. This may include articulating “transition pathways” for each TQ based on a Theory of Change at sector level, and reviewing the ATQs to better understand transition gaps by sector.

2. Consider modelling to better understand TI. Modelling may greatly help better understand the wider impact of the Bank’s interventions and bridge the “missing middle”; undertaking a feasibility study is suggested to understand which and how modelling tools are suitable for the EBRD.

3. Re-think the operationalisation of TI. Rethink the conceptual definition of the TQs and their operationalisation, for example, starting with the Resilience TQ, to better understand, capture, and measure TI, including in a context of crises.

4. Better capture TI in times of crisis. Review the evaluation’s methodology as the key tool to incentivise and measure impact and include it in the context of crisis. This can be complemented by developing a dedicated assessment to capture TI stemming from policy work non-associated with investments.