Venture Capital Investment Programme (VCIP, https://www.ebrdvcip.com/) is one of the several instruments that the Bank put in place to promote innovation in its countries of operation. The distinctive element of the VCIP is the provision of direct equity to early and growth-stage companies in software and web services, semiconductors and materials, communications, mobility and media, and cleantech. VCIP I and VCIP II are the first and second frameworks of this programme. The Bank gave the green light to VCIP I and VCIP II in September 2011 and May 2018, respectively. The principal objective of these frameworks are to support the development of technology innovation and its commercialisation, and to promote venture capital investments into the EBRD Region where there are scarce financing companies. The objective of the evaluation is to assess the merits of the VCIP, including the achievement of objectives and results, and to gather insights contributing to the improvement of the Programme in the future. This evaluation covers all investments that the Bank made under VCIP I.
Evaluability is essential in helping EBRD contribute effectively to fight climate change. Effective monitoring of results and progress will help the EBRD tell its green finance story clearly and credibly. The objective of this report is to provide an initial evaluability assessment of the EBRD’s Green Economy Transition to present suggestions to enhance the Bank’s approach to measure the impact of its green financing.
Globally, the energy sector is transforming rapidly. The green transition and the urgency of addressing the climate crisis is leading to significant changes in how energy systems operate. Lessons from evaluation can provide a valuable perspective on how to develop green energy systems that are sustainable, secure and affordable.
This knowledge paper focuses on providing evidence-based insights from evaluation on two central themes: the technical integration of intermittent renewable energy and building markets to set the right incentives. It draws lessons from the evaluations produced by Multi-Lateral Development Banks (MDBs) on their approaches to energy systems, supplemented with EBRD data and strategic documents outlining how MDBs approach energy systems.
This evaluation assesses the ‘early transition country’ (ETC) classification, its use in the EBRD, and contribution to furthering transition impact in EBRD’s smaller, and less advanced countries, in advance of EBRD’s expansion to Sub-Saharan Africa and Iraq. EvD evaluated the results of EBRD’s work from 2017 to 2022, across the three strategic priorities: i) private sector development and access to finance; ii) sustainable infrastructure development and renewable energy; and, iii) inter-regional connectivity and international integration. Findings and insights provide learning from experience as EBRD expands to the new regions. EvD makes three recommendations to strengthen focus on smaller, less advanced countries in the future.
EBRD has been implementing the Green Cities Programme (GrCP) since 2016 to promote urban solutions that reduce negative environmental changes and to scale up Green Economy Transition investments. Under the GrCP, municipal stakeholders are supported in the preparation of the Green City Action Plan, bringing together policy, regulatory, and investment priorities, which later on inform investment decisions of the EBRD and other financiers. EBRD has concentrated investments in the GrCP under the Green Cities Frameworks, which were approved by the Board in 2016 and 2018, and then extended in 2020 and 2021.
This report presents the results of the interim evaluation of the first five years of GrCP implementation (2016-2021), analyzing the extent to which the programme has been able to function as a catalyst for municipal solutions to mitigate or adapt to climate change.
The EBRD Shareholder Special Fund (SSF) was established in 2008 so the Bank could respond more effectively to the remaining transition challenges in its region of operations by providing essential grant resources – technical cooperation and co-investment grants – to support the Bank’s operations and policy dialogue. Since its inception, more than €1.1 billion of the Bank’s net income allocation has been channelled through the SSF, supporting upwards of 2,200 TC and co-investment grant projects throughout the EBRD regions. In parallel to backing the Bank’s transition agenda, the SSF has also become an integral part of its crisis response toolbox. The last full evaluation of the SSF took place in 2014. This latest evaluation offers an independent, comprehensive up-to-date assessment of the Fund. It looked principally at the period January 2016 to December 2020, though it also considered changes in and performance of the SSF since, where relevant.
This report contains a review of EBRD operations and policy dialogue in support of utility-scale solar projects until the end of 2020. Based on the assessment of a sample of 10 projects in six countries and a thorough portfolio analysis, the evaluation identifies trends, lessons and themes relevant to this sector.
Rapid Assessment of EBRD's Solidarity Package
In this video, the independent Evaluation Department explains the main findings of its Rapid Assessment of EBRD’s Solidarity Package in response to the COVID-19 pandemic.
This evaluation provides a strategic review of EBRD’s Public Sector Operations (PSO), which is almost exclusively financing infrastructure. EBRD’s Sustainable Infrastructure Group (SIG) finances energy, transport, and municipal infrastructure in countries of operation (COOs) using a combination of debt, equity and technical cooperation (TC) grants. The evaluation assesses EBRD’s PSO contribution to structural and institutional change in its COOs that facilitated transition Impact (TI). The evaluation period is 2010-2020.
The evaluation is based on a Theory of Change (TOC), which identifies demand for public sector finance and assesses EBRD’s supply response. Objectives and results frameworks are drawn from EBRD strategies and financing documents. The evaluation looks at inputs (markets, products, policies, staff and finance).
Results are assessed looking at benefits (relevance of objectives, and effectiveness of outputs, outcomes and impacts), and costs to EBRD (efficiency), relative to targets in results frameworks. Due to constraints arising from the Covid Crisis, it was not possible to make field trips to COOs to interview stakeholders. The study draws on desk research and remote interviews.
The Bank’s engagement in the West Bank & Gaza (WB&G) has to be assessed five years following its approval by the EBRD Board of Governors. This evaluation was performed in a short timeframe to deliver findings and recommendations on time for Management’s next Net Income Allocation request and to inform the strategic engagement update in the WB&G that Management is undertaking.
EBRD has engaged in activities in the WB&G under three strategic themes aimed at: (1) enhancing private sector led growth through direct support for competitiveness and innovation, (2) strengthening the capacity of financial intermediaries with a particular focus on increasing access to finance for MSMEs and (3) fostering energy efficiency and sustainability.
The questions posed by the evaluation team on EBRD’s investments activities in the WB&G are based on four evaluation criteria and grouped under the four headings:
I. Relevance: To what extent have EBRD’s activities in the WB&G been responsive to the needs of the private sector in the region, donors and the WB&G authorities?
II. Internal and External coherence: How coherent have the Bank’s activities in the WB&G been internally and externally?
III. Effectiveness: What are the early results achieved by the Bank in the region; and to what extent Bank’s projects have achieved, or are positioned to achieve, their intended objectives?
IV. Efficiency: Have EBRD’s resourcing and management supported efficiently the achievement of its intended results?
Following approval of Strategic and Capital Framework SCF 2021-2025 (SCF2), the Board of Directors asked Evaluation Department to review previous Strategy Implementation Plans (SIPs) to provide findings and lessons on early experiences that could usefully inform the ongoing SIP process and content with a view to improving the operationalisation of the SCF.
This study aims to provide the Board with findings and lessons on the design and the content of the SIP; it is intended as a discussion paper rather than a full formal evaluation for several reasons. These include ongoing changes in key elements of the Bank’s strategic architecture, the fact that SCF2, and its first SIP, depart sharply from SCF1 and its SIPs, and the unusual circumstances due to the Covid-19 crisis.
The purpose of this evaluation is to provide evidence-based analysis contributing to: i) institutional accountability by evaluating the characteristics of the past operations against expectations; and, ii) institutional learning by offering insights relevant for the development of future strategies and project design.
It is structured around the three sub-sectors of Sustainable Infrastructure – Municipal and Environmental Infrastructure (MEI), Transport sector operations, and Energy sector operations. At the same time it focusses specifically on three individual Advanced Transition Countries, each of which had a critical mass of operations in one of the chosen sub-sectors. The objective was to be able to identify both common features and cumulative effects in the individual country contexts that are the Bank’s stated benchmark for transition progress. The selected sub-sector/country clusters were: i) MEI operations in Croatia; ii) Transport operations in Hungary; and, iii) Energy operations in Poland.
Learning and Knowledge Management (LKM) at the European Bank for Reconstruction and Development (EBRD) is a thematic evaluation included in the Evaluation department’s (EvD’s) 2020 work programme following a recommendation of the 2019 independent external evaluation of the EBRD’s evaluation system (the Kirk Report).
Although this study stems from a recommendation directed at learning from evaluation, it takes a broader perspective on the way the EBRD learns as an organisation and what place knowledge has among its operational assets.
The scope of the evaluation covers organisational learning (which includes experiential learning) and the knowledge management (KM) processes that contribute to it. Both explicit and tacit knowledge are considered. The evaluation does not look into the EBRD’s role as a source of knowledge for clients or external stakeholders – a separate paper on this topic may be prepared at a later date.
This is the first EvD evaluation to offer a comprehensive analysis of key building blocks enabling the EBRD to be a knowledge and learning organisation.
This report consists of several distinct parts targeted at different audiences with differing interests. A summary report presents key conclusions and actionable recommendations of the LKM evaluation, with the primary focus on the Bank’s decision makers. Six technical papers (“TPs”) present in-depth analysis of the evidence and offer conclusions along the main building blocks of the LKM system.
The EBRD’s operational engagement in healthcare equipment and services was minimal for most of its first two decades. After extensive internal debate, the EBRD introduced an Updated Approach (UA) in 2014 for engagement in specific areas of healthcare services. This evaluation is a first focussed EvD effort to assess activities under the UA. This evaluation covers both investment and non-investment activities approved between 2008 and 2018, whose use of proceeds and/or transition ambition directly affected the Healthcare Sector; it is not limited to operations covered by the UA.
The EBRD has used Nominee directors (NDs) since its inception: as a key element of its direct equity investments, as an instrument of transition impact, and to represent EBRD’s interests as a minority shareholder. This study is EvD’s first review of the Nominee Director programme for direct equity investments. The study used an extensive EBRD document and wider literature review, and prepared 14 case studies representing multiple sectors and company types. EvD interviewed operation leaders and NDs in each of the case studies, along with a wide range of staff across the Bank, including from Economics Policy and Governance Group, Legal Transition Team, Office of the Chief Compliance Officer and Equity Portfolio Monitoring Unit. EvD also surveyed current NDs and EBRD operations staff managing direct equity investments with ND roles.
The EBRD’s Trade Facilitation Programme (TFP) was established in 1993 to support trade flows in Bank Countries of Operation through short-term facilities to provide liquidity. A new annual limit of €3.0 billion was agreed by the Board in July 2020. This evaluation provides background on trends in international trade and their major drivers, reviews other MDBs’ approaches to trade facilitation, and summarises the findings of previous evaluations of trade facilitation activities.
This report contains a review of the Bank’s hydrocarbon operations. It includes the evaluation of a sample of six projects, which is presented in the broader context of the Bank’s past hydrocarbon operations and portfolio analysis. It seeks to identify trends, as well as common lessons and themes that are relevant to this sector by utilising findings from the sample assessment and taking into account other recent evaluations. The linkages and incorporation of hydrocarbons into selected country and sector strategies are examined, along with how other IFIs approach hydrocarbons and how the EBRD collaborated with them.
A first independent review of the Bank’s evaluation systems completed in 2019 (the Kirk Report) covered both the overall structure of the Bank’s systems as well as key specific components, asking the core question of the degree to which intended operational results can be reliably and credibly verified. It raised additional specific concerns about the evaluability of the “Transition Qualities” at the heart of the Bank’s new approach, and recommended that the independent Evaluation Department (EvD) complete “a brief assessment of the evaluability of the transition qualities as concepts and operational tools.” This paper is EvD’s response.
The Legal Transition Programme (LTP) is the EBRD’s flagship provider of legal and regulatory advisory work and technical assistance, combining it with investment activities to support legal reforms needed to strengthen investment climates and build markets. LTP activities are intended to provide and to demonstrate the critical links between the Bank’s discrete investments and the wider systemic reforms and transition that it was created to help drive. LTP work touches every EBRD Country of Operation and virtually the full range of the Bank’s lending and non-lending operations.
The Evaluation Department (EvD) was asked by the Board of Directors to evaluate the activities and performance of the LTP between 2011 and 2018, and to assess Management’s implementation of recommendations made by EvD in its previous evaluation of the Programme (2012). This paper provides an overview of LTP operations over the period, identifies key findings and conclusions, and makes recommendations for consideration by Management and the Executive Board.
The EBRD’s ability to combine non-sovereign investment and policy engagement was an important contributor to shareholders’ decision to offer membership to countries of the South-east Mediterranean and North Africa region (known as the SEMED region in EBRD) following the major political upheavals of 2011. EBRD’s experience with policy engagement work in these new and challenging member countries led to an opportunity for a provisional assessment of achievements. This evaluation reviews the content, objectives and, to the extent possible, performance of the Bank’s policy engagement in SEMED between 2011 and 2018. It looks at how the EBRD implemented the 2015 Enhanced and Structured Approach to Policy Dialogue in the SEMED region. The study provides findings for future policy work in SEMED, and potential new countries of operations.
Mobilisation of incremental private sector finance by the multilateral development institutions has always been formally at the core of their public purpose. In some it is explicitly established as a central objective; EBRD’s Articles specifically do so, as do the founding Articles of the World Bank, IFC and other comparator institutions.
The study seeks to contribute to Board and Management understanding, strategic thinking and operational decision-making about the Bank’s role in mobilising additional private capital to support transition. Organisationally it begins by providing essential factual background, including details on the instruments used, and current MDB strategies, practice and outcomes; key findings of existing performance assessments and evaluations. The bulk of the paper then looks closely at mobilisation practices and issues specific to EBRD and its performance.
This report presents an evaluation of nine infrastructure projects promoting cross-border connectivity, eight in transport and one in energy. It assesses their overall performance, but gives particular emphasis to their intended and observable impact on cross-border connectivity and integration.
The evaluation seeks to establish a broader context for its project-specific work - starting with a brief analysis of the evolution of the Bank’s approach to regional integration, a review of the portfolio of early cross-border projects, as well as of more recent operations after “Integrated” became one of the six transition qualities. The linkages and incorporation of regional integration into selected country and sector strategies are examined, along with how other IFIs approach regional integration and how EBRD collaborates with them.
This is an EvD cluster evaluation of the mining portfolio in Mongolia since EBRD operations began in 2006 and involved a review of the themes emerging from 14 private sector projects, worth €734m of net cumulative investment (€704m debt, €30m equity) and associated donor funded EBRD technical assistance, most of which was non-transactional. The mining sector in Mongolia is critical to the economy, representing around 27% of GDP, 19% of state budget, and 88% of exports, mainly to China. Top mineral exports are coal, followed by copper and gold.
The first period of EBRD investment, from 2006 to 2012, was characterised by a mining boom with the EBRD becoming closely involved, less risk adverse and more prepared to support small and domestic players. From 2012 to 2016, global commodity prices fell. Around half the existing portfolio ended up in Corporate Recovery in part due to the global commodity price slump, business conduct issues and lack of sponsor support.
The review makes three recommendations based on its findings, which cover evaluation of the biggest current operation, improvements more generally to project design for stronger delivery of the Bank’s transition mandate, and critically, development of an improved programme of policy dialogue and IFI collaboration.
The ‘EBRD Ukraine Stabilisation and Sustainable Growth Multi-Donor Account’ (Ukraine MDA) was launched in 2014 as part of a multilateral crisis-response measures to provide urgent support to Ukraine in the wake of major political, security, economic and humanitarian crises.
A total of fourteen donors have contributed €53.5 million to the Ukraine MDA, out of which a portfolio of 38 projects have so far been or are being delivered, with a total earmarked value of €36.9 million. The Fund is managed by the EBRD and is the only such single-country focused arrangement in the Bank.
Bank’s independent Evaluation Department (EvD) was asked by EBRD management and the MDA donors to produce an interim evaluation on an accelerated basis in order to inform discussion and decisions about Fund’s effectiveness and potential extension ahead of its current July 2020 closure date.
The evaluation provides an assessment of: MDA relevance and effectiveness as a crisis response mechanism; its governance principles; the efficiency and effectiveness of its activities and sustainability of its results; and, any impact it has on the Ukrainian investment climate and economic growth, where possible to assess. The evaluation covers nine completed projects and 27 ongoing projects spanning 2014 to mid-2019.
This paper provides EvD’s initial contribution to the Management review as directed by a first-ever “Independent External Evaluation of EBRD's Evaluation System” (Kirk Report). It consolidates analysis, findings and recommendations that EvD has largely provided in other contexts and that are directly relevant to the Kirk findings and Management’s work ahead. The objective is to contribute in an advisory capacity to a wider Management-led process in which EvD will also be systematically engaged. It also seeks to inform the Board and assist it to provide effective guidance for further system development. Overall, however, the findings and insights that are already available – from the Kirk Report, from the accumulated work of EvD, and from relevant experience across the IFI system -- provide a high-value basis on which Management can proceed.
The Women in Business programme in Turkey was the first EBRD approach to “support women’s access to finance and entrepreneurship, combining dedicated financing, backed by technical advice to local financial institutions and a direct engagement with women entrepreneurs through access to training, advisory services, mentoring and networking.” This study seeks to review the first phase of the programme in Turkey from 2014 to 2017 and assess its relevance, its effectiveness in achieving intended results and its efficiency. The Evaluation Department aims to answer three evaluation questions to in this review:
Recognition of climate change as a major global public policy challenge has grown since key concerns were raised in 1992. The G8 elevated the importance of climate change in 2005 and there have been a series of international initiatives since that time. In 2015 developed countries were urged to mobilize US$100bn annually by 2020 and a comprehensive framework was agreed in Paris based on enforceable country-level commitments to carbon targets. Against this backdrop EBRD substantially increased the strategic priority to be given to work on sustainability and, more specifically, climate. This evaluation reviews the main climate initiatives in terms of strategic objectives, operational focus and instruments, organization arrangements, portfolio structure, and performance. It confirms that EBRD has initiated major steps on an institution-wide basis to respond to shareholder directions. At the same time, the evaluation also confirms opportunities to strengthen existing approaches and the pressing need to improve processes. This includes much improved clarity as to priorities and objectives, particularly with respect to results; a more systematic and transparent approach to allocating resources; and, building capacity to assess actual results in accordance with country needs.
This study was requested by the Board Effectiveness and Efficiency Group, following the implementation by the Bank of a pilot, which increased the EBRD’s delegated authority threshold from €10 million to €25 million. On the completion of this pilot, the increased threshold became the Bank’s permanent policy. At the same time, the Evaluation Department (EvD) was asked to analyse the changes in the Bank’s approvals structure, quality of information provided to the Board on sub-projects approved by delegation, as well as quality of reporting on framework and sub-project performance.
The EBRD’s 2010 Property Strategy was strongly shaped by the financial crisis of 2008, which hit the Property and Tourism sector (P&T) particularly hard. “Crisis response” was an immediate and dominant operational priority. Longer-term, the Strategy’s objectives were limited to a demonstration of energy efficiency in buildings, “institution building” and an increased emphasis on projects in less advanced countries and secondary cities.
During the Strategy period (2010-2017) the Bank financed 65 P&T projects for €1.18 billion, broadly unchanged from under the previous strategy; sub-sectoral and country composition of operations were also similar, albeit with sharply lower post-crisis investments into property funds.
Since its launch in 1995 as an ad hoc technical assistance resource, the EBRD’s Legal Transition Programme has been through many transformations. Presently it is a key driver of legal reforms in the Bank’s countries of operation.
The Evaluation Department’s (EvD) 2018 Work Programme includes a thematic evaluation of the LTP, which will focus on the impact and sustainability of activities approved and implemented over the period 2011-2018.
The evaluation team has identified several evaluation questions that will guide the analysis and shape final findings and recommendations. The emphasis of the evaluation will centre on efficiency, effectiveness, impact, and sustainability both externally (benefiting the country) and internally (benefiting the Bank’s own policies and operations). The evaluation will also include three or four case studies, which will be representative of various transition stages.
Lending through financial intermediaries, widely known internally as “credit lines,” has been a significant part of the Bank's operations since its inception. Credit lines have allowed it to reach larger numbers of smaller borrowers than possible through direct lending, and alongside other forms of support have contributed to financial sector development and growth of strong partner intermediaries. They have also evolved significantly over time and are critical instruments for delivery by the Bank on a wide and ambitious range of new commitments.
This evaluation examines the design, targeting, execution, monitoring and effects of a large sample of recent credit lines, defined for its purposes as “term loans extended to financial intermediaries with a defined use of proceeds.” and focuses principally on the market context, strategic approach and signed operations of the period 2011-2015.
The transport sector is a key enabler of economic growth and transition in the region and investment in the sector has been central to the Bank’s strategy and operations since its inception in 1992. This is a review of the Bank’s 2013 Transport Sector Strategy and resulting operations; it is intended to inform Management development and Board consideration of a planned new sector strategic document in early 2019.
EvD outlined the intended focus of its review in an Approach Paper discussed and agreed with Management. Three evaluation questions were asked: whether the TSS set appropriate priorities; what results are observable while the strategic period is not ended yet and how well the Bank has implemented the strategy. EvD reviewed 82 operations approved between January 2013 and May 2017 amounting to nearly €5billion, together with data on 114 transport TC operations for €40.5 million.
Mobilizing private capital has been central to the Bank’s core transition mandate since its founding, and to achieving major recent institutional commitments such as supporting the SDGs, climate resilience, and inclusion. Significantly greater private investment, both domestic and foreign, is in turn critically dependent upon a supportive investment climate, which has long been identified as a strategic and operational priority for the Bank and an area of comparative advantage.
This EvD study identifies the principal objectives and components of the Bank’s work on investment climate (IC) improvement, assesses the effectiveness of design and delivery, and presents findings and conclusions that may contribute to programmatic performance in the future. Its intended scope includes efficiency and resource deployment across multiple work streams and observable contributions toward IC improvements, including analysis of the Investment Climate and Governance Initiative (ICGI) and other initiatives, and case studies. Its principal purpose is to draw out findings relevant to existing and emerging operational and strategical priorities and provide evidence-based insights useful for future programme design and delivery and internal learning.
The proposed study is intended to:
Large infrastructure financing gaps in EBRD countries of operation have long been identified as a key challenge and opportunity for the Bank. There is an increasing recognition that traditional government funding and methods are not sufficient to close growing gaps, and increased private sector engagement is now widely understood to be essential.
The EBRD Infrastructure Project Preparation Facility (IPPF) was a high profile initiative approved by the Board in October 2014 to scale up infrastructure support. As a “delivery mechanism to improve the efficiency, quality and replicability of infrastructure projects,” the IPPF is meant to help governments build an infrastructure project pipeline using additional and improved project preparation support along with policy dialogue.
On approval of the IPPF the Board requested a stock taking after three years and this interim review by the Evaluation Department is intended to assess IPPF’s performance against its wider objectives, identifying any early results and difficulties, and its potential to be a co-investment multi donor fund for specific countries and sectors.
The Review of the 2013 EBRD Energy Sector Strategy investigates key issues around: what were the stated intentions and expectations of the 2013 Energy Sector Strategy; how did it reflect and integrate transition challenges and develop operational priorities; were clear and measurable goals established; were critical execution issues such as resources and cross-institutional collaboration addressed effectively; what results are observable thus far. Lastly, the Review identifies actionable recommendations for the next EBRD Energy Sector Strategy to be approved by end 2018.
Additionality is one of EBRD’s three foundational operating mandates. It has featured centrally since 1992 in investment selection and design, in Bank representations of its role and purpose, and in Board/shareholder engagement with Management. For EBRD shareholders Additionality is the foundation of their claim for public resources and therefore has been of consistently high interest and concern. This review seeks to fill some gaps that exist around additionality in EBRD and contribute to emerging Board and Management efforts to develop a stronger and more satisfactory institutional framework.
