Special Studies (themes)

2016: Country Strategies - an initial review

This paper summarises an external assessment of eight new country strategies which use the EBRD's new country strategy results framework. The assessment is conducted against the core objectives of the new results framework design and good practice features.
The paper also presents a quality assessment checklist drawing on the good practice of other IFIs adjusted for the EBRD context, covering four key areas: analysis, selection, instruments and results.
 

2010: EBRD's response to the 2008 - 2009 Crisis

In the autumn of 2008, the EBRD announced an initiative called the "Crisis Response" as part of a number of actions taken by the G-20 governments and international financial institutions to changed market conditions after the bankruptcy of Lehman Brothers in the US. This study, carried out by the Bank’s independent Evaluation Department, focuses on the crisis response from an EBRD perspective and is grounded in the EBRD's unique mandate and instruments. The study makes observations and offers recommendations on possible adjustments to the Bank's future activities in the light of the study's findings.
 

2016: Special Study EBRD's Sustainable Energy Financing Facilities (SEFFs)

This study is an evaluation of the design, implementation and impacts of the EBRD’s Sustainable Energy Financing Facilities (SEFFs) from 2004 to 2013. The €2.4 billion SEFF portfolio includes 27 facilities across 20 countries of operations, working with 90 local financial intermediaries. The facilities are packages of credit lines and grant financed technical assistance to participating financial intermediaries and sub borrowers to support energy efficiency and small scale renewable energy investments. The facilities often include incentive payments to ultimate beneficiaries – small and medium sized enterprises and households. This study examines the extent to which SEFFs have met objectives and contributed to transition impact, and suggests ways to enhance performance going forward.
 

2008: Environmental Policy 2003 Review: Achieving the Bank's environmental mandate through direct investments

This report looks at the effectiveness and efficiency of the implementation of the Bank's 2003 Environmental Policy, through direct investments, between 2003-2006. Taken together with EUD's companion study on the Financial Intermediaries sector, these two reports comprise EUD's contribution to the drafting of a new Environmental Policy.
 
 

2006: Achieving the Bank's environmental mandate through financial intermediaries

The report looks at the effectiveness of the financial intermediaries in implementing the Bank's policies and procedures and how the subprojects have improved environmental quality in line with the Bank's requirements.
 

2001: Environmental performance

Covers a sample of projects and other activities of the EBRD in environmentally important areas from 1991-2001. It aims to identify the Bank's environmental impact in the region and help to improve its future environmental performance. 
 

2017: Equity Operations

This evaluation focuses on the Bank’s equity portfolio approach and developments between 2005 and 2016. It identifies significant and difficult issues regarding performance and approach and, on the basis of these findings, makes several recommendations for consideration by the Board and Management.

2013: Achieving equity investment objectives: A review

This study assesses the extent to which initiatives taken in equity investment and management by the EBRD from 2007 to 2011 are contributing to better achievement of the Bank’s equity investment objectives. It also considers how far equity objectives are specified at entry in a way that is measurable and supports effective monitoring. The study is based on a desk review of core project records for a sample of 17 out of 116 direct equity investments. It is presented around the three components that form the ‘equity story’ in the EBRD and need to come together to support sound equity investment: value creation; transition impact; and risk.
 
 

2015: Evaluation insights on gender mainstreaming

The main purpose of this report is to provide a brief review of some key features of the experience of other IFIs in order to constructively inform the internal EBRD strategy development process. The EBRD has elevated gender as a strategic priority and is currently developing a Bank-wide Strategy for the Promotion of Gender. Other international financial institutions (IFIs) have substantially intensified their efforts in this area and made significant changes at the levels of strategy, corporate structure and operations. EBRD can learn from this experience.

The paper describes significant features of the way IFIs are approaching gender today – in terms of corporate commitments, attention to private sector operations, and for the first time, a more consistent approach to incorporating gender in evaluation. It finds that having set up the corporate mechanics to support their gender commitments, IFIs have adopted a largely pragmatic approach to adapting practices for best results on the ground. Many IFIs focussed on the public sector initially to gain traction and awareness on gender. Having done this, many are now moving to include the private sector where this increased attention is balanced with increased interest from clients. Surprisingly, given the maturity and strength of the gender agenda within most IFIs, much less progress is observed in addressing gender within evaluation, though many expect substantial changes here in the near future.
 
 

2012: Legal transition programme review

This study is an evaluation of the Legal Transition Programme’s activities from 2001 – 2011, through a review of a sample of 30 legal reform projects and advisory projects in Armenia, Hungary, Mongolia, Russia and Serbia. It was conducted by the Evaluation department in conjunction with three external experts: Professor Douglas Arner (University of Hong Kong), Professor Charles Booth (University of Hawaii) and Professor Gordon R Walker (La Trobe University). Overall the programme was found to be successful due to its compatibility with the Bank’s activities and highly relevant due to its support of the Bank’s investments through contributions to legal improvements. The programme’s projects have made a core contribution to the transition process, influencing domestic policy formulation and contributing to stronger free market economies. The transition impact and sustainability of the programme were found to be excellent.
 
 

2017: Local Capital Markets

This evaluation is focused specifically on the Bank’s local capital markets development work under the local currency strategy. It does not assess the substantial amount of work focused on Llocal currency activities, though in many cases this may have had secondary local capital markets dimensions. It confirms that considerable effort and skill has gone into delivering specific transactions and technical cooperation projects (TCs) in challenging contexts, resulting in discrete accomplishments in numerous cases. The commitment to a higher institutional profile and priority at the level of sector and country strategy and operations laid out in the Strategy depends on the content of new-format country strategies and is not yet observable. Solid evidence of wider contributions to local capital markets is limited.

2013: Performance metrics- How well do EBRD projects specify expected results?

The purpose of this study was to develop and test an evaluability assessment checklist that could have broader relevance as a quality assurance and learning tool at the EBRD. Further, the results from pilot testing of the checklist on a statistically valid sample of recent Operation Reports approved by the Bank between 1 July 2011 and 30 June 2012 provides immediate feedback to Operations staff on the feasibility of applying the checklist as well as an initial baseline of experience for future assessments.
 
 

2012: Discussion brief- Evaluability and the EBRD

The paper introduces the idea of evaluability, a concept widely used by international finance institutions (IFIs) to improve the performance of their operations, and their ability to report on the results achieved. Since the EBRD does not explicitly use evaluability, this brief explores the relevance of doing so. The paper aims to stimulate internal discussion on whether the concept offers value to the EBRD and whether it should be adopted.
 

2014: The EBRD’s experience with policy dialogue in Ukraine

This evaluation presents the findings of a study of the EBRD’s experience with policy dialogue in Ukraine. It provides insights to inform future policy dialogue efforts in Ukraine, and in other countries where the experience in Ukraine is considered relevant. It also provides ideas on how policy dialogue by the Bank might be made more effective as it moves to strengthen this area of its business.
 

Case studies:

 
 

2011: Policy dialogue 2010

Policy dialogue is one of the EBRD’s major instruments for assisting its countries of operations to achieve their transition objectives. This report summarises the Evaluation Department’s (EvD) 2010 review of the concept and practice of policy dialogue in the EBRD. The report summarises the findings, lessons and recommendations to help the EBRD improve its policy dialogue in the years to come. It focuses on the role, concept and scope of policy dialogue in the EBRD, reviews international-, country-, sector- and project-level policy dialogue, drawing on four case studies, and finally summarises the key findings and lessons, and makes recommendations.

2016: The EBRD experience with resident offices

In the EBRD’s 25th anniversary year, with a new strategic and capital framework for 2016 to 2020 in place and an operational effectiveness and efficiency initiative underway it is timely for the Bank to take stock of its field presence system. This evaluation identifies major challenges and opportunities for the Bank. Recommendations from this study suggest a new way forward for the resident office system.

 
 

2015: IFI operations in SEMED review of evaluation findings

EvD completed a review of evaluation material available from other organisations to assess the state of analysis and to extract findings that might be immediately useful to the EBRD.

After an initial review of 281 publically available documents, this paper reviews a total of 28 evaluations found to contain relevant and useful material and summarises their main insights and findings. Most are drawn from evaluations of private sector related operations completed between 2000 and 2014.

 
 

2012: Evaluation of the EBRD-Italy Western Balkans Local Enterprise Facility (LEF)

This study examines the extent to which LEF is fulfills its objectives and contributes to the Bank’s transition mandate. The framework was found to match the Bank's regional approach and country strategy priorities. The majority of individual projects were found to have met expectations of transition impact. It was found that the LEF approach is an effective model to implement higher risk smaller projects, due to its intensive origination and monitoring processes. Several recommendations were made including: to target higher level transition impact, avoid overlap with partner institutions, and to articulate the objectives and approach to new countries into the framework.
 
 

2012: Facility for medium-sized projects

This study, requested by the Board, represents the first full evaluation of the Facility for Medium-Sized Projects ("the Facility") which was approved in March 2008. The €140 million Facility provides equity and debt to corporate enterprises in countries not covered by other EBRD initiatives and targets subsidiaries of new international sponsors, regional cross-border investors and locally owned companies. The framework was appraised according to the OECD/DAC criteria, and concludes with a recommendation for a review of the various channels used to deliver investment products to small and medium sized enterprises.
 
 

2011: Russian regional banks

This examines the relationship with six Russian banks with regional networks. All six are privately owned and closely held (with a small number of major shareholders). The EBRD has equity stakes in five banks and provides capital support in the form of subordinated debt to three. Common objectives of cooperation included diversification of their funding base, lengthening of maturities and enhanced financial intermediation in the MSME sector.
 

2005: Delivery Mechanisms for MSME Financing

This special study outlines the development of the Bank's approach to the MSME sector in the transition countries and derives key generic lessons from PED evaluations of MSME operations.
 

2003: Russia Small Business Fund

Identifies key issues and puts forward recommendations aimed at helping the programme achieve its intended objectives and maximise transition impact.
 

2017: Small Business Support Programme

The Small Business Support Programme (SBS) has been the EBRD’s main programme to support development of small and medium enterprises since its inception in the mid-1990s. This evaluation focuses primarily on the relevance, effectiveness and efficiency of the 2011 to 2015 SBS Strategic Plan. The evaluation makes use of both qualitative and quantitative data through documentary review, interviews, portfolio analysis and country and thematic case studies.
 
 
 

2007: Business Advisory Services Programme

This study provides a mid-term review of the Business Advisory Services Programme. The report gives a thorough examination of the extent to which the programme is meeting its objectives and makes recommendations on measures to enhance programme performance.
 
 
 

2004: TurnAround Management Programme

A highly successful advisory facility by the Bank.
 
 
 

2016: Special Study Transactions with State Owned Enterprises

The EBRD has long conducted transactions with state-owned enterprises to encourage reform and transition, through a blend of finance and analytical and advisory work. This evaluation examines EBRD transactions with these enterprises in the power and energy and transport sectors over the period from 2000 to 2013, exploring questions about project rationale, progress towards privatisation and commercialisation, sectoral impact and potential marketplace distortion.

 
 

2016: Special Study Use of Subsidies Phase 1

The EBRD’s use of subsidies in projects has increased substantially over the past decade, on par with the overall increase of Bank business volume. Key drivers of growth include the financial crisis and the Bank’s expansion into new areas including climate change, clean energy and resource efficiency. The role of subsidies is likely to increase further under the Bank’s current strategic direction. Phase 1 of this EvD study maps subsidies approved between 2010 to 2014 by type, sector, geography and donor, and their consistency with Bank policies.

 
 

2016: Special Study Supply Chains and Backward Linkages

Bank projects often target transition impact by extending markets through direct or or indirect positive effects on supply chains and backward linkages. This study examines projects where this has been a key objective, assessing how efforts are integrated into project design and the results of this work. Several recommendations to strengthen the Bank's approach and activities in this area are made.

 
 

2014: The EBRD Shareholder Special Fund – interim evaluation

The Evaluation department has prepared an evaluation of the EBRD Shareholder Special Fund at Management’s request to assess specific issues of its operations and management. This is to feed into Management and Board strategic and operational planning work, including with respect to grants and donor support. The evaluation draws specific attention to important operational, planning and governance issues of the fund flowing largely from divergent views on its intended priorities and how best to operationalise them.

2013: TC experiences in new countries of operations - Mongolia Cooperation Fund

The Mongolia Cooperation Fund was a multi-donor fund established in 2001 to finance technical cooperation activities in a then new member country prior to its eligibility for regular EBRD operations. This evaluation serves primarily to examine this experience with early use of technical cooperation funds, for potential findings and lessons applicable to subsequent new member countries.
 

2012: Synthesis paper- Findings and insights from technical cooperation evaluations

This study presents a consolidation of the knowledge and findings from past evaluations of technical cooperation initiatives. Between 2000 and 2010, the Evaluation department wrote 60 technical cooperation evaluations, producing 262 findings. Yet 92 per cent of the findings relate to processes for managing technical cooperation rather than to actual results, relevance to the mandate or future sustainability. This invites reflection on the process for both the design and management of initiatives, and the evaluation approach that has been employed in this area for more than a decade.
 
 
 

2011: FAO Framework Agreements

The cooperation between the Food and Agriculture Organisation of the United Nations (FAO), and the EBRD started soon after the creation of the Bank in 1991. This is an evaluation of the framework cooperation agreements to share costs of technical cooperation initiatives in joint FAO and EBRD countries of operations
 
 

2010: EBRD Shareholder's Special Fund

Established in April 2008 with the net income that the Bank achieved in the previous year, its function is to facilitate the Bank’s operations by financing technical cooperation (TC) measures and non-TC support, and supplementing the grant funds provided from donor organisations. This report looks at the functioning and appropriateness of the procedures set up for its implementation, on the features of its portfolio to date, and makes recommendations for the future.
 
 

2008: Japan-European Cooperation Fund mid-term review

An evaluation of the Japan-European Cooperation Fund, a bilateral trust fund that has contributed to a wide range of Bank projects. On account of the Fund's prolific activities in technical cooperation (TC) operations, this offers a comprehensive analysis of the role of TCs using representative case studies.
 
 
 

2005: Mongolia Co-operation Fund

Set up in 2001 in preparation of Mongolia becoming a new country of operation and supported by four existing bank member countries, provided financing for technical cooperation operation in various fields.
 
 
 
 

Project completion report assessments for technical cooperation projects

From 2006 to 2010, EvD annually assessed technical cooperation (TC) Project Completion Reports produced by the Official Co-financing Unit (OCU, now under the Technical Cooperation department). EvD also reviewed reports prepared by consultants and conducted interviews with operation leaders on TC projects. Findings were aggregated to highlight a number of lessons for future technical cooperation projects. This annual report was discontinued in 2011, and TC is now evaluated as part of the operation self-assessment, EvD validations and EvD operation evaluation process.