How we assess transition impact

The three key principles governing EBRD operations – as set out in Article I of the “Agreement establishing the European Bank for Reconstruction and Development- are: sound banking, transition impact (TI) and additionality.

Article I of the Agreement Establishing the Bank states that “the purpose of the EBRD shall be to foster the transition towards open market-oriented economies and to promote private and entrepreneurial initiative”.

The Concept of Transition was first defined in 1997 and laid down a framework for the systematic assessment of projects in light of the Bank’s mandate. Initially, the focus was on how projects contribute to promoting “Economic interactions on a market basis and private initiative.”

Over time, the TI methodology has evolved and adapted to keep pace with new challenges facing the economies where the EBRD invests. In 2016, a new transition assessment framework was introduced. The framework is more suited to the current challenges in our expanded regions of operation, such as the Southern and Eastern Mediterranean region.

How we assessed transition impact before 2016

The EBRD’s new transition concept argues that a well-functioning and sustainable market economy should be characterised by six key transition qualities, i.e. it is an economy that is Competitive, Well-governed, Green, Inclusive, Resilient and Integrated.

This concept has been operationalised throughout the Bank’s Results Framework, which covers the Bank’s country strategies as well as policy dialogue and project activities. At the project level, we developed a methodology to assess the expected transition impact project (TOMS) and measure its actual delivery (TIMS).

The Sector Economics Pillar (SEP) in the Economics, Policy and Governance (EPG) department of EBRD, in conjunction with project teams, assesses and monitors the Transition Impact of EBRD projects in line with the TOMS and TIMS methodologies.

Transition impact analysis

For each EBRD investment, the SEP team assesses how the project contributes to the Bank’s mandate to promote transition. Projects are scored by looking at the two main transition qualities they contribute to, but all are reviewed along the six qualities to make sure there are no elements that could potentially produce negative impact on the other qualities.

Monitoring and the corresponding attribution of a portfolio score is also structured along the six qualities and is based on set of standardised TI monitoring indicators.

Over the past years, EPG developed a tool known as TOMS to streamline and automatise the TI assessment at origination, and increase the transparency and predictability of TI ratings.  The following section describes the core features of the application.

Transition Objectives Measurement System (TOMS)

TOMS identifies and assesses the TI of the Bank’s projects, and ensures that projects are aligned with the Bank’s mandate and carry sufficient value in terms of achieving transition. Besides being based on the “six qualities framework”, it also relies on the results of the Assessment of Transition Qualities (ATQs, gap analysis) and of the Country Strategies (CS).

A typical TOMS project submission and assessment workflow consists of three steps:

(i) identifying the general features of a project and whether it requires enhanced scrutiny,

(ii) determining the ambition of the project with respect to the transition mandate and attributing a score, and

(iii) committing on the delivery of outcomes/outputs in line with the stated ambition.

In the case of frameworks (bundles of projects), a rating is assigned for the framework based on the tool, and it applies automatically to corresponding sub-operations, assuming they match the framework criteria. 

Expected Transition Impact (ETI) scoring:

ETI scoring reflects both the intrinsic value of a project and its contextual value. Selected TI objectives all carry a number that enters the calculation of the score, but such a score is then adjusted to reflect country context and support of Bank-wide strategic initiatives.

The country context adjustment is based on a gap analysis along the six qualities of transition. If a country is relatively less advanced for a given quality, its projects in that quality will receive a positive adjustment. Conversely, the same objectives in more advanced countries will yield lower ETI scores.

Strategic relevance is defined by compliance/fit of the project with Country Strategies priorities and the use of financing instruments that are specifically supported by the Bank (equity and local currency lending).

ETI is scored between 0 and 100.

The bulk of projects fall between 60 and 70 and are considered to have a Good Transition Impact. Projects between 70 and 80 are strong projects and there are only several cases per year – deemed excellent – that receive a score higher than 90.

Transition Impact Monitoring System (TIMS)

TIMS is designed to fulfill the commitments made at appraisal and uphold the EBRD’s mission to deliver the highest possible transition impact. The system spans the time period beginning one year after signing through the end of an investment project, that is the time of full repayment of a loan or exit from an equity investment.

TIMS serves two key functions at the EBRD: (i) as a mechanism to monitor progress against transition objectives during project implementation and to facilitate remedial actions; and (ii) as a basis to assess and report the Bank’s transition performance and results achieved as well as lessons learned.

On the back of the TOMS implementation the EBRD upgraded its monitoring application in order to align the appraisal and monitoring methodology. Therefore, TIMS mirrors the six-transition qualities approach so that monitoring and reporting arebased on an updated and consistent results framework when measuring and communicating on impact.

Portfolio Transition Impact (PTI) scoring:

The TIMS system allows the calculation of a Portfolio Transition Impact (PTI) score for each project, which provides incentives to:

• set ambitious commitments at origination, while taking into account the probability of delivery, and
• manage the delivery of TI during project implementation and monitoring, to maximise the portfolio’s overall performance.

Projects are monitored at regular intervals (e.g. annually) up until completion/repayment of the project or the time when all monitoring indicators have had a final assessment. The system will also accommodate monitoring of project frameworks at regular intervals.

There will be reporting in two phases: (i) monitoring phase (for ongoing projects), and (ii) evaluation phase (for finalised projects).

The first step of the overall approach under the new monitoring methodology is to assess performance under each TI monitoring indicator (or benchmark) on a percentage delivered basis – in automated fashion for quantitative indicators and through self-assessment by operation leaders for qualitative indicators.

Once each indicator is scored for its delivery rate, the scores are aggregated at the project level, yielding a percentage-based score that shows how much of the ETI has been achieved or is on track to be achieved at a given year. This score is also used to compute PTI rating of a project through a pre-determined formula.