Methodology

The Evaluation department introduced a new performance rating methodology in 2016 which can more accurately capture and explain results. The new methodology applies to all project evaluations, including self-assessments, validations and full operation evaluations. Overall results are reported annually in the Annual Evaluation Review. The new system closely follows the Good Practice Standards established by the Evaluation Cooperation Group.

Every evaluation produced by the Evaluation department, including thematic studies, looks to identify objectives and results at three levels. These levels are identified as:

• Outputs - the products, capital goods and services which result from an operation
• Outcomes - the short and medium-term effects directly attributable to outputs
• Impacts - the positive or negative long-term effects to which an operation contributes, directly or indirectly, intended or unintended

Evaluation questions for operation evaluations are designed around the OECD Development Assistance Committee (DAC) Criteria: relevance, effectiveness, efficiency, impact and sustainability. For special studies, evaluation questions may relate to fulfillment of objectives, sector or country strategies, additionality and transition impact.

The information used in evaluations is collected from documentation including background document reviews, literature reviews and case studies as well as interaction with participants through interviews, focus groups, structured questionnaires, anonymous surveys and field inspections.

Overall performance is assessed based on the following main subcriteria:

• Relevance (includes strategic and design relevance and additionality)
• Results (includes outputs, outcomes and impacts)
• Efficiency (includes financial performance, profitability, Bank handling, consultants)