Organisation: Hoffnungszeichen | Sign of Hope e.V. www.hoffnungszeichen.de: www.sign-of-hope.org

Project: Improving food security and increasing resilience to climate change impacts of smallholder farmers in Mirab Abaya, Southern Peoples, Nations and Nationalities Region, Ethiopia

Position Type: Short-Term Consultancy

Study/Assessment Topic: Midterm evaluation

Position Location: Ethiopia

Reporting To: Head of Department, International Aid and Cooperation

Working With: Spiritan Community OutReach Ethiopia (SCORE)

Starting Date: 15.07.2021

Application Deadline: 30.06.2021

Background and Rationale: Hoffnungszeichen | Sign of Hope e.V. is a Christian motivated organisation for human rights, humanitarian aid and development cooperation. The organisation is headquartered in Konstance, Germany with East Africa Branch Office in Nairobi, Kenya.

Spiritan Community OutReach Ethiopia (SCORE) and Hoffnungszeichen | Sign of Hope e.V. are currently implementing the project “Improving food security and increasing resilience to climate change impacts of smallholder farmers in Mirab Abaya, Southern Peoples, Nations & Nationalities Region, Ethiopia” since January 2019. This is a 5 year EUR 612,898 project, which started on 01.01.2019 and is expected to end on 31.12.2023. The German Ministry for Economic Cooperation and Development (BMZ) is co-funding 75% of the project.

The project’s goal is to improve food security of 368 disadvantaged smallholder families in the Mirab Abaya district and ensure that they have built up year-round resilience to periods of hunger. It does so by implementing activities that ensure that these smallholder households use drought-resistant crops, make efficient use of rehabilitated farmland and use soil-friendly cultivation methods. The target households are expected to use available water to increase the yield of their fields, and so bridge periods of hunger and further invest their savings in improved food intake. 2,576 people in four Kebeles of Yaykie, Fetele Doronje, Kolla Barenna and are direct beneficiaries of the project. It is expected that the ripple effect of the project will benefit the whole population (11,072 people) of the target Kebeles indirectly.

2 below is a summary of the outputs and activities of this project:-

Project Outputs & Activities

Output 1: Improvement of agricultural production and productivity of farmers

Construction of rainwater harvesting ponds; promotion of drought resistant seeds; construction of diffused light storage constructions; supporting improved livestock farming; promotion of poultry production; promotion of sheep and goat farming; promotion of dairy farming; construction of two treatment stands for cattle; distribution of multipurpose seeds & cuttings (soil protection and fodder); growing of useful trees; construction and renaturation of simple Natural Resource Management (NRM) structures; integration of schools’ health (nutrition) programme including establishment of school clubs and training for nutrition and health teachers, having educational campaigns for parents and development of educational material in local language.

Output 2: Water protection measures to increase agricultural productivity

An assessment of water pipe damage and repair of drinking water system; rehabilitation and extension of water pipes and construction of water intake points; strengthening water user associations; and construction of a drinking water reservoir.

Output 3: Alternative income promotion in small businesses

Basic business skills training for 140 young adults; promotion of small business promotion; promotion of commercial fattening livestock farming; promotion of commercial poultry farming, beekeeping groups and women self-help groups.

Purpose, Objectives and Use:

The purpose of the feasibility study is to provide Hoffnungszeichen | Sign of Hope e.V. and SCORE with a comprehensive assessment, conducted systematically and impartially, of the project, as well as analysis of the achievement of its both expected and unexpected results. The study will provide concrete, credible and practical recommendations that will assist with the timely incorporation of the findings into the decision-making processes of both organisations (adaptive management). The study will support the project quality assurance measures implemented by Hoffnungszeichen | Sign of Hope e.V.

The objectives of the midterm evaluation is to:

1. To assess whether the implemented measures are showing the expected results in order to reach the projected outcomes. A baseline study conducted in the project area at the beginning of the project as well as human interest stories and case studies, field visits and monitoring reports by Hoffnungszeichen | Sign of Hope e.V. and/or SCORE staff, quarterly and annual reports may serve as a point of reference for the evaluation.
2. To determine the pathway of change (qualitative and quantitative) in each intervention supported by the project.
3. To determine the effects of the COVID-19 pandemic on the project implementation.
4. To analyse the approaches used by the project team in the implementation of the project to determine what is working or not working well to inform the rest of the project life.
5. To determine sustainability measures put in place or planned by the project team to enable continuity of key project interventions on exit of project teams.
6. To provide recommendations on needed project revision aspects with focus on:
   • The measurability and appropriateness of some of the indicators (the consultant to propose the most appropriate indicators).
   • The project strategies, interventions and target groups among others.

The results of the evaluation should outline the progress of the project according to the given indicators set out in the log frame (see attachment). In addition, the evaluation will identify risks and make recommendations for the completion of the project as appropriate.

Scope of Work: Methods can include collection and analysis of primary and secondary data (both qualitative and quantitative). These methods can include desk review of available information sources, focused group discussions, key informant interviews, transect walks, observations and household survey among others. Furthermore, the consultant(s) should explain the methods that will be used to ensure quality of data collected, including but not limited to the process of selection, management and training of enumerators, data collection tools and platforms and piloting of the study.

The study will be guided by both the objectives of the evaluation and the OECD DAC evaluation criteria of relevance, coherence, effectiveness, efficiency, sustainability and impact (please see https://www.oecd.org/dac/evaluation/revised-evaluation-criteria-dec2019.pdf).

Guiding Evaluation Questions:

• What is the current cumulative value of the targets of the log frame indicators (Impact, outcome, output and process level) since the project start?
• How do the current levels of achievement of the target indicators compare with the baseline survey and the set targets?
• Do the indicators levels differ across gender and Kebeles?
• What is the level of resilience amongst the targeted beneficiaries?
• What effect did Covid-19 have to the organization?
• What effect did Covid-19 have to the project and its activities?
• How did the project adapt or not adapt to Covid-19 outbreak?
• What are current approaches being used in the project?
• What is working well with the approaches?
• What is not working well and what would be the likely solutions?
• What measure are planned or are ongoing to ensure financial, institutional, environmental, technical and social sustainability of the project?
• What are the recommended project revision or changes for future implementation?

Study Quality and Ethical Standards

The study team shall take all reasonable steps to ensure that the study is designed and conducted to respect and protect the rights and welfare of the people and communities involved. Furthermore, the study should be technically accurate and reliable, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. More specifically, the study should be aligned to the following standards and applicable practices:

Utility: Study must be useful and used.

Feasibility: Study must be realistic, diplomatic, and managed in a sensible, cost effective manner.

Ethics and Legality: Study must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the Study.

Impartiality and Independence; Study should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.

Transparency: Study activities should reflect an attitude of openness and transparency.

Accuracy: Study should be technically accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.

Participation: Stakeholders should be consulted and meaningfully involved in the study process when feasible and appropriate.
Collaboration: Collaboration between key operating partners in the study process improves the legitimacy and utility of the study.

The study must be in line with Hoffnungszeichen | Sign of Hope e.V. guiding principles and Code of Conduct.

Process

Phase 1
The evaluator will be provided with all the relevant project related documents to get familiarised with the project objectives and activities. There will be a briefing in SCORE’s head office in Addis Ababa before the site visit to the project location and the consultant(s) will design the evaluation specifics afterwards.
Phase 2
A site visit to the project location will be carried out to gather information and data from the beneficiaries and relevant stakeholders. Accommodation and travel can be logistically supported by SCORE’s staff, but may be carried out individually by the consultant(s) without being accompanied by project staff. During the site visit, the field office staff will facilitate access to the target group and logistics.
Phase 3
Analyzing the collected data and preparing an evaluation report.
Phase 4 Presenting the findings and draft report to the implementing organizations. Receiving organization’s feedback and producing a final version including all the contributions.

Tentative Timetable The proposed timeframe is four (4) weeks starting 05.07.2021. A total of 20 working days can be budgeted. The consultant shall submit a technical and financial proposal including study design, data collection tools, timeframe and an estimation of costs.
Days & Activity

1 day -Inception meeting, handover of relevant documents

4 days – Initial documents review, inception report, data collection tools and identification of enumerators

2 days-Training of enumerators and pre-test of tools

5 days- Field data collection

5 days- Data analysis and draft midterm evaluation report sharing

3 days Review of feedback and final report submission and dissemination

Expected Deliverables

1. Inception report detailing the study design, study tools, agreed budget and work plan.
2. Draft report that will culminate in the final report.
3. A power point presentation highlighting key findings from the midterm evaluation will be presented at a feedback meeting to be held after completing the draft report.
4. Final midterm evaluation report submitted by the agreed deadline and which should include at least the following components: Critical analysis of indicators:
   • Display a log frame with the gathered data / results
   • Verifiable derivation of the published results for each indicator (methodology, sources, definition, calculation)- – Assessment of the implications for the impact of the project / reasons in case of deviations from the target5 
5. A realistic estimation on achievement of the project indicators/target as displayed in the project agreement
6. Reporting on the OECD DAC Criteria
7. Gaps and challenges
8. Recommendations for improvement / further follow up
9. Capacity analysis of the implementing organizations
10. Copies of original and cleaned data sets

The language of the report is English. A standard format for the structure of the final report will be provided by Hoffnungszeichen | Sign of Hope e.V. Working languages in the field are Amharic, Gamotho and/or English. In case there is a need for translation, this will be discussed with SCORE to provide persons who can provide this service.

Qualifications and Experience

Essential:

• Professional knowledge of Amharic and English both spoken and written
• Technical knowledge in relevant areas, including WASH, food security/sustainable agriculture or livelihoods in Ethiopia or in East Africa in general
• Proven experience in evaluating projects in international development sector
• Experience in both qualitative and quantitative data analysis method

Desirable:

• Knowledge of the Core Humanitarian Standard on Quality and Accountability (CHS) or SPHERE standards
• Experience of thematically related projects in Ethiopia
• Demonstrable experience in conducting similar studies for BMZ-funded projects

Academic Background

An advanced level university degree (Masters’ degree or equivalent) or first-level university degree with over 7 years of experience in social sciences, disaster management, humanitarian aid, public health, international development studies, law and/or relevant fields. The consultant(s) must be independent and impartial with no liabilities to any organization or government entity regarding this evaluation.