|
|  |
Seeing Results – a day with the external evaluation team by John Morris
 External evaluation is a process that objectively assesses if a project meets its goals. It creates an opportunity to learn from an affected community the true outcome of any assistance that was provided
My second day’s evaluation work in Kaniyakumari District, at the southern tip of India, started very badly indeed. My wake-up call was sent to the wrong room and, when I eventually woke up, it seemed unlikely that I would make it to breakfast in time. I phoned the dining room manager and tried to organise a very quick breakfast.
Markus Muntwiler, Head of Mission and Development for The Salvation Army’s Switzerland, Austria and Hungary Territory, says:
External evaluation showed how challenges, changes, decisions and management in general affect a project in its implementation process and finally its achievement, its quality and the impact on the beneficiaries. |  | | (From left) local evaluator Dr Johnson Raj and John Morris talk with Territorial Tsunami Coordinator Major Jeya Sekar |  | | Skills training in Kadiapattinam, India |  | | New housing in Kaniyakumari |  | | The new fishing fleet in Kaniyakumari, India |
|
‘I am late!’ I told the manager.
‘Omelette sir?’ came the reply
‘No,’ I said, raising my voice in frustration, ‘I ... am … late, I … am … late!’
‘Yes sir, two omelettes,’ came the calm reply. ‘What kind of omelette, sir?’
I eventually managed to grab a quick breakfast with the rest of the team (no eggs in sight I hasten to add) and we set off pretty much on time. I just had to hope that the day would get better!
The eight of us boarded our vehicle and headed to one of the communities where a Salvation Army project had taken place. Among those with me in the car were three independent Indian consultants, all of whom had undertaken evaluations before: an engineer who was very experienced in community housing, a community development economist who had done survey work on tsunami damage in the two villages covered by the Salvation Army project and a social development specialist.
Also on the team were the Salvation Army’s Tsunami Coordinator from International Headquarters, the local (Tamil Nadu) Salvation Army Tsunami Coordinator, and two of the project implementing team. In all we were six men and two women.
We drove out to Kadiapattinam village to talk to some of those who were expected to have benefited from the project, to hear what they thought of the quality and appropriateness of the Salvation Army’s support and assistance since that fateful day in December 2004.
Our aim was to hear from as many people and representative groups from the community as possible during the evaluation of the project. We wanted to be as inclusive as possible.
The Roman Catholic church was our first stop and we enjoyed an introductory meeting with the Roman Catholic father of the predominantly Catholic community in Kadiapattinam. Following our meeting there – and the very welcome offer of a cooked lunch – we set off to find the owners of the houses and fishing boats provided by the project. We used a random numbers process to identify the people we would speak to – about 40 per cent of those assisted by the project – from lists held in the project office files we had looked through the previous day.
Although these lists contained both names and addresses, there had recently been a house renumbering exercise undertaken in the village which rather complicated matters. Still, with the help of some village council members, by the end of the day we managed to find everyone we needed. In my experience of disaster response evaluation this in itself was an impressive result!
Each person we spoke to was able to verify that their house, boat or fishing equipment had been destroyed by the tsunami wave which ‘came without warning and was as high as the palm trees’. We then asked for their views on the items they had received and, just as importantly, on The Salvation Army’s work and relationship with the community as a whole. Evaluations don’t just look at the number of houses built or items distributed, they also verify whether people’s lives have changed because of the support offered.
After about six hours visiting people in their homes we all met back at the community hall and discussed our main findings with a group of community and church leaders. A final cup of tea together and then we started our journey back to the hotel.
After an evening meal (ordered in Tamil by one of the other evaluation team members so as to avoid the confusion which started my day!) we worked at tabulating and bringing together our findings from the day.
We were all greatly encouraged by what people had said to us. People who had lost everything except the clothes they were wearing had been given emergency food and accommodation and had been helped to restore their homes and livelihoods. Even more, The Salvation Army had helped people to become part of the community again.
Dedicated Salvation Army staff had met human needs without discrimination as in their mission statement. They had also given the people hope.
The community, the Roman Catholic church, the project team and the donors – The Salvation Army and Tearfund – could be rightly proud of the work they had done together.
A final review and slight adjustment of our questionnaires and methodology for the next morning brought the long but rewarding day to a close.
And so I settled down to sleep reflecting on the day’s events – but not before I had set the alarm on my mobile phone to wake me the next morning. I had learned my lesson!
John Morris is a development consultant and evaluation specialist who undertook an evaluation of The Salvation Army’s community development projects worldwide
|