Indicators
An indicator provides evidence or signs which show that change has taken place. Good indicators should be clear and understandable to everyone involved. Indicators can either measure or count results, or can use words to describe how people feel about changes. Daily egg production, tree growth, number of girls attending school – are all examples of indicators.
Baselines
To measure impact effectively, you first need a ‘baseline’ or starting point to understand the situation before any activity is started. Without this knowledge it will be difficult to identify the impact. There are two kinds of indicators – quantitative indicators, which can be expressed in numbers, and qualitative indicators, which measure changes in attitude and behaviour. Examples of quantitative indicators include literacy rates, average household income, number of children in school, crop yields, number of meals a day, eggs per day. Qualitative indicators would include levels of participation or decision-making ability, for example.
Some common problems
Programmes that seek to raise awareness or educate people about their human rights may find it more difficult to work out baseline indicators. But just because changes in attitude cannot be measured in numbers, it does not mean they can be ignored. Qualitative data is very important in understanding people’s feelings and attitudes.
Encourage participation
If planning is carried out in a participatory way, and people are encouraged to share their observations and experiences, it will often be easier to identify relevant baseline information and indicators. Encourage participation in collecting information about indicators and measuring impact. It is very important that the right people own indicators, especially if it is their behaviour that is being measured.
Be flexible
Good planning is important, but development work doesn’t always go as planned (indeed, very often activities may change from the original plan). Don’t rely on just one or two indicators. Instead, choose a range of indicators that are more likely to capture the full range of changes taking place. Indicators should help people understand the processes taking place and encourage a sense of ownership in a programme.
Involving the community
Ranking Useful for comparing both measures and feelings. For example number of pumpkins harvested weighing less than 2kg, between 2kg and 2.5kg, and over 2.5kg.
A ranking exercise using stomach sizes!
Counting A simple way to count a range of answers is to use a tally – which simply bundles together five responses. 28 responses would be recorded like this:
Listening Take time to listen to what people think about their lives. Find out what is important to them, what their hopes are, what they believe is important. Open-ended questions beginning with ‘how’ or ‘why’, and a listening ear are very valuable in measuring impact!
Mapping Encourage people to draw a map of their local area to explain changes.
Percentages Reporting that 5 people out of 28 thought the development worker useful during a baseline survey and that 9 people out of 33 thought him useful two years later, doesn’t really make the impact very clear. Percentages make it easy to compare different figures:
5/28 x 100 = 500/28 = 17.85 (round up to 18%)
9/33 x 100 = 900/33 = 27.27 (round down to 27%)
(NB If a figure ends with .5 or higher, round up: if less than .5, round down.)
Now you know that after two years the number of people who found the development worker useful increased from 18% to 27%. Try working out the remaining figures from the table below for practice!
Always remember that you need the total number of people to work out a percentage. If you are measuring literacy rates and 21 people in a meeting say they are literate in French, this figure is of no value unless you also count the total people present. Twenty-one out of 38 people gives a literacy rate of 55%.
Coding Asking people what they think about things may result in all kinds of fascinating information that nobody knows what to do with! One way of measuring attitudes is to ‘code’ a range of attitudes. For example, people could be asked to consider what they think of a development worker from a nearby NGO. They could be asked to select one of the following answers:
very helpful, useful, OK, not very useful, unhelpful.
A few relevant quotes could then be added to highlight particular points. Coding can also be done after information is gathered, if appropriate – but this may take longer to analyse.
|
Attitudes to worker |
very useful |
useful |
OK |
not very useful |
unhelpful |
TOTAL |
|
Baseline survey 2001 |
2 |
5 |
6 |
8 |
7 |
28 |
|
Assessment
2003 |
11 |
9 |
9 |
4 |
0 |
33 |
Matrix Useful when comparing different activities or responses. For example, members of Tuliwalale Women’s Group, Uganda, were asked about the information they received from various sources about three new agricultural techniques. They were asked to rank how useful the information was on a scale of 0 (very poor) to 5 (excellent). They thought the NGO, SAFA, gave the most useful information. Less information was available about new crop varieties.
|
Source |
New techniques |
|
|
Modern farming |
Improved crop varieties |
Broiler chickens |
TOTAL |
|
women’s groups |
1 |
1 |
0 |
2 |
|
radio |
3 |
1 |
4 |
8 |
|
husbands |
2 |
0 |
2 |
4 |
|
SAFA NGO |
5 |
5 |
5 |
15 |
|
TOTAL |
11 |
7 |
11 |
|
Be sensitive Be careful in choosing who should interview and discuss change with community members. Sometimes this is best done by outsiders who are not worried about losing their jobs if a programme has not been successful. But if outsiders are used they must be sensitive and put people at ease.