The real value of metrics in social action

What is the real value of metrics in measuring the impact of social action? There is no doubt that monitoring and evaluation play a crucial role in assessing the real value of what social purpose organizations, either for profit or not for profit, are trying to achieve.

Metrics comes out of a  renewed attention at ensuring that every dollar provided by a donor or impact investor can be “value for money”. Indeed frankly speaking, we know that there still are huge margins of improvement when we talk about effectiveness and efficiency of development. Metrics can help getting rid of what does not work or work that is not efficient.

Indeed the art of measuring has become increasingly more complicated. There are many different approaches including the ones attempting at “monetizing” the impact of social actions. This means that through complex formulas we can predict, project and compare different actions in terms of the economic value they generate.

For example you can understand what the financial return of  short term vocational training is in relation to a more traditional diploma; what is the median income of a disadvantaged student attending a short term vocational training program versus the income projection of a similar beneficiary attending a more traditional learning pathway? Would an apprenticeship program be of more value? This is just an example. Simplifying a bit, one can name an action with a supposed clear outcome and you can figure out its financial value.

Is it that simple? Is it real? Is it really necessary and indispensible to reach this level of metrics when we look at the impact of development work?

A few years ago moved by a strong interest to understand more about the craziness of metrics, I purchased a book written by senior officers at the Robin Hood Foundation, a massive not for profit set up by successful(read millionaires) businessmen working in the finance sector. The book The Robin Hood Rules for Smart Giving offers valuable insights on “relentless monetization” that relies on modern economics techniques, including cost-benefit analysis to understand which programs have more of a chance to succeed.

The foundation has become famous for the outstanding impact of its grants in New York City where it operates. The nonprofit outfit is also renowned for its aggressive approach to metrics when it comes to evaluation of its grants. You can believe me that anytime I find the courage to open this book, I feel like I need a PHD in math or statistics.

Are we going a little too far in terms of monitoring and evaluation? There is no doubt that it is attractive and useful to be able to quantify, in financial terms, what NGOs, charities or social entrepreneurs are doing.

Is this the only way? What about the value of more traditional, simpler approaches?

Scientifically proven methodologies to measure social or economic impact are certainly welcome but we also need to be realistic in terms of practicality of adopting “extreme” approaches.

In April Michael Bloomberg, the former Mayor of New York City and a real innovator in the field of philanthropy (he is also a billionaire) in an interview with the New York Times renewed his strong belief in innovating and taking risk by trying new approaches but warns about micromanaging and refrains from too complex forms of monitoring and evaluation. In short he doubted whether they are really good and play that huge of a positive role. In the same article, Leslie Lenkowsky, professor of philanthropy at Indiana University School of Public and Environmental Affairs said “Metrics certainly enable you to try to distinguish between good intentions and good results. But they are very difficult to do, and they can be carried to excess.”

We needed someone authoritative like Bloomberg to state something bold against the current mainstream school of thought. When I read the interview, I really felt relieved. I was wondering how small NGOs doing certainly good things can adopt such methodologies to prove their achievements.

In the recent Asian Pacific Venture Philanthropy Conference held in Singapore last May, the issue of monitoring and evaluation emerged strongly. There was  general agreement that at the end of the day what we need is common sense and huge doses of practicality. Extremely complicated methods also have shortcomings, might not be practical and might also be wrong or unrealistic.

Metrics while important are not everything. Hopefully organizations like the Robin Hood Foundation will be interested in undertaking studies to simplify metrics and make them accessible for all. This would be real value for money.

Ps

If you are a math or statistics graduate, Robin Hood might hunt for you. But after all, best of luck also if you are, more unassumingly, from a social science background. There are still plenty of organizations where you can make a difference, with or without metrics!