What does what we discussed in last tuesday’s post: Indicators – Curse or Blessing? tell us about urban policies? First, that framing the objective is the key to good implementation. When we frame the objective in terms of being healthy we open up more options than when we focus on measuring losing weight. Second, that we need to focus on results first and indicators second. Don’t let someone tell you that something can’t be measured (for example organisational change or social capital). The same was said of quality of life, wellbeing and happiness which are now being touted as replacements for Gross Domestic Product (GDP).
Interference and Attribution
The difficulty with some of these more slippery concepts is not the basic measurement but the measurement of Tako’s notion of interference. We need to identify which results were caused by our actions and which by those of others. This is called attribution. For an URBACT local action plan you can also examine the question at two levels. You can ask what has the Local Support Group (LSG) achieved and then what will the Local Action Plan (LAP) achieve. Remember that the LAP itself is mostly funded by third party funds outside of the URBACT programme. But there is all the difference in the world between an intelligently designed LAP and a badly designed one. The difference is seen not in the outputs which might be the same (for example sq. metres of workspace) but in what this impact delivers for the city (perhaps high quality jobs).
I sat for 6 years on the board of a Regional Development Agency in England’s East Midlands. We had as one of our objectives to grow the GDP of our region. Every year the GDP went up and every year our Chief executive and Chairman (for it was a man) praised our RDA for its achievements at the annual meeting. I had left by the time of the collapse of Lehman brothers and various British banks and the onset of the crisis but you can be sure that when GDP went into reverse they argued that international and national forces were the cause (just as they always had been during the boom). Of course disentangling the additional effects of a small agency with a budget of about 1% of regional GDP or 250m euros per year from the shifts in a regional economy covering 4million people worth 150 billion euros per year is difficult. But for this very reason it is better to focus on a more meaningful result indicator than GDP. The Commission evaluation unit will tell you that the objective and its result indicator always needs to be more specific.
Subsidiarity for Indicators
So where does this leave us for URBACT networks? First of all the indicators must be developed at the lowest appropriate level. Yes, subsidiarity for indicators – for most local support groups it will be their discussion of what success will look like that should focus the indicator. Second, the nature of the indicators will depend on the topic or theme of the network and the cities involved. The important thing is to ask the right questions. The best result question is to ask your local support group: ‘What is the changed situation in the future that you wish to achieve? How will you measure that you have achieved it? These are powerful questions which speak to a future that we can control and manage. Better still you can ask them without invoking the word ‘indicator’.
Tangible Achievements and Raison d’Etre
Discussion in our meeting on how to do this in real networks showed just how specific is the problematic of each network. Making tangible achievements on Roma inclusion might mean getting Roma questions into mainstream policy documents. It could mean that local press reporting is more balanced. Or it could mean that local organisations involve both Roma and non-Roma people. Over the medium term the result of the local action plan could be to reduce disparities between Roma communities and everyone else. But this needs to be seen as a long term goal which is the very raison d’etre of the local support groups.
Electrical Vehicle in Urban Europe (EVUE) is a network looking at how to introduce electric vehicles in the city. Most of the LAPs have focused on the immediate task of installing charging points and other electric vehicle infrastructure. Westminster have focused on commercial vehicles such as delivery trucks and taxis which because they are diesel are responsible for much of the pollution and poor air quality in central London. Of course such an output only makes sense if the charging points are well used. Lead Partner Matthew Noon suggested that the result of the URBACT network might be that partner cities are using health arguments around particulate emissions to promote clean forms of transport such as electric vehicles. The ultimate result of action plans might be to reduce polluting emission, increase modal share of electric vehicles – especially in commercial uses.
Different Versions of Success
One side effect of smart phones and taxi apps like Hailo could be to reduce the amount of time that taxis spend on the road plying for passengers. This is estimated at 30% of kilometres driven by taxis in London. Reducing the wasted tons of non-productive emissions could be as useful and more immediate as converting 100000 taxis to electric – itself a worthwhile task but one that may take 30 years to achieve because of the long life cycle of the taxi. Specifying the objective well allows different versions of success to become possible. In contrast, focusing on narrow output indicators such as ‘more electric cars’ might work against alternative solutions. Using the health argument is also a reason for resisting the taxi drivers who are defending their right to pollute by not allowing us to have a taxi app on our smartphones. The mobility workstream found that a good LSG process which really looks at the right questions can end up with more intelligent, more future oriented and cheaper solutions. For example this might lead to urban re-design rather than a much more expensive new road.
Using Composite Indicators
Measuring integration is a perennial question. It is an intrinsically difficult thing to measure. It also raises the question of which stakeholders should be concerned about integration? Is it a topic for everyone in the LSG or is it something that the city itself should monitor? How to measure integration is a second challenge. There is no hard and fast technique but it is possible to break integration down into a series of statements which taken together produce a composite indicator that tells us how well we are doing. The Local Action plan self-assessment tool tries to do this. Perhaps in the future the Reference Framework for Sustainable Cities (RFSC) which is being taken under URBACT’s wing could also be a way of understanding our level of integration. The RFSC contains 280 indicators developed for use by cities.
Using results in this way can drive better policy making and inspire innovations in delivery which may take us in new directions. We need to make indicators intrinsic to our work instead of extrinsic. It follows that city practitioners should develop and own their indicators. Good indicators can become an enabler rather than a strait jacket. Most of all it is evident that real indicators are created at the level of the city by the stake holders that are round the table. They are not determined at network level nor at programme level. As Ann Morton Hyde (lead expert for Romanet) pointed out ‘this is a process’. Her concern was whether there are enough resources for the lead expert to do this work. This is clearly a matter of priorities and there may be difficulties for a shorter delivery network which is now half way into its timeframe. However, in a new network a key task of the expert in the development phase will be for the expert to discuss and even negotiate with the new local support group their first version of the result that they intend to achieve through their action plan. This may have to be revised during the drafting of the local action plan as reality sets in.
The Emperor’s New Clothes?
Alison Partridge (lead expert for Economic Strategies and Innovations in MEdium-sized Cities, ESIMEC) asked whether this is an example of the ‘Emperor’s New Clothes’. Is the new framework just a revamp of the terminology using new words (result instead of impact) that does not really change anything on the ground? I would argue that the new framework, if used properly, and devolved to the local support groups will radically improve practice on the ground. If my assertion that more cities know their outputs than know their results is true and widespread then there is plenty of room for improvement. The new approach gets us out of the output strait-jacket and allows for innovation in how to achieve the result. Many will say that they already have ‘key performance indicators’ (or KPIs). But most KPIs are efficiency indicators (measuring the output per input ratio) and as we know from history being more efficient about doing the wrong actions does not always produce a good result.
URBACT will need to design and report on indicators at programme level and sometimes bother the networks to collect them, but your city level indicators need to be specific and local, not generic and European. There will be URBACT programme level indicators, which the work of LSGs will contribute to but this is a story for another day.
By Peter Ramsden, URBACT Thematic Pole Manager. Find him on Twitter!
