Insights from the webinar The hard challenge of assessing transformative impact in urban regeneration
On January 20, Centrinno, Hub-In and T-Factor organized the webinar The hard challenge of assessing transformative impact in urban regeneration, aimed at exploring common challenges on impact assessment in urban regeneration projects and insights to face them.
Even though each project works from a different perspective, the 3 Horizon projects, CENTRINNO, HUB-IN and T-Factor, share a common mission: developing strategies and approaches to transformative urban regeneration that unleash entrepreneurship, creativity and social inclusion. In 2021, the 3 sister projects released a common manifesto with the values and principles we share, and a road map to exchange learnings, methods and tools.
As part of this road map, the three Horizon 2020 projects hosted in March 2022 a first event focused on impact assessment in urban regeneration, Regenerating cities for the post-pandemic: what counts for transformative impact?. The new webinar hosted in January 20 was aimed at going deeper into this topic, exploring the challenges related to impact assessment from a practical approach and sharing key learnings gained along the way.
Maro Pantazidou, How does transformative change happen?
The first speaker of the webinar was Maro Pantazidou, PhD Researcher Candidate at University of York and former deputy director of Global Impact and Learning at Amnesty International. As she explained at the beginning of her speech, the question that should motivate any methodology for impact assessment should be “how does transformative change happen and how do we know?”, because all of us have different conceptions on what transformative change actually means.
As Maro Pantazidou explained, organisations share common problems when it comes to measuring impact:
- Tensions in teams because of the lack of clarity on how we think change may happen and where we should put our resources and motivation.
- Increased awareness: the difficulty of converting this “beautiful vision on the top of the mountain” into practice.
- Vanity metrics: when we focus mainly on quantitative metrics to show that we are doing well (number of followers on social media, number of press releases sent,…) This data is necessary, but those numbers can hardly tell something about deep change.
Maro Pantazidou shared some of the lessons learned from Amnesty International Impact Framework. We have to ask:
- What kind of intermediate changes do we want to put in motion?
- As change happens through the shift in attitudes and behaviors of specific actors, which types of actors are we targeting?
- What progress markers will we use to measure change?
- What kind of interventions contribute to what kind of outcomes?
Impact framework is about aligning activities to vision, knowing why we are doing what we are doing, knowing what “doing well” towards your goal might look like, and being sure that what you are measuring matters. But, among all, Pantazidou highlighted that it is a learning process that should always be participatory.
Alex Pazaitis, Centrinno
The Horizon 2020 Project CENTRINNO focuses on regenerating industrial sites into hubs of local productions and citizens-led economies, exploring how initiatives such as fab labs and innovation hubs can be drivers of these novel models of economies. In the webinar, Alex Pazaitis explained the challenges they face related to measuring impact and some of their key learnings along the way.
CENTRINNO project mobilizes transformative change through 3 steps:
- Mapping the local ecosystem
- Building a community around it
- Implementing Fab City Hubs
Pazaitis shared three “hot potatoes” related to impact assessment that are common in urban regeneration projects, and how they face them:
Getting “thick” with data: Data alone doesn’t tell the whole story. So, they develop stories that help organise this data in a meaningful manner, and then, turn these stories into practices that can be transferred.
Tension between control and emergence: For the change we want to happen, we have to let the boundering actors to self organise, and it means losing control. But, at the same time, we need some control to monitor and report, so we have to find the balance in between.
The hubs they are a-changin’: The problem with monitoring change is that you can not plan ahead. So, instead of measuring the change itself, CENTRINNO team focuses on the capacities, how people developed new skills, or got access to new spaces, infrastructures and resources.
Chris Taylor, HUB-IN
Horizon 2020 HUB-IN project focuses on promoting the urban transformation and regeneration of historic urban areas using as the main catalyst innovation and entrepreneurship while preserving the unique identity regarding cultural and social values.
As Chris Taylor explained at the webinar, in impact assessment they explore a balance between process evaluation (the activities and outputs produced), and impact evaluation (short term and long term results). But, as evaluating is not only a matter of quantitative data, they try to capture insights related to the “so what”: the change in feelings of belonging, the civic participation or the increase in cultural awareness, for instance.
Taylor also shared some of the challenges they face, like credibly capturing the value that HUB-IN adds, and knowing where to focus effort. To mitigate these challenges, HUB-IN team developed these key learnings:
- Apply the Theory of Change: You can not measure the end results perfectly, but you can determine the direction of the travel and how well you are moving towards it.
- Generate relevant local datasets: with pre- and post- community surveys and accelerator reports that help learn what is working.
- Leverage the intel of key stakeholders.
- Focus on continuous improvement: Which are the barriers that stop us from going further and what are the emerging opportunities that we can capitalize.
- Priorice indicators and find a balance between quantitative and qualitative data.
Ignace De Somer and Harshita Thakare, T-Factor
Horizon 2020 T-Factor Project’s main objective is to unlock the transformative potential of temporary uses in urban regeneration to move towards more inclusive and resilient cities. As their sister projects, T-Factor team faces the complexity of measuring the transformative impact of their activities, and shared some of the key challenges of their monitoring process:
- Difficulty in measurement: as meanwhile uses occur within broader and complex processes, where many factors are simultaneously at play.
- The problem of attribution: how can we attribute long term effects such as economic and social benefits to meanwhile interventions, specially when it comes to intangible effects.
- Inclusive evaluation: the commitment to be flexible in the evaluation approach, involve the diversity of stakeholders and gather the data from a variety of sources.
Finally, Harshita Thakare and Ignace De Somer shared some of the key learnings gained by the team so far:
- Focus on the processes and pathways of change rather than proving the outcomes.
- Include different voices in the evaluation.
- Capture stories as evidence.
- Give more emphasis to peer exchange because this is where the most interesting insights emerge.
Watch the full webinar:
