How to mix innovation methods to tackle society’s trickiest problems
We share five things we’ve learnt about ways of working in mixed methods teams
This article was co-written by Oliver Zanetti, the Mission Manager for Nesta’s Sustainable Future mission team, Chantale Tippett, Senior Lead of Engagement and Inclusive Impact in their Data Analytics Practice, Laurie Smith, Nesta’s Senior Foresight Lead and Sean Kelleher-Clarke, Senior Programme Manager & Operations Lead for their Data Analytics Practice. This article was first published by Nesta, the UK's innovation agency for social good. Read the original article here.
How do people with vastly different ways of thinking about the world collaborate productively? It’s a knotty problem, and the simplest answer is that very often they don’t. Rather, even in the innovation sphere, people stick to the disciplines they know and deploy the associated methodological tools they’re familiar with. But innovation emerges from novel mixtures of ideas, techniques and mindsets. Getting people talking – and most importantly, collaborating – has many advantages.
We’ve been looking into how and why people use mixed innovation methods in practice, talking to colleagues in Nesta, people from other organisations, and looking into the literature. We’ve learnt a great deal. We’re still digesting its nuance, but for us the most important headline is that mixed methods work. Our interviewees spoke with enthusiasm about how mixing methods enabled them to approach problems in new ways and identify novel and effective solutions.
Mixing methods can lead to really powerful outcomes addressing some of the world’s trickiest problems.
1. Recognise that different methods have different measures of rigour
In a recent Twitter thread on mixed methods, a sociology professor expressed her frustration that ‘publishing qualitative research means clarifying 10,000 times that you are aware that findings from your small sample may not apply to every human being in the universe’. In a sympathetic response, a data scientist critiqued his own discipline noting that this happens ‘while quantitative research keeps making overgeneralized inferences ignoring nuances in the data and population of interest’. ‘Very true,’ came a diplomatic reply, ‘that’s why a mixed-methods approach is often very useful’. We couldn’t agree more.
The frustrations expressed by both sides are at their heart questions about what counts as rigour. Done well, all methods are rigorous in that they capture the particular aspects of a phenomenon they are designed to capture. In this way, sample size is no more an intrinsic measure of rigour than is interview duration or fullness of observation notebook during an ethnographic visit. This echoes the assertions of Justin Parkhurst in his book, The Politics of Evidence (PDF) (see p.123), where he argues that methods should be judged by their own internal quality standards and their suitability for the task at hand.
Judging methods differently can be challenging. In one of our evidence-gathering interviews, the respondent highlighted how there can be a disconnect when executives and those carrying out the work come from different methodological backgrounds, each with their own standards of what a successful project should deliver.
2. Remember that no methods are neutral
All methods are underpinned by their own systems of thought and measures of value. Because there’s no such thing as a neutral method, there’s no point in going looking for methods without bias. Just as different methods have different measures of rigour, different methods address their inevitable and inherent biases in different ways. Working in a mixed-methods team entails engaging with the different imperfections that each method has, and having the confidence to recognise that often failable methods will nevertheless produce findings which are good enough to work from.
3. Learn to speak the same language
Terms like ‘mixed methods’ can mean different things to different people. Likewise, as another interviewee described, the methods people use can employ the same word for different things or different words for the same thing. This is a consequence of methodologies in long-siloed disciplines that have evolved their own vocabulary for the work they do in them.
When working in a mixed-methods domain, it’s important to clarify the meaning behind the terminology you’re using to make sure everyone in the conversation is talking about the same thing. Where there are differences in terminology or method, it’s also important to identify whether there are meaningful differences between them. This is often a question of pragmatism. Is a service safari (PDF) really substantially different from ethnography, for instance, or is data science just a fancy rebranding of what used to be called statistics? Sometimes vocabulary choices can be simply semantics, though they may too be acts of gatekeeping by insiders of a method or approach.
4. Create good communication tools and try to be comfortable with humility
What was clear from all our interviewees with experience in mixed-methods approaches was that humility is essential to working in this way. Openness to the unexpected, being willing to put yourself in situations you don’t quite understand, and being generous with colleagues who ask questions by communicating answers clearly and simply are attributes that make mixing methods work well. Creating a work environment where this is possible is vital, where trust, experimentation and curiosity are valued as highly as any other project output.
High-level expertise can enable cutting-edge work, but when it leads to practitioners wedded to their single method or worldview it can also be a barrier to working in a mixed-methods team.
Humility and good communication extend to other elements of day-to-day practice too. A strong and healthy team dynamic is vital as mixed-methods teams often encounter new challenges for which there is no clear precedent. This may engender debate and disagreement, but there are ways of making this productive not divisive. In agile working, creating this dynamic in mixed-methods teams may be done with shared ceremonies and other novel ways of creating dialogue.
One of our interviewees ran a team of data scientists and sociologists. To open the conversation between each of the methods, participants were encouraged to use the communication tools of their colleagues: the sociologists were signed up to Github and Medium while data scientists found themselves attending reading groups to discuss writing on issues like justice or equity.
5. Stay flexible and open to difference
They say that to a hammer, every problem looks like a nail. As such, to a devout data scientist, there is…