Mixed-Methods Data Analysis

concentric circles slide 2Following my post last month about using concentric circles for gathering research data, I had a question from a reader. Nieky van Veggel asked me, “How would I analyse the outcomes of this method?” This is a good question and, like many good questions, it has more than one answer.

First, you can do quantitative analysis: counting and measuring. If you have the participant at the centre, you can count the number of people, agencies, or whatever it is that they have drawn or placed around the concentric circles. Then in either system you can measure the distance, or distances, between the fixed central point and the drawn or placed point(s) chosen by the participant. Once you have the raw numerical data from your counting and/or measurement, you can use statistical calculations as appropriate to your sample size and sampling technique.

Second, you can do qualitative analysis. You can look at the types of relationships depicted and sort those into categories and themes. You can cross-tabulate relationships with other participant attributes, e.g. age or gender. You can also cross-tabulate with any other data you have collected to see if there is a relationship.

Third, you can do both. Then you can synthesize your qualitative and quantitative analyses – or, at least, you can try. There are too many ways of synthesizing data to give full details in a blog post, but you can find more information, references, and examples on pages 106-109 of my book on creative research methods. This post is designed to give you an overview of the subject.

Data synthesis, or data integration as it is also known, can be useful in a number of ways. For example, it can be used to triangulate your data, or to enrich your analysis, and it can yield results which could not be obtained through the analysis of any single dataset. The findings of each single dataset will help to answer your research questions up to a point, but bringing those findings together may give a fuller explanatory narrative. However, integrating findings from different datasets can be one of the most challenging aspects of mixed-methods data analysis. Therefore, it makes sense to have a rationale for doing this, rather than trying to do it for its own sake.

Broadly, when you try to integrate your data, one of three things can happen:

  1. The findings from the different datasets agree. Sadly this is not as common as you might think.
  2. The findings from the different datasets agree in some respects but not in others. This is probably the most common outcome, and requires hard thinking and more analysis to try to resolve the disagreements as far as possible, with further research required where resolution cannot be reached.
  3. The findings from the different datasets do not agree at all. This almost certainly indicates a need for further research – which is not always a palatable message for research managers, commissioners, or funders.

When you write up your data integration process and findings, you need to show how each element relates to the others. The danger with this is it can make your article or report rather ‘methods-heavy’, so be concise where you can.

Australian researchers Reesa Sorin, Tamara Brooks and Ute Haring did some research into children’s understandings of their physical environment. In the process, they developed an analytical procedure using three different methods to analyse a dataset made up of children’s artworks and stories. They began with a quantitative technique: content analysis. This involved identifying the main features of children’s drawings and putting them into categories such as animals, houses and trees. Then they counted the number and frequency of items in each category, reasoning that the more frequently something appeared, the more meaningful it was to children. The other two methods were qualitative. One was interpretive analysis, in which they identified more categories, this time based on the presentation of each drawing, its mood, and the messages in the story the child had told about their drawing. The other qualitative method was developmental analysis, which suggests that stages in the development of children’s artworks can be correlated with their ages. So the content analysis outlined the features of the drawings, the interpretive analysis added depth by showing multiple meanings, and the developmental analysis added ages and stages. The researchers concluded that this combination of analytic methods can ‘provide deep insights into young children’s understandings’ (Sorin, Brooks and Haring 2012: 29).

Data analysis is at the core of our interpretive work as researchers, yet it is rarely discussed and often misunderstood. You can’t learn how to analyse data from a blog post, but it may help you to figure out what some of your current questions are. And I hope, Nieky van Veggel, that this post will provide a step on the way to ticking off another item on your impossible list. Good luck!

Social Media: Is It Just A Numbers Game?

jumbled numbersGoodness me, such a busy week, I almost forgot to blog. This time of year is often very pressured for independents and non-academics with 31 March being a crucial end-of-financial-year deadline by which many projects must be finished and invoices paid. So much so that I haven’t been around on social media anywhere near as much as usual.

Nevertheless, in the last couple of weeks, I have passed the 3,000 follower mark on Twitter, hit 200 followers on Instagram, and reached the magic 500+ on LinkedIn. Ding!

I’ve been on Twitter since 2009 and Instagram since 2014, so these figures aren’t particularly impressive. Publishers, for example, don’t start taking notice till you reach 10,000 followers on Twitter and Instagram. Part of the reason is I’ve never tried to attract followers, other than by being around and talking to people, and following those I feel a connection with. Others take very different approaches. I know a fiction writer on Instagram, someone I’ve met IRL a couple of times, who reached 10,000 followers in less than a year and is now coaching other writers on how to attract followers like honeysuckle attracts bees. She wrote a blog post with a few pointers, such as: choose, and stick to, a colour palette, so that when someone looks at your Instagram profile and sees your last nine photos, they give a coherent message. There were other tips, like how to schedule posts for maximum impact, all of which seemed entirely feasible to implement.

So, I thought, I could do that.

I’d probably sell more books that way.

But.

I can’t do that.

I simply can’t bring myself to be so contrived. It’s not me at all. The thought of choosing a colour palette, and scheduling my posts for maximum impact, makes me feel queasy.

One of the things I love about social media is the randomness. Yesterday I mislaid my landline handset, wailed on Twitter, and a woman from Brisbane told me how to find it. On Instagram I seek out interesting academics and I talk to a Romanian woman studying for a PhD in Japan, a Dutch academic in America, an Australian academic in America, and so on. The glimpses of their lives are fascinating to me; I hope mine are to them.

I should take the ‘colour palette’ approach; it’s sensible marketing. I should create and nurture a brand for myself. To an extent, this blog has a colour palette. The designer friend who made it for me asked what my favourite colours were; I told her; she liked and used those colours. But I don’t pick photos in those colours for my blog posts (though I did for this post, because I came across one on freeimages.com and it amused me), and I certainly can’t Instagram everything in purple, turquoise, and hot pink. It would be exhausting even to try.

I was musing about all this to a friend who is quite the social media expert.

“I just like, y’know, hanging out with people online,” I said. “I don’t want to do this ‘brand’ thing. I want the weird, the random, the serendipitous. I like making friends.”

“That,” he said, “that IS your brand.”

I guess he’s right. It’s not much of a brand, though, in marketing terms. It’s just me, bimbling around online like I do offline, forgetting things sometimes, doing my best. I could change; I could do this whole thing differently. Maybe, sometime in the future, I will. But, for now… I don’t want to put myself under the pressure of trying to present myself as something I’m not, something polished and shiny. It would be too much like having a proper job. So I shall carry on dropping in and out of social media as I please, chatting when I have something to say, and otherwise lurking or taking time out. That makes me happy. And while my approach may not sell as many books as the ‘colour palette’ system, it has made me some great friends and taken me all around the world. So, in my terms, it works just fine.

New Directions in Qualitative Research Ethics

TSRMcover 1..2I have co-edited a special issue of the International Journal of Social Research Methodology with my friend and colleague Dr Lucy Pickering from the University of Glasgow. It is online today – International Women’s Day, how timely! – and it’s called New Directions in Qualitative Research Ethics.

I am the ethics lead for the UK and Ireland Social Research Association, and Lucy is the ethics lead for the Association of Social Anthropologists of the UK and Commonwealth. Lucy and I both help our respective associations’ members with real-life ethical dilemmas (and trilemmas, and quadrilemmas). We have both found that the emphasis of the ethical review process on data gathering and participant well-being leaves researchers ill-equipped to cope with ethical difficulties at other stages of the research process. Overall, the ethics literature is similarly skewed, with most articles and chapters focusing on data gathering and participant well-being. Our aim was to produce a special issue that would help to shift this balance. We wrote one of the articles, on the ethics of presentation and representation, and of course the editorial. Other articles cover the teaching of research ethics, the ethics of recruitment and sampling, working ethically with participants who have profound intellectual disabilities, what to do when ethics committees’ stipulations prove irrelevant in the field, and some of the ethical issues raised by cross-cultural research. There is also a useful review essay of three recent books on research and social justice.

We are proud that our authors are truly international, hailing from New Zealand, Australia, Finland, America and Thailand as well as the UK. The journal is paywalled, but there are 50 free downloads of every article, so get in quick if you don’t have access.

 

How To Become An Indie Ally

cat-and-dogCalling academics! Do you want to be a useful ally to independent researchers? Then here’s how you can. No, wait, let’s start with why it’s a good idea. Independent researchers can add considerable value to academic research and teaching projects. We bring a fresh perspective, which can be useful to help disentangle problems that seem entrenched, or simply to provide a new view of a situation. We have time to think, because we don’t have to tangle with time-consuming internal meetings and university bureaucracy. And we are not limited in what we work on by managerial directives or departmental policy. Also, we are flexible and can sometimes help out at short notice, such as when a colleague has an unexpected leave of absence at a crucial stage in a project. One potential downside is that an indie researcher is unlikely to have the depth of knowledge in any one subject of a professor who has spent decades studying a single area. On the other other hand, indie researchers often bring a breadth of knowledge across several related areas, and are skilled in bringing themselves up-to-date fast in any area they haven’t worked on for a while.

Another reason it might be a good idea to support independent researchers is that, as the options for tenure in academia decrease, the likelihood of any academic ending up as an indie increases. So supporting indie researchers and scholars may prove to be an investment in your own future. An academic of my acquaintance told me recently that she wonders why staff at her post-92 university are regularly asked to give free support to universities in the much richer Russell Group (another structural faultline of academic inequality). She has decided to stop offering free training to other universities, whatever their grouping, because it affects the market for independent workers. Be like her!

So those are some reasons why it’s a good idea to use indie researchers; now let’s look at how they can be used. The three main ways academic departments use indie researchers are: as part of a team on a funded research project; to augment a teaching programme; or to fill gaps in capacity. Of course there are many other ways, from delivering a single class or seminar to providing years of doctoral supervision.

Here’s how to help make that happen.

  1. Get to know your local indie researchers and/or the indie researchers who work in your field. This way, when you need some help in a hurry, you’ll have an existing relationship as a springboard.
  1. Be mindful that indie researchers don’t receive a salary; nobody is paying for their time. Any decent indie researcher should be willing to come to an exploratory meeting without expecting to be paid. However, it will be helpful if you can acknowledge the imbalance: you are drawing a salary for your time at that meeting; they are not. It will be even more helpful if you can at least reimburse their travel expenses, and maybe give them lunch. Please do not expect an indie researcher to come to more than one meeting without recompense. Some academics still think it’s OK to ask an indie to run a workshop, speak at a conference, and write a chapter for an edited collection. A salaried academic could say ‘yes’ to all of those without pausing for breath, even though the tasks probably require 2-3 weeks of full-time work to complete. If you’re not paying an indie, you’re asking them to do that in their own time. That’s equivalent to asking a salaried academic to work on a dozen consecutive Sundays. If the latter would give you pause, so should the former.
  1. Understand how independent researchers’ day rates work. These day rates look high, but at times we go for weeks or months with no paid work, and we have none of the benefits of employment such as holiday pay or sick pay or conference budgets. For example, I charge universities £800 per day and in 2015-16 I was able to pay myself £17,000 – around one-third of what I would be taking home if I’d spent the last 17 years in academia. In the last five years, I’ve had two good years and three lean years. There are other compensations to the indie lifestyle so this is not intended as a sob story. But it’s surprising how many intelligent people still think ‘high day rate’ equals ‘rich person’.
  1. If you really can’t pay an independent researcher, but you want them to work with you, think about what you can offer them in exchange for their skills and labour. They might be glad to have use of your library, an honorary position with access to paywalled journals, or a free place on a professional training course. Most indie researchers are open to barter as long as you can offer something that is of value to them. What won’t be of value is ‘exposure’, because in these days of social media we can all expose ourselves.

 

  1. Where appropriate, allocate time and costs in your funding bids for input from one or more independent researchers. This sends a positive message to funders: it shows that you are thinking beyond the walls of the academy and taking a creative approach to your bid and your project design. Any credible independent researcher who you plan to include should be willing to put in some unpaid desk work up front, perhaps to write a section of the bid or to give feedback on a draft.
  1. Raise awareness among your colleagues of the value, and support needs, of independent researchers. If you have the contacts, and want to earn serious brownie points from the indies in your networks and beyond, lobby for indie researchers to have access to research funding.

One caveat: it is important to perform due diligence. Ask for a CV, with references; follow up the references, and spot-check a couple of items from the CV. If the independent researcher hasn’t been independent for long, it would be worth quizzing them about their intentions. Due to the economic climate and the casualisation of academic work, some people are setting up as independent researchers in the hope of earning a few quid while they’re searching for salaried employment. It won’t help your research plan if, by the time you secure funding for your three-year project, your nominated indie researcher is now a full-time lecturer at the other end of the country.

I hosted a lively Twitterchat about independent research for #ecrchat on 24 February, and was hoping to link to the resulting Storify from this post but technical problems have intervened. If we are able to storify the chat in future I’ll include the link here. I was also hoping to refer to the Storify for any points I may have missed, as I’m not at all sure the above list is exhaustive, so if you have any points to add, please include them in the comments below.