The Ethics of Independent Research Work #1

ethicsI guess we all know by now that I bang on a fair bit about research ethics, but I haven’t written about the ethical aspects of working as an independent researcher. I have come up with ten ethical principles for indie researchers. Many of these no doubt apply to other forms of self-employment too, but they definitely all apply to independent research work. This post contains the first five principles; I will post the other five next week.

  1. Be honest about what you don’t know

If a client says, ‘You know the legislation that…’ and you don’t, it’s best to say so. It can be tempting to nod while making a mental note to look it up online later, but that can lead to disaster. People often fear that saying they don’t know something will make them look stupid, but paradoxically the reverse is true. If you are clear about what you do know and honest about what you don’t, you will build trust with your clients much more quickly and effectively.

  1. Be clear about your capacity

Allied to this: don’t take on work you haven’t got time to do, because that won’t do anyone any favours. You won’t produce your best work for your clients, and you’ll end up burned out. OK there are times where you may choose to work at maximum capacity for a short time, e.g. as one contract ends while another begins, or to fit in a quick piece of work for a valued client. But keep these brief and infrequent, and make sure you build in recovery time. Independent research is a great career (at least, in my view), but no career is worth damage to your health and relationships.

  1. Charge a fair rate for the job

If possible, find out what the going rate is, and charge that. The going rate will vary across sectors and between countries. I have written before about how I charge for work: in brief, I charge less for charities and longer projects, more for universities, governments, and work I don’t really want to do.

Also, don’t take on jobs with inadequate budgets, unless you’re desperate for the money and prepared to accept a very low day rate. I’ve been offered a three-year national evaluation with a total budget of £5,000. Perhaps someone ended up doing that work for that money, but they would either have done a very poor job or effectively accepted an extremely low day rate.

  1. Don’t accept work on an unethical basis

One potential client rang me towards the end of the financial year to ask if I could invoice her for several thousand pounds that she had left in her budget. She said she was a bit busy, so could we sort out what I would do for the money at a later date? I didn’t know her so I asked why she had rung me. She told me she had wanted person A, but they were too busy so they suggested person B, who couldn’t take it on either and suggested me. Nowadays I would probably say a simple ‘no’, but it was early in my career, and person B was quite influential. I agreed to invoice, but only after meeting with my potential client to decide whether we could work together and what I would do for her.

Another time a commissioner rang me to ask me to evaluate a service because he wanted to close it down. I said I would evaluate the service if he wished, but I would not pre-determine the findings; they would be based on my analysis of the data I gathered. He agreed to this. I did the evaluation, and found – unequivocally – that the service was highly valued and doing necessary work. The commissioner paid my invoice, then found someone else to do another evaluation saying the service should be closed down, whereupon he closed it down. Again, with the benefit of hindsight I probably should have said ‘no’ to the assignment, but I naïvely thought that if I did the research the commissioner would abide by the findings.

  1. Don’t take work outside your areas of expertise

You may have more than one area of expertise. I have a few: children/young people/families, housing/homelessness, substance misuse, volunteering, service user involvement, third sector, training. Each of these areas formed part of my professional work before I became an independent researcher.

Earlier this decade I got an email asking me to do some work around learning disability. I replied, explaining that it was not one of my areas of expertise, and saying I didn’t think I was the best person for the job. The potential client came back saying they thought I was right and apologising for having bothered me. (I didn’t mind. I never mind answering queries about possible paid work.)

Oddly enough, a few weeks later I got another email, from someone completely different, asking me to do some work around learning disability. After rolling my eyes and thinking about buses, I sent a similar reply. This time the potential client came back saying that I sounded perfect for the piece of work they wanted to commission. They thought someone with a good knowledge of research methods but little knowledge of learning disability would bring a usefully fresh perspective to the problems they were trying to solve. Which is further evidence for (1) above.

So there you have the first five principles of ethical research work, according to me. Come back next week for the other five.

Academic taboos #1: what cannot be said

An earlier version of this article first appeared in Funding Insight in summer 2017; this updated version is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com.

what can't be saidAcademia is a community with conventions, customs, and no-go areas. These vary, to some extent, between disciplines. For example, in most STEM subjects it is taboo for research authors to refer to themselves in writing in the first person. This leads to some astonishing linguistic contortions. Conversely, in arts disciplines, and increasingly in the humanities and social sciences, it is permissible to use more natural language.

It seems, though, that some conventions exist across all disciplines. For example, conference “provocations” are rarely provocative, though they may stretch the discussion’s comfort zone by a millimetre or two. Then conference “questions” are rarely questions that will draw more interesting and useful material from the speaker. Instead, they are taken as opportunities for academic grandstanding. Someone will seize the floor, and spend as long as they can get away with, effectively saying: “Look at me, aren’t I clever?” I have found, through personal experiment, that asking an actual question at a conference can cause consternation. I confess it amuses me to do this.

Perhaps the most interesting conventions are those around what cannot be said. Rosalind Gill, Professor of Cultural and Social Analysis at City University of London, UK, has noted the taboo around admitting how difficult, even impossible, it can be to cope with the pressures of life as an academic (2010:229). The airy tone when a colleague is heard to say: “I’m so shattered. The jobs on my to-do list seem to be multiplying. Haha, you know how it is.” Such statements can be a smokescreen for serious mental health problems.

A journal article published in 2017 by the theoretical physicist Oliver Rosten made a heartfelt statement about this in its acknowledgements, dedicating the article to the memory of a late colleague, and referring to “the psychological brutality of the post-doctoral system”. Several journals accepted the article for its scientific quality but refused to publish the acknowledgements in full; it took Rosten years to find a journal that would publish what he wrote. He has left academia and now works as a Senior Software Developer at Future Facilities Ltd in Brighton, UK.

Another thing that cannot be said, identified by Tseen Khoo, a Lecturer in Research Education and Development at La Trobe University, Melbourne, Australia, is that some academic research doesn’t need funding, it just needs time. This is anathema because everyone accepts that external funding makes the academic world go round. But what if it didn’t? What if student fees, other income (e.g. from hiring out university premises in the holidays), and careful stewardship was enough? What if all the time academics spent on funding applications, and making their research fit funders’ priorities, was actually spent on independent scholarship? It seems this is not only unsayable but also unthinkable. One of Khoo’s interlocutors described this as “a failure of the imagination”.

Another unspeakable truth I’m aware of is for someone to say that the system of research ethics governance is itself unethical. Ethics governance is something to comply with, not to question. That has led us to the situation where most research training contains little or no time spent on research ethics itself. Instead, young researchers learn that working ethically equates to filling in an audit form about participant welfare and data storage. They don’t receive the detailed reflective instruction necessary to equip them to manage the manifold ethical difficulties any researcher will encounter in the field.

I wonder what role the lack of research ethics education plays in the increasing number of journal articles that are retracted each year? I would argue that we need to separate ethical audit from ethical research, because they have different aims. The former exists to protect institutions, the latter to promote the quality of research and ensure the well-being of all concerned.

These areas of silence are particularly interesting given that academia exists to enable and develop conversations. However, I think that as well as acknowledging what academia enables, we also need to take a long hard look at what academia silences.

I Finished The Book!

Research ethics in the real world [FC]For the last three-and-a-quarter years I have been writing a book on research ethics. It has been like doing another PhD, only with reviewers instead of supervisors. Four sets of reviewers: two sets of proposal reviews and two sets of typescript reviews. I have to thank my lovely publisher, Policy Press (part of Bristol University Press), for giving me so much support to get this book right.

This has been the hardest book I’ve written and I hope never to write another as difficult. On the plus side, I’m happy with the result. It is different from other books on research ethics in three main ways. First, it doesn’t treat research ethics as though they exist in isolation. I look at the relationships between research ethics and individual, social, institutional, professional, and political ethics, and how those relationships play out in practice in the work of research ethics committees and in evaluation research. That makes up part 1 of the book.

Second, it demonstrates the need for ethical thinking and action throughout the research process. In part 2 there is a chapter covering the ethical aspects of each stage of the research process, from planning a research project through to aftercare. There is also a chapter on researcher well-being.

Third, the book sets the Indigenous and Euro-Western research paradigms side by side. This is not to try to decide which is ‘better’, but is intended to increase researchers’ ethical options and vocabularies. I am writing primarily for Euro-Western readers, though the book may be of use to some Indigenous researchers. There is a sizeable and growing body of literature on Indigenous research and ethics, including books, journals, and journal articles. Using this literature requires care – as indeed using all literature requires care (see chapter 7 of my forthcoming book for more on that). But Indigenous literature, as with other literatures by marginalised peoples, requires particular care to avoid tokenism or appropriation.

Many Euro-Western researchers are completely ignorant of Indigenous research. Some know of it but are under the misapprehension that it is an offshoot of Euro-Western research. In fact it is a separate paradigm that stands alone and predates Euro-Western research by tens of thousands of years. Some Indigenous researchers and scholars are now calling for Euro-Western academics to recognise this and use Indigenous work alongside their own. My book is, in part, a response to these calls.

It was so, so hard to cram all of that into 75,000 words – and that includes the bibliography which, as you can imagine, is extensive. There was so much to read that I was still reading, and incorporating, new material on the morning of the day I finished the book. I’ve found more work, since, that I’d love to include – but I had to stop somewhere.

I awaited my final review with great trepidation, aware of the possibility that the reviewer might loathe my book – some previous reviewers had – and that that could put an end to my hopes of publication. Was I looking at three-and-a-quarter years of wasted work? I was so relieved when my editor emailed to say the review was positive. Then the reviewer’s comments blew me away. Here’s one of my favourite parts: “In my view the author through excellent writing skills has covered very dense material (a ton of content) in a very accessible way.”

I was even more delighted because this review came from an Indigenous researcher. She waived anonymity, so I have been able to credit and thank her in the book. I will not name her here, as I do not have her permission to do so; you’ll have to read the book if you want to find out.

Finishing a book feels great, and also weird. It’s like losing a part of your identity, particularly with a book you’ve lived with for so long. Though there’s still lots of work to do: I have to write the companion website, give input on the book’s design, read the proofs, start marketing… publication is due on 1 November, which feels a long way off but I know how quickly five months can pass.

I think this book will be controversial. A senior and very knowledgeable academic told me that one reason I could write such a book is because I’m not in academia. I’m glad if I can use my independence to say things others cannot say – as long as I’m saying useful things, at least.

More than anything else, I hope the book helps to make a difference. In particular, I would like to make a difference to the current system of ethical regulation which is too focused on institutional protection and insufficiently focused on ethical research. It is also terrible at recognising and valuing the work of Indigenous research and of Euro-Western community-based or participatory research. When I was preparing to write the book, I interviewed 18 people around the world and promised them anonymity. Some were research ethics committee members and others had sought formal approval from ethics committees (or institutional review boards in the US). I heard tales of people completing ethical approval forms with information that committees wanted to see rather than with actual facts; people teaching students how to get through the ethical approval system instead of teaching them how to conduct ethical research; people acting ethically yet in complete contravention of their committee’s instructions; people struggling to balance ethical research with Indigenous communities with the inflexible conditions set by ethics committees. Although many of the people who serve on ethics committees are highly ethical, the system within which they are forced to work often prevents them from acting in entirely ethical ways. It seems to me that this system is not currently fit for purpose, and there are many other people who think the same. I hope the evidence I have gathered and presented will help to create much-needed change.

As an independent researcher, I am self-employed. This means I do all my writing in my own time; I don’t have a salary to support my work. Do you like what I do on this blog, or in my books, or anywhere else, enough that you might buy me a coffee now and again if we were co-located? If so, please consider supporting my independent work through Patreon for as little as one dollar per month. In exchange you’ll get exclusive previews of, and insights into, my work. Thank you.

How Open Is Open Access?

This article first appeared in Funding Insight on 18 January 2018 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com.

partly open doorThose outside the UK probably won’t be aware of Jisc. The non-profit organisation’s role is to provide technological solutions to academic problems, including researching and developing new ways of working supported by technology. (Full disclosure: they are also one of my clients.) Jisc is publicly funded by UK taxpayers’ money and member subscriptions. Its members are from the UK, and its objectives are designed to create benefit for the staff and students of adult education institutions in the UK.

But its strategy includes a stated intention of ‘growing our offering internationally to further benefit our members’.

Jisc is also very keen on open sharing of information and resources. It advocates open access to research publications, which its says means making them ‘freely available so anyone can benefit from reading and using research’. It promotes the sharing of research data, and the use of non-restrictive Creative Commons licensing to enable re-use of resources. Jisc identifies various potential benefits of this, one of which is that ‘researchers in developing countries can see your work’.

So far, so many good intentions. I’m sure most of my Euro-Western colleagues will be nodding their heads and thinking yes, marvellous, jolly well done Jisc. And indeed I am not writing this post to criticise those at Jisc, who are doing their best to be good guys, and who after all exist in the UK for the benefit of their UK members. My point here is to critique our more general Euro-Western academic mindset, which Jisc’s example illustrates.

You see, until very recently, I would have been one of those people nodding along, with a satisfied expression, thinking “oh Jisc you are doing well”. But my eyes have been opened by a recent blog post written by Andy Nobes of international development charity INASP, featuring the work of Florence Piron from Université Laval in Québec and her colleagues from around the world. Piron and her colleagues have written – in French – some publications that offer new perspectives and ideas to Euro-Western advocates of open access. In particular, they challenge the idea that Euro-Western researchers simply making their work visible to ‘researchers in developing countries’, as Jisc suggests, equates to open access. By contrast, they see it as an extension of colonialism and an ‘alienation epistemic’. This is because it does nothing to make knowledge generated in other parts of the world equally visible to researchers in Euro-Western countries. In turn, that serves to reinforce the use of Euro-Western theories and models as normative, which is to the detriment of local epistemologies in other parts of the world (Piron et al 2017).

We don’t think of these things, do we, us Euro-Western researchers? We’re too confident that we’re doing OK as long as we’re making some kind of gesture towards those with fewer privileges.

Piron and her colleagues point out that many Euro-Western academics are unable even to think that ‘valid and relevant knowledge’ could exist in other places and other ways; they can be ‘blind to epistemological diversity’ and regard Western science as universal (ibid). Even those Euro-Western academics who do respect other forms of knowledge are unlikely to engage in truly reciprocal knowledge exchanges. Collaborative projects often involve Euro-Western academics acting as principal investigators while researchers from other parts of the world are restricted to data-gathering and administrative work (Sherwood 2013, Yantio 2013).

In some Euro-Western academic circles there are moves afoot to ‘decolonise the curriculum’. But this is not only needed in Euro-Western establishments. Some teachers in other parts of the world also choose exclusively Euro-Western literature and examples for their students (Mboa Nkoudou 2016). This is a direct real-world consequence of the pervasive Euro-Western conviction that any other way of thinking must be inferior to our own. It makes life harder – not easier – for people in other parts of the world to solve their own local problems in appropriate and sustainable ways (ibid).

Truly open access will involve a two-way exchange of – and respect for – knowledge and the epistemological positions on which it is based. Obviously this is beyond the power of a single organisation, such as Jisc, or a single individual, such as you or I. However, all Euro-Western researchers, and those who work with them, need to be aware of the difference between open access as we tend to purvey it, and genuinely open access. Only with such awareness will we find ways to move from our one-way, take-it-or-leave-it approach to a true openness and sharing with other academics around the world.

The Ethics of Expertise

expertLast week I wrote about the ethics of research evidence, in which I cited Charles Knight’s contention that evidence should be used by people with expertise. Knight also questions how we can identify people with expertise. He suggests they would ‘have to do the sorts of things experts do – read the literature, do research, have satisfied clients, mentor novices, and so on’. He adds, ‘This approach is not likely to concentrate expertise in a few hands.’ (Knight 2004:2)

I like Knight’s attempts to widen the pool of acknowledged experts. He is evidently aware of the scope for tension between expert privilege and democracy. Conventionally, experts are few in number, specialists, and revered or at least respected for their expertise. However, this can also be viewed as exclusionary, particularly as most experts of this kind are older white men. Also, I’m not sure Knight goes far enough.

Knight was writing at the start of the century and, more recently, different definitions of ‘expert’ have begun to creep into the lexicon. For example, the UK’s Care Quality Commission (CQC), which inspects and regulates health and social care services, has defined ‘experts by experience‘. These are people with personal experience of using, or caring for someone who uses, services that the CQC oversees. Experts by experience take an active part in service inspections, and their findings are used to support the work of the CQC’s professional inspectors.

In research, there is a specific participatory approach known as critical communicative methodology (CCM) which was developed around 10 years ago. CCM takes the view that everyone is an expert in something, everyone has something to teach others, and everyone is capable of critical analysis. This is a fully egalitarian methodology which uses respectful dialogue as its main method.

However, in most of research and science, experts are still viewed as those rare beings who have developed enough knowledge of a specialist area to be able to claim mastery of their subject. There is a myth that experts are infallible, which of course they’re not; they are human, with all the associated incentives and pressures that implies. It seems that experts are falling from grace daily at present for committing social sins from fraud to sexual harassment (and getting caught).

Perhaps more worryingly, the work of scientific experts is also falling from grace, in the form of the replication crisis. This refers to the finding that scientific discoveries are not as easy to replicate as was once supposed. As replication is one of the key criteria scientists use to validate the quality of each other’s work, this is a Big Problem. There is an excellent explanation of the replication crisis, in graphic form, online here.

My own view is that replication is associated with positivism, objectivity, the neutrality of the researcher, and associated ideas which have now been fairly thoroughly discredited. I think this ‘crisis’ could be a really good moment for science, as it may lead more people to understand that realities are multiple, researchers influence and are influenced by their work, and the wider context inevitably plays a supporting and sometimes a starring role.

As a result of various factors, including the replication crisis, it seems that the conventional concept of an expert is under threat. This too may be no bad thing, if it leads us to value everyone’s expertise. Perhaps it could also help to overturn the ‘deficit model’ which still prevails in so much social science, where (expert) researchers focus on people’s deficits – their poverty, ill-health, low educational attainment, unemployment, inadequate housing, and so on – rather than on their strengths and the positive contributions they make to our society. The main argument in favour of the deficit model is that these are problems research can help to solve, but if that were true, I think they would have been solved long since.

For sure, at times you need an expert you can trust. For example, if your car goes wrong, you’ll want to take it to an expert mechanic; if you develop a health problem, you’ll want to seek advice from an expert medic. It doesn’t seem either ethical or sensible, to me, to try to discard the conventional role of the expert altogether. But it does seem sensible to attack the links between expertise and privilege. After all, experts can’t exercise their expertise without input from others. At its simplest, the mechanic needs you to tell them what kind of a funny noise your car is making, and under what circumstances; the medic needs you to explain where and when you feel pain. Also, it doesn’t seem sensible to restrict conventional experts to a single area of expertise. That mechanic may also be an expert bassoon player; the medic may know more about antique jewellery than you ever thought possible.

In my view, the ethical approach to expertise is to treat everyone as an expert in matters relating to their own life, and beyond that, as someone who has a positive contribution to make to a specific task at hand and/or wider society in general. Imagine a world in which we all acknowledged and valued each other’s knowledge, experience, and skills. You may say I’m a dreamer – but I’m not the only one.

The Ethics of Research Evidence

Like so many of the terms used in research, ‘evidence’ has no single agreed meaning. Nor does there seem to be much consensus about what constitutes good or reliable evidence. The differing approaches of other professions may confuse the picture. For example, evidence that would convince a judge to hand down a life sentence would be dismissed by many researchers as anecdote.

evidenceGiven that evidence is such a slippery, contentious topic, how can researchers begin to address its ethical aspects? A working definition might help: evidence is ‘information or data that people select to help them answer questions’ (Knight 2004:1). Using that definition, we can look at the ethical aspects of our relationship with evidence: how we choose, use, and apply the evidence we gather and construct.

Evidence is often talked and written about as though it is something neutral that simply exists, like a brick or a table, to be used by researchers at will. Knight’s definition is helpful because it highlights the fact that researchers select the evidence they use. Evidence, in the form of facts or artefacts, is neither ethical nor unethical. But in the process of selection, there is always room for bias, and that is where ethical considerations come into play.

To choose evidence ethically, I would argue that first you need to recognise the role of choice in the process, and the associated potential for bias. Then you need to consider some key questions, such as:

  • What is the question you want to answer?
  • What are your existing thoughts and feelings about that topic?
  • How might they affect your choices about evidence?
  • What can you do to make those choices open and defensible?

The aim is to be able to demonstrate that you have chosen the information or data you intend to define as ‘evidence’ in as ethical a way as possible.

Once you have chosen your evidence, you need to use it ethically within the research process. This means subjecting all your evidence to rigorous analysis, interpreting your findings accurately, and reporting in ways that will communicate effectively with your audiences. These are some of the key responsibilities of ethical researchers.

Research is a process that converts evidence into research evidence. It starts with the information or data that researchers choose to use as evidence, which may be anything from statistics to artworks. Then, through the process of (one would hope) diligent research, that evidence becomes research evidence. Whether and how research evidence is applied in the wider world is the third ethical aspect.

Sadly, there is a great deal of evidence that evidence is not applied well, or not applied at all. Most professional researchers have tales to tell of evidence being buried by research funders or commissioners. This seems particularly likely where findings conflict with political or money-making ambitions. In some sectors, such as third sector evaluation, this is widespread (Fiennes 2014). How can anyone make an evidence-based decision if the evidence collected by researchers has not been converted into evidence they can use?

The use of research evidence is often beyond the control of researchers. One practical action a researcher can take is to suggest a dissemination plan at the outset. This can be regarded as ethical, because such a plan should increase the likelihood of research evidence being used. But it could also be regarded as manipulative: using the initial excitement around a new project to persuade people to sign up to a plan they might later regret.

It seems that ethics and evidence are uneasy bedfellows. Again, Knight tries to help us here, by suggesting that research evidence should be used by people with expertise. This raises a further, pertinent question: what is the ethics of expertise? I will address that next week.

A version of this article was originally published in ‘Research Matters’, the quarterly newsletter for members of the UK and Ireland Social Research Association.

Dissemination, Social Media, and Ethics

twitterstormI inadvertently caused a minor Twitterstorm last week, and am considering what I can learn from this.

I spotted a tweet from @exerciseworks reporting some research. It said “One in 12 deaths could be prevented with 30 minutes of exercise five times a week” (originally tweeted by @exerciseworks on 22 Sept, retweeted on the morning of 10 October). The tweet also included this link but I didn’t click through, I just responded directly to the content of the tweet.

Here’s their tweet and my reply:

 

The @exerciseworks account replied saying it wasn’t their headline. This was true; the article is in the prestigious British Medical Journal (BMJ) which should know better. And so should I: in retrospect, I should have checked the link, and overtly aimed my comment at the BMJ as well.

Then @exerciseworks blocked me on Twitter. Perhaps they felt I might damage their brand, or they just didn’t like the cut of my jib. It is of course their right to choose who to engage with on Twitter, though I’m a little disappointed that they weren’t up for debate.

I was surprised how many people picked up the tweet and retweeted it, sometimes with comment, such as this:

Rajat Chauhan tweet

and this:

Alan J Taylor tweet

which was ‘liked’ by the BMJ itself – presumably they are up for debate; I would certainly hope so. (It also led me to check out @AdamMeakins, a straight-talking sports physiotherapist who I was pleased to be bracketed with.)

Talking to people about this, the most common reaction was to describe @exerciseworks as a snowflake or similar, and say they should get over themselves. This is arguable, of course, though I think it is important to remember that we never know what – sometimes we don’t know who – is behind a Twitter account. Even with individual accounts where people disclose personal information, we should not assume that the struggles someone discloses are all the struggles they face. And with corporate or other collective accounts, we should remember that there is an individual person reading and responding to tweets, and that person has their own feelings and struggles.

Twitter is a fast-moving environment and it’s easy to make a point swiftly then move on. Being blocked has made me pause for thought, particularly as @exerciseworks is an account I’ve been following and interacting with for some time.

I stand by the point I made. It riles me when statistical research findings are reported as evidence that death is preventable. Yes, of course lives can be saved, and so death avoided at that particular time. Also, sensible life choices such as taking exercise are likely to help postpone death. But prevent death? No chance. To suggest that is inaccurate and therefore unethical. However, forgetting that there is an actual person behind each Twitter account is also unethical, so I’m going to try to take a little more time and care in future.

How to evaluate excellence in arts-based research

This article first appeared in Funding Insight on 19 May 2016 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com.

judgementResearchers, research commissioners, and research funders all struggle with identifying good quality arts-based research. ‘I know it when I see it’ just doesn’t pass muster. Fortunately, Sarah J Tracy of Arizona State University has developed a helpful set of criteria that are now being used extensively to assess the quality of qualitative research, including arts-based and qualitative mixed-methods research.

Tracy’s conceptualisation includes eight criteria: worthy topic, rich rigour, sincerity, credibility, resonance, significant contribution, ethics, and meaningful coherence. Let’s look at each of those in a bit more detail.

A worthy topic is likely to be significant, meaningful, interesting, revealing, relevant, and timely. Such a topic may arise from contemporary social or personal phenomena, or from disciplinary priorities.

Rich rigour involves care and attention, particularly to sampling, data collection, and data analysis. It is the antithesis of the ‘quick and dirty’ research project, requiring diligence on the part of the researcher and leaving no room for short-cuts.

Sincerity involves honesty and transparency. Reflexivity is the key route to honesty, requiring researchers to interrogate and display their own impact on the research they conduct. Transparency focuses on the research process, and entails researchers disclosing their methods and decisions, the challenges they faced, any unexpected events that affected the research, and so on. It also involves crediting all those who have helped the researcher, such as funders, participants, or colleagues.

Credibility is a more complex criterion which, when achieved, produces research that can be perceived as trustworthy and on which people are willing to base decisions. Tracy suggests that there are four dimensions to achieving credibility: thick description, triangulation/crystallization, multiple voices, and participant input beyond data provision. Thick description means lots of detail and illustration to elucidate meanings which are clearly located in terms of theoretical, cultural, geographic, temporal, and other such location markers. Triangulation and crystallisation are both terms that refer to the use of multiplicity within research, such as through using multiple researchers, theories, methods, and/or data sources. The point of multiplicity is to consider the research question in a variety of ways, to enable the exploration of different facets of that question and thereby create deeper understanding. The use of multiple voices, particularly in research reporting, enables researchers more accurately to reflect the complexity of the research situation. Participant input beyond data provision provides opportunities for verification and elaboration of findings, and helps to ensure that research outputs are understandable and implementable.

Although all eight criteria are potentially relevant to arts-based research, resonance is perhaps the most directly relevant. It refers to the ability of research to have an emotional impact on its audiences or readers. Resonance has three aspects: aesthetic merit, generalisability, and transferability. Aesthetic merit means that style counts alongside, and works with, content, such that research is presented in a beautiful, evocative, artistic and accessible way. Generalisability refers to the potential for research to be valuable in a range of contexts, settings, or circumstances. Transferability is when an individual reader or audience member can take ideas from the research and apply them to their own situation.

Research can contribute to knowledge, policy, and/or practice, and will make a significant contribution if it extends knowledge or improves policy or practice. Research may also make a significant contribution to the development of methodology; there is a lot of scope for this with arts-based methods.

Several of the other criteria touch on ethical aspects of research. For example, many researchers would argue that reflexivity is an ethical necessity. However, ethics in research is so important that it also requires a criterion of its own. Tracy’s conceptualisation of ethics for research evaluation involves procedural, situational, relational, and exiting ethics. Procedural ethics refers to the system of research governance – or, for those whose research is not subject to formal ethical approval, the considerations therein such as participant welfare and data storage. Situational ethics requires consideration of the specific context for the research and how that might or should affect ethical decisions. Relational ethics involve treating others well during the research process: offering respect, extending compassion, keeping promises, and so on. And exiting ethics cover the ways in which researchers present and share findings, as well as aftercare for participants and others involved in the research.

Research that has meaningful coherence effectively does what it sets out to do. It will tell a clear story. That story may include paradox and contradiction, mess and disturbance. Nevertheless, it will bring together theory, literature, data and analysis in an interconnected and comprehensible way.

These criteria are not an unarguable rubric to which every qualitative researcher must adhere. Indeed there are times when they will conflict in practice. For example, you may have a delightfully resonant vignette, but be unable to use it because it would identify the participant concerned; participants may not be willing or able to be involved beyond data provision; and all the diligence in the world can’t guarantee a significant contribution. So, as always, researchers need to exercise their powers of thought, creativity, and improvisation in the service of good quality research, and use the criteria flexibly, as guidelines rather than rules. However, what these criteria do offer is a very helpful framework for assessing the likely quality of research at the design stage, and the actual quality of research on completion.

Next week I will post a case study demonstrating how these criteria can be used.

Working With Indigenous Literature

Indigenous methods booksI have been working with Indigenous literature on research methods and ethics for over a year now: reading, annotating, thinking, re-reading, and writing from this literature alongside Euro-Western literature on the same topics. During this process I have been reflecting on how I should, and how I do, work with Indigenous literature.

Working well with any literature requires careful and thorough reading and reflection. Beyond that, though, it seems to me there are some specific requirements for working with Indigenous literature.

To begin with, readers need to understand that although the current body of literature is small, it represents a much longer research tradition than that represented by Euro-Western methods/ethics literature. Indigenous researchers around the world, through their oral history, can trace their research tradition back over 40,000 years (Passingan 2013:361; Steere 2013:388-391). By contrast, in Euro-Western terms, anything over 6,000 years old is pre-history. However, there is also a translation problem. Although the first language of some Indigenous people these days may be English, the community and tribal languages of Indigenous peoples worldwide are very different. The complex concepts people develop when talking and thinking about research and its context may not translate directly between a community or tribal language and English, or vice versa. Even when words can be translated, the ideas those words describe may not be fully translatable to people with different world views. For example, some Indigenous peoples regard their ancestors as involved in the research they conduct. This is an unusual concept for most Euro-Western researchers and it is unlikely that we can understand fully what that involvement means to the Indigenous peoples concerned.

Then readers need an awareness that translating an oral tradition into writing is a big intellectual and emotional challenge. There is evidence within the Indigenous research literature that some of its authors are ambivalent about writing. For example, Shawn Wilson states that ‘A problem with writing down stories is that it makes it very difficult to change them as we gain new learning and insights.’ (2008:22) He solves this problem in his own work by writing, not in the Euro-Western linear style of introduction, methods, findings etc, but in ‘a more cyclical pattern that introduces ideas or themes, then returns to them at intervals with different levels of understanding’ (ibid:42). He also tries to write relationally, in accordance with Indigenous ethical principles, i.e. to build a relationship between himself as author, his readers, and the ideas he puts forward (ibid:6). With this aim in view, he uses a literary device of addressing his three children in sections of the book. He chose to do this because, unlike his unknown readers, he has an existing relationship with his children which enables him to use a more personal ‘voice’ in writing for them (ibid:9). Wilson’s hope is that these sections, alongside the more conventionally written ones, will help his readers to develop their own relationships with the ideas and themes in his book.

Wilson and others (e.g. Lambert 2014) also include stories from other named people in their books. This requires a change in citation practices, as simply citing the name of the person on the cover of the book does not adequately acknowledge the source of information. I have solved this problem by citing this way: ‘Lewis in Wilson 2008:110’ or ‘Leslie Camel in Lambert 2014:167’. (I have yet to find out whether my publisher regards this as OK or not, though I hope they will.)

It seems to me that Indigenous literature should be worked with in its own ethical terms. Indigenous research ethical principles are not the same as Euro-Western research ethical principles. While they are described differently by different writers, there are four key ethical principles that recur in the Indigenous literature: respect, accountability, relationality and reciprocity. For me, Indigenous literature should be treated with respect, such as by not dismissing concepts that seem unusual or even alien to me, but by giving them full consideration. If I can’t embrace a particular concept myself, at the very least I need to acknowledge and respect its significance for some Indigenous peoples.

Accountability comes with putting my name to my writing on these topics, making my views available on my blog and (next year) in a book, and giving anyone the right to comment or reply, whether or not they agree with me. Relationality involves careful reading and thinking and use of the literature, trying to understand the different works and authors in relation to me, and me in relation to them, as well as the relationships between the works and authors themselves. This is very much a work in progress and I hope one day to extend it into actual relationship with individual Indigenous researchers, though whether or not any Indigenous researcher would find value in that remains to be seen.

Reciprocity is the principle I find the most difficult when working only with literature. I hope I can achieve this by writing from the Indigenous literature in a way that its authors will value, and in a way that will enable other Euro-Western researchers and scholars to find and learn from that literature as I myself have done. I have been very concerned that working with the literature could be extractive, i.e. taking ideas and concepts and using them for my benefit alone. It is true that the books I write help me to earn a living, but I am not getting rich from them. Nobody pays me while I write, and my income from writing in 2016-17 was around £1,500 (I don’t have the exact figure yet, but will post it when I do). Also, I don’t write niche academic monographs, but textbooks that have proven to be of interest to postgraduate students and academics around the world. I found comfort in direct requests from some Indigenous writers to include their literature in Euro-Western works (Kovach 2009:13,25; Graham Smith in Kovach 2009:88-89; Chilisa 2012:56), and I hope that raising the profile of this literature within the Euro-Western research world will prove to be a positive act. If it backfires in some unforeseen way, on Indigenous individuals and communities who have already borne too much pain, all I will be able to do is acknowledge and own my responsibility for my part in that process, to remain accountable, learn, and do what I can to right any wrongs.

This post is a starting point rather than a conclusion. I have figured all this out myself, so I may be on the wrong track. If you are working with, or have an interest in, this body of literature, please contribute your views in the comments.

On Being An Indigenous Ally

alliesI posted a book list earlier this month, and a surprisingly large number of people have shown interest. So I thought maybe I should explain myself, as best I can.

Most allies of Indigenous peoples seem to be non-Indigenous settlers in Indigenous lands. I am an indigenous British woman – the small ‘i’ denotes that I am a native of a colonising, not a colonised, country. I do not see Indigenous peoples on the streets of my country; I rarely read about them in my national newspapers, and when I do, they are in the international section that reports news from other countries.

So why am I an Indigenous ally?

Good question.

I wasn’t taught about colonisation at school. Our history lessons were all about British and European monarchs, mostly kings, and wars and conquests. People of colour didn’t feature (I wonder, now, how my school-friends of colour felt about our history lessons) and women played bit-parts.

I didn’t like history at school, I think mainly because I didn’t identify with the people it featured (not being royal, or a man). When I was in my 20s, more interesting history books began to be published, including Peter Fryer’s groundbreaking works Staying Power: The History of Black People in Britain and Black People in the British Empire (and The Women’s History of the World by Rosalind Miles, but that’s a different blog post). These books taught me some things about the roles people of colour had played in the history of my people, and about how my people had oppressed and exploited others. Then ten years later, when I was doing my masters’ degree in social research methods, the first edition of Linda Tuhiwai Smith’s seminal work Decolonising Methodologies was published. That book taught me about the role of research in colonialism.

It horrifies me that a majority of my fellow Brits think we should be proud of our Imperial legacy, and that over three times as many people think colonised countries are better off as a result than think they are worse off. It seems to me, from all I have read, and heard, and seen online, that the opposite is true. Colonisation is also, usually, spoken of as something that is past. I don’t agree with that either. Professor Bagele Chilisa of Botswana expresses it beautifully:

Colonization can be described as an attempt by the Western world to order the whole world according to Western standards of culture, politics, economic structures, and policies. (2012:81).

This is still happening. For example, several UK honours, given to British people by our Queen for exceptional achievements or service, still reference the British Empire. More broadly, Western peoples are still getting richer from colonisation, e.g. through buying cheap clothes made, or cheap food grown, by people of colour who work in appalling conditions for very little money. There must be many other examples of the Western world dictating how other parts of the world can act. I haven’t been able to find any research quantifying the current benefits accruing to colonisers from colonisation – we need a political economist with skills equivalent to Peter Fryer’s – but I am certain they exist.

This is so wrong and so unjust that it needs bigger words than ‘wrong’ and ‘unjust’. It is evil and I want no part of it, but I am a white person from a colonising country so I have a part in it whether I want one or not. The question that remains for me, then, is how to play that part.

I’m still figuring that out. Some things I know to do, such as shop and bank as ethically as possible, and talk to other white people about our role in colonialism and oppression. I support Indigenous activism where I can. I am reading, citing, and promoting Indigenous literature in my field. I think, too, that I may be able to help with my own writing – particularly my next book, and the second edition of my last book which will follow in due course.

There are probably other things I could do and should do. If you have thoughts or ideas, please pass them on in the comments. There are also things I would like to do. Last year I was able to attend one 90-minute seminar presented by Indigenous researchers, and the year before I was able to help an Indigenous researcher plan a couple of research projects they wanted to do (I learned at least as much as they did). Apart from that, all I have learned has come from books. I would love to spend some time with one or more Indigenous communities, to learn some of the things that can’t be learned from books, such as: how to be a better ally.

Clare Land has written a very good book called Decolonizing Solidarity: Dilemmas and Directions for Supporters of Indigenous Struggles (2015). It is really useful, but as Land is an ‘Anglo-identified non-Aboriginal person living and working in south-east Australia’, it is inevitably focused on Australia. I wonder whether I could write something on how to be an ally for coloniser people who are not settlers. It would be different from Clare Land’s book, though I don’t know in what ways. I hope, over time, I may be able to find out.