Methodology, Method, and Theory

debatingLike last week’s post, this one was inspired by @leenie48 on Twitter. My post of the week before was on how to choose a research question, and @leenie48’s view was that I should not tackle that topic without considering theory. Last week’s post dealt with why I didn’t include theory in the previous post (I hope you’re all keeping up at the back). This week’s post, as promised, explains why I think theory sits with methodology rather than with method.

Some people think ‘methodology’ is just a posh word for ‘method’. This is a bit like how some people think ‘statistical significance’ is a more important version of ordinary everyday ‘significance’. As in, it’s completely wrong.

Methods are the tools researchers use to practice our craft: to gather and analyse information, write and present findings. We have methods for searching literature and sources, gathering and analysing data, reporting, presenting, and disseminating findings. Methodologies are the frameworks within which we do all of this work, and are built from opinions, beliefs, and values. These frameworks guide us in selecting the tools we use, though they are not entirely prescriptive. Therefore one method, such as interviewing, may be used for research within different methodologies, such as realist evaluation or feminist research.

Here, as almost everywhere in the field of research methods, terminology is contested. But most people agree that there are several overarching categories of methodologies, such as post-positivist, constructivist, and interpretivist, and that within those overarching categories there are more specific methodologies, such as post-modernism and phenomenology. There are debates about what each category and methodology is, and how different methodologies should be used. These debates are mostly based on theory.

As I explained last week, theory also comes in many forms and is widely debated. These kinds of debates keep some academics in full-time work and are much too complex to summarise in a blog post. What I can say here is that @leenie48 and I disagree on a fundamental point. She thinks it is not an option to ‘jump from rq [research question] to method choice with no consideration of theory’. I know it is an option because I have seen it done many times, and have done it myself as an independent researcher working on commission for clients who are not interested in considering theory or in paying me to consider theory. The kind of briefs I often work to say, for example, ‘We want to know what our service users think about the service we provide, please do a set of interviews to find out.’ The commissioners don’t want a literature review or any explicit theoretical underpinnings, they simply want me to use my independent research skills to find out something they don’t know which will help them take their service forward. In a different context, I have taught and externally examined Masters’ level students, in subjects such as business studies and advice work, who are learning to do research. Their projects focus on method, not theory. It is as much as they can do, in their small word allocation, to contextualise their work, give a rationale for the method they have chosen, and describe and discuss their findings.

Masters’ level students in some other subjects would need to engage with theory, as I did in my own studies for MSc Social Research Methods, and I cannot imagine anyone doing research at doctoral level without using a theoretical perspective. I agree with @leenie48 that theory is important and has a lot to offer to research. In an ideal world, theory would form an equal part of a triad with research and practice.

In a comment on last week’s blog post, Sherrie Lee suggested that theory may be always present in some form, even if it is not explicitly considered. I think she makes a good point. I would like to use theory explicitly in all the research I do, rather than just some of it, but I am not sure that day will ever come. Much commissioned research isn’t explicit about methodology either. There is a lot of practice-based, and practice, research that goes on in the world where people simply move straight from research question to method. While this is not ideal, it is pragmatic. I think @leenie48 and I will have to agree to disagree on this one.

Why Not Include Theory?

theoryLast week I wrote a post about how to choose a research method. It received a fair amount of approval on social media, and a very interesting response from @leenie48 from Brisbane, Australia, with a couple of contributions from @DrNomyn. I’ve tidied up our exchange a little; it actually ended up in two threads over several hours, so wasn’t as neat as it seems here. I was travelling and in and out of meetings so undoubtedly didn’t give it the attention it deserved. I couldn’t embed the tweets without tedious repetition, so have typed out most of the discussion; our timelines are accessible if anyone feels the need to verify. Here goes:

EH: Your post suggests one can jump from rq to method choice with no consideration of theory. I disagree.

HK:I teach, and write for, students at different levels. Here in the UK masters’ students in many subjects have to do research with no consideration or knowledge of theory.

EH: Perhaps it might be useful to point out advice is for specific readers. Bit sick of having to explain to new phd students that this kind of advice is not for them!

HK: You’re right, and I am sorry for causing you so much inconvenience. I’ll re-tag all my blog posts, though that will take a while as there’s a sizeable archive.

HK: That seems unnecessarily pejorative. I don’t regard practice-based masters’ research as ‘pretend’, but as a learning opportunity for students. Commissioned research and practice-based research is professional rather than academic. Not wrong, simply different.

EH: Then why not include theory?

HK: I’ve explained why I didn’t include it in my blog post, so I’m not sure what you’re asking here?

And that’s where the discussion ended, with me confused as @leenie48’s question was on the other thread. Having put this into a single conversation, though, for the purposes of this post, it makes more sense. I think @leenie48 was asking why not include theory in masters’ level or practice-based research.

My conversation with @leenie48 might lead the uninitiated reader to believe that theory is a homogeneous ‘thing’. Not so. Theory is multiple and multifaceted. There are formal and informal theories; social and scientific theories; grand and engaged theories; Euro-Western and Southern theories. These are oppositional theory labels; there are also aligned options such as post-colonial and Indigenous theories.

I studied a module on social theory for my MSc in Social Research Methods, and used hermeneutic theory (a grand-ish formal Euro-Western social theory) for my PhD. Yet I don’t think I understood what theory is for, i.e. how it can be used as a lens to help us look at our subjects of study, until well after I’d finished my doctoral work.

If you’re doing academic research, theory can be very useful. Some, like @leenie48, may argue that it is essential. It is certainly a powerful counter when you’re playing the academic game. Yet theory is, like everything, value-laden. At present, in the UK, the French social theorist Bourdieu is so fashionable that the British Sociological Association is often spoken of, tongue in cheek, as the Bourdieu Sociological Association. At the other extreme, social theories from the Southern hemisphere are often ignored or unknown. So I would argue that if we are to include theory, we need to engage with the attributes of the theory or theories on which we wish to draw, and give a rationale for our choice. I find it frustrating that so much of academia seems to regard any use of theory as acceptable as long as there is use of theory, rather than questioning why a particular theory is being used.

This kind of engagement and rationale-building takes time and a certain amount of academic expertise. If you’re doing research for more practical reasons, such as to obtain a masters’ degree, evaluate a service, or assess the training needs of an organisation’s staff, theory is a luxury. These kinds of research are done with minimal resources to achieve specific ends. I don’t think this is, as @leenie48 would have it, ‘pretend research’. For sure it’s not aiming to contribute to the global body of knowledge, but I can see the point in working to discover particular information that will enable certain people to move forward in useful ways.

I have still to tackle two other points raised by @leenie48: the ‘methodology vs method’ question, and the issue of writing for masters’ students vs doctoral students on this blog and elsewhere. So that’s my next two blog posts sorted out then!

How To Choose A Research Method

chooseBecause you do things in a sensible order, you have your research question, right? Good. It’s very important to have that first. The method (or methods) you choose should be the one (or ones) most likely to help you answer your question. You can’t figure out which methods are most likely to help if you don’t yet know what your question is. So if you’re actually not sure of your question, stop reading this RIGHT NOW and go settle your question, then come back and carry on reading.

OK, now you definitely have a question. You’ll probably have an idea of what kind of research method may be most help. (If you don’t, I recommend you get into the research methods literature, such as this book.) Let’s say you think your question could be answered most usefully by doing a bunch of interviews. Your next step is to think through the pros and cons of interviews as thoroughly as you can. You may find it helps to read relevant excerpts from the research methods literature. For example, here is a breakdown of the pros and cons of interviewing, taken from page 143 of my book Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide (2nd edn; Policy Press, 2017).

Pros Cons
Interviews yield rich data Interviews are time-consuming for researchers and participants
Face-to-face interviews let the interviewer include observational elements, e.g. from the participant’s appearance or body language, that are not available with other methods The researcher’s interpretation of the data from a face-to-face interview may be affected by the quality of the rapport they developed with their participant
Interviews can be conducted by telephone, which saves time and costs and increases anonymity Not everyone is comfortable using the telephone, and it can be harder to create a rapport over the phone than in person
An interview equivalent can be conducted by email, which avoids transcription and so saves time and money; this also helps in reaching some groups of people e.g. those with severe hearing impairment Conducting an ‘interview’ by email can make it more difficult to follow up interesting answers with supplementary questions
Interviewers can follow up interesting answers with supplementary questions Interviewers’ input can influence participants’ answers
Unstructured interviews can be particularly useful at the exploratory stage of a research project Unstructured interviews run the risks of missing important issues or degenerating into a general chat
Semi-structured interviews allow participants to participate in setting the research agenda, which may be more politically acceptable, lead to more useful data, or both Semi-structured interviews make it harder to compare data from different individuals or groups
Structured interviews enable clearer comparison of data from different individuals or groups Structured interviews require the question designer to be able to consider all the issues that are relevant to the participants
Recording data enables exact reproduction of someone’s words and pauses Transcribing interview data is time-consuming and expensive

This kind of thinking will help you to decide on your research method. Also, you will need to be pragmatic. For example, if you have a very tight deadline and no time or budget for transcription, then interviewing is not a good idea however much it might fit the research question. In such a case you would need to consider other methods. My book contains similar ‘pros and cons’ tables for using secondary data, questionnaires, focus groups, documents as data, observational data, visual data, and collecting data online. Of course this is not an exhaustive list, and if you’re considering using, say, mobile methods, soundscapes, or ethnography, you might need to construct a ‘pros and cons’ list of your own. To do this, you would need to read, watch videos, and talk to people with more knowledge about the method of interest.

Once you have established the pros and cons of the method, these need to be weighed against pragmatic considerations of available time, money, and other resources. This assessment will be different for each research project, in its own context; there are no hard-and-fast rules. But however you do an assessment like this, your results will always be better than those of someone who uses the method that first springs to mind. For sure you may end up using the method you first thought of, in which case you might say to me, Helen, what is the point of doing all that thinking? The point is you’ll be making a considered and informed decision to use the method. That means you’ll be able to justify your decision to readers, reviewers, tutors, supervisors, managers, examiners, or whoever else has an interest in your work.

Free Online Research Ethics Resources

freeAre you grappling with research ethics? If so, fear not, for there are numerous free resources online to help you. Here are some examples.

Ethical codes and guidelines

There are loads of ethical codes and guidelines online. For example, some countries have national codes of research ethics, such as the Australian Code for the Responsible Conduct of Research, or the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. This was developed in partnership between the Canadian Institutes of Health Research, the Natural Sciences and Engineering Research Council of Canada, and the Social Sciences and Humanities Research Council of Canada.

There are also codes of research ethics produced by Indigenous peoples who wish their own ethical principles to be followed by any researchers who wish to work with them. Examples of these include Te Ara Tika, Guidelines for Māori Research Ethics, from New Zealand, and the San Code of Research Ethics from South Africa.

Then there are professional and disciplinary codes of research ethics. Examples include the UK-based Market Research Society’s Code of Conduct, and the Code of Ethics of the Australian Association for Research in Education.

There must be a huge number of these kinds of codes and guidelines worldwide. They are not all the same, and the careful reader can find places where one code or guideline may contradict another. This is because of cultural (in its widest sense) differences in ideas of what is ethical. Nevertheless, they can be useful to read for learning, ideas, or of course specific contextual information.

Applying to a research ethics committee

If you have to apply to a research ethics committee for formal ethical approval, you might find it useful to see some other researchers’ successful application forms. You can find examples of these on The Research Ethics Application Database (TREAD), originally set up by Martin Tolich at Otago University in New Zealand and now hosted by The Global Health Network and the Social Research Association. This database holds copies of successful ethics applications from around the world which you can search and use for inspiration and learning. Applications are anonymised, though the researcher(s) must be named. Researchers often submit accompanying documents, such as consent forms and participant information sheets, which can be very useful to look through for ideas. The database managers are keen to add more applications, to help make formal ethical approval processes more accessible and less onerous. If you have an application you could submit, there is information on the website about how to share it via the database.

General guidance

The Research Ethics Guidebook is intended to provide general guidance for social scientists, but may also be useful for people from other fields. The Guidebook is supported by the UK’s Economic and Social Research Council, with the Researcher Development Initiative of the National Centre for Research Methods, and London University’s Institute of Education. Like TREAD, the Research Ethics Guidebook holds useful information about applying for formal ethical approval. However, it also covers other areas such as ethics in research design, conducting research, reporting, and dissemination. The Guidebook is ideal for reference at the start of a project, and also during research as unforeseen ethical dilemmas occur.

Ethics training

There are two free online courses in research ethics which are primarily geared towards health researchers and so focus heavily on participant wellbeing. Both have been through peer review and other quality assurance processes, and both offer certificates to students who complete the course successfully with a score of 80% or more. One is called Research Ethics Online Training and is adapted from an e-learning course and resource package designed and produced by the World Health Organisation. It contains 14 individual modules, plus resources in the form of a glossary, a “resource library” (aka bibliography), some case studies, examples of ethics guidelines, videos on research ethics, and links to other ethics websites. The second is Essential Elements of Ethics, adapted from an ethics tool kit created to support researchers at Harvard University in America. This course contains 11 modules, plus resources including a workbook and checklist of points to consider, and a discussion forum though this is not very active.

Free research ethics modules with a wider perspective are offered by Duke University in America. These cover topics such as cultural awareness and humility, ethical photography, power and privilege, and working with children. They are delivered through videos with transcripts also available.

Online research

For internet-based research, the Association of Internet Researchers has some useful resources free for download. The British Psychological Society offers Ethics Guidelines for Internet-mediated Research. The South East European Network for Professionalization of Media has produced Social Media Research: A Guide to Ethics.

Visual research

The International Visual Sociology Association has produced a Code of Research Ethics and Guidelines covering visual research.

Ethics of research publication

The Committee on Publication Ethics has a whole range of downloadable resources covering how to detect, prevent and handle misconduct, responsible publication standards for editors and authors, ethical guidelines for peer reviewers, and much more.

This list of resources is by no means exhaustive. There are loads more out there. It would be a huge task to identify them all. These are the ones I have found particularly useful. If there are any you like to use, which aren’t in this post, please add them in the comments below.

Cartoons, Comics, and Graphic Novels in Research and Academia #2

comics with catsMy last post on this topic argued that this is more of a movement than a moment. In fact it was even more than I knew (no doubt still is) as I’ve found out about a number of other resources and activities on the topic since then.

In my last post I mentioned the Journal of Graphic Novels and Comics, which is published by Taylor & Francis which is part of the multinational Informa plc, and so is paywalled, although there are a couple of open access articles. I have also found another paywalled journal, Studies in Comics, which has an open access issue from its archive. Ernesto Priego kindly reminded me, via Twitter, of The Comics Grid: Journal of Comics Scholarship which is open access and well worth exploring.

In September, The Annals of the Entomological Society of America published an open access article called Sequential Science: A Guide to Communication Through Comics by Carly Melissa Tribull. This is both interesting and useful, as it contains information and resources on how to make comics and use them in educational settings. I thank Jonathan O’Donnell from RMIT in Melbourne for bringing this to my attention.

Also, as well as the BA in Cartoon and Comic Arts offered by Staffordshire University, the UK also offers a Comics and Graphic Novel Masters degree at the University of Dundee. This is overseen by Professor Chris Murray whose main research area is comics.

The pocket-sized Graphic Guides have been around for a long time. They describe themselves as ‘comic book style’, but this isn’t really accurate in my view; they’re more like short guides illustrated with cartoons. Perhaps for this reason, I’ve never got on with them particularly well, though I know a lot of people love them. However, I’ve discovered an exception in Queer: A Graphic History. I’ve had a copy of this book for some time, and I think it’s terrific, but I only discovered last Friday that it’s part of the Graphic Guides series. This isn’t obvious because it’s in a larger format with a different type of cover. Queer is written by Meg-John Barker (aka MJ), who spoke about their book at an event I went to in London recently (see below for more on this). It is always interesting to hear from other authors about their processes, and about the delights and sadnesses, triumphs and pitfalls they have experienced along the way. I have a great deal of respect for MJ because they are able to make difficult concepts accessible in a way I admire and long to emulate. People at the event were talking about how much their students love Queer and how useful it is for teaching. I can see why; I would have loved it as an undergraduate maybe even more than I love it now.

If you’re into graphic medicine, there’s a tasty-looking conference coming up in Cambridge on 16-17 February 2018. It’s called Comic Epidemic: Cartoons, Caricatures and Graphic Novels, and the call for papers is out now for anyone from the social sciences or the medical humanities; deadline 15 December. Successful applicants will be offered two nights’ accommodation in Cambridge and up to £100 for travel costs.

The seminar series Look Who’s Talking: Eliciting the Voices of Children from Birth to Seven, funded by the University of Strathclyde and held earlier this year, produced visual minutes of the sessions. Some of the people involved with this series have also used cartoon storyboards in research, to investigate the perceptions of students aged 4-15 about learning something new. Both of these projects were led by Professor Kate Wall from the University of Strathclyde, who kindly alerted me to them via Twitter.

Last month I was fortunate to be able to attend the event An Agenda for Graphic Social Science at the Open University in Camden. This is where Meg-John Barker presented their work, along with several other interesting and interested people including a journal editor and a publisher as well as several scholars. This account of the event identifies five areas of activity – blog posts, networking event, curation of resources, academic event, and sharing/recording expertise – and calls for volunteers to help take them forward. If you want to play with the cool kids, get in touch and get involved. We’d love to have you.

I’m sure that’s not all… but it’s all for now. And enough, I think, to argue that these media are moving from the fringes towards the mainstream in research and academia. Even so, if you have anything to add, please do so in the comments.


Cartoons, Comics, and Graphic Novels in Research and Academia

Cartoons, comics, and graphic novels in research and academia are having a moment. Actually it’s a bit more than a moment, but before I go into that, let’s start with some definitions. In terms of the visual arts, as I understand it, a cartoon is generally understood to be a single drawing; a cartoon strip or a comic strip is a series of a few sequential cartoons. Comics and graphic novels are more interchangeable terms for longer works, though people tend to use ‘comics’ to refer to the more lightweight end of the spectrum or reading matter for children. Conversely, ‘graphic novels’ are viewed as more serious and adult. In fact, though, they’re essentially the same thing: a graphic art medium for storytelling.

Academics from a range of disciplines are beginning to realise that this form has a great deal to offer for research communication and teaching. It is taught in some universities, though usually bundled in with other arts techniques such as illustration or animation. Few universities are offering sequential graphic art as a stand-alone or interdisciplinary subject at present, though there are some exceptions at undergraduate and postgraduate levels in the UK and the US. For example, in the UK, Staffordshire University’s BA in Cartoon and Comic Arts has been running for some years now. In the US, the University of Oregon offers an interdisciplinary Comics and Cartoon Studies minor, and Minneapolis College of Art and Design offers a Comic Arts major option on their Bachelor of Fine Arts degree. Savannah College of Art and Design offers undergraduate and postgraduate degrees in sequential art, which are also available in Hong Kong and online. California College of the Arts offers a Master of Fine Arts degree in comics, as does the Center for Cartoon Studies in Vermont. No doubt there are others too, though I haven’t found any outside the US/UK. For example, there are no such courses in Australia at present, though the Sydney Comics Guild suggests keeping an eye on MIT in Melbourne, presumably because they may run one in time to come.

Although the UK has few courses, it does have other initiatives. For example, the Journal of Graphic Novels and Comics is based in the UK. This journal focuses on the production and consumption of comics in their cultural, institutional, and creative contexts. An international Research, Outreach and Pedagogy Network (ReOPeN) for graphic novels and comics is based at Lancaster University. A mini-conference on comics and graphic novels in academia was held in Kendal in October, part of the Lakes International Comic Art Festival. I was lucky enough to go to this and meet academics working with comics and graphic novels from as far afield as Perth, Australia, as well as closer to home. Then there’s this free event in London tomorrow (tickets still available!) on graphic social science, which I will also be attending. Next year in London the historians are holding a two-day conference on the pre-modern world in comics. And it’s not only the academics who are using comics: this video demonstrates the use of graphic art, animation, and puppets in health evaluation research.

Comics are also being used for communicating science and medicine. As these links suggest, there is also a wider international focus on graphic art in academia and research. For example, the PathoGraphics conference in Berlin last month looked at the use of comics to communicate about illness and disability, and a recent Twitterchat via the #MethodsMatter hashtag focused on work in this area from various African countries.

I’m sure there are many more examples. I was teaching writing for publication to staff at the University of Derby yesterday and I mentioned graphic novels. One staff member said she didn’t like them as a reader, but she thought she should take steps to overcome her dislike, because she understood they were gaining traction. Another staff member, Dr Katy Vigurs, had been involved in producing a comic about student debt with students in her previous role at Staffordshire University. There’s a lot of this about, and it’s growing. So I think comics and graphic novels in research and academia are not so much having a moment as becoming a movement.

How To Market Your Academic Journal Article


What can you do to stand out in such a crowd?

Last week I wrote about how to market your academic book. Journal articles, too, benefit from marketing. If you’ve ever had one published, you have probably had one or more emails from the publisher encouraging you to help market your article. It is in the publisher’s interest for you to help them with marketing, because higher visibility usually leads to more citations, and more citations (within two years of publication) help the journal concerned increase its impact factor. It may, though, be in your interest too.

Around two and a half million academic journal articles are published each year on this planet. Generally speaking, if you want your one journal article to be noticed and read among these millions, you need to help it along.

Marketing your journal article begins before you finish writing. A clear and descriptive title, and the most relevant keywords or phrases, will help your article to be visible online. Use as many keywords or phrases as the journal permits. If you find it difficult to come up with enough keywords or phrases, think about what your readers might search for. Also, make sure at least three or four of your keywords or phrases appear at least once in your abstract. This all helps to make your article easier for search engines to find.

The abstract, too, is important. It should tell a clear story in itself, and should include the key ‘take-away’ point you have made. To figure out what that is, it may help to think what headline a journalist would give to a piece based on your article. A well written and structured abstract will entice more people to read further.

Once your article is published, there is plenty more work to be done if you want your research to have an impact. Some publishers make your article free to access online for a specific period or give you a limited number of free eprints. Either way, you can advertise this through email, discussion lists, and social media. Talking of email, it can be useful to add a link to your article as part of your email signature. Also, you can add the link to any online presence, such as your institutional web page and your LinkedIn profile.

If you teach a course for which your article would be suitable material, add it to the reading list. Also, unless it’s open access, check whether your university library subscribes to the journal concerned; if not, recommend it to them.

It’s helpful to write a blog post about your research with a link to your article. This could be on your own blog (if you have one), or on a blog in your field with a wide readership, or on your publisher’s blog. Again, you can advertise the link through social media.

An infographic can be useful too, either as part of a blog post or as a stand-alone information source – or both. Information here on how to create an infographic.

You can make a short video abstract of your journal article and upload it to a video sharing site such as YouTube or Vimeo. This is becoming an increasingly popular way to share information, and there are some great examples online such as this one on social jetlag. It’s not difficult to do, and can be done using a smartphone; there is a tutorial here.

Another option is to create a press release to alert the mainstream media. This is generally only worth doing if the journal article contains information that will interest a lot of people. It also needs to be ‘newsworthy’, e.g. relevant to current news coverage, providing a new perspective on past news coverage, or coinciding with an anniversary. A press release is a short document, usually only a page or at most two, with a specific format; details here.

If this all sounds like a lot of extra work: it is. I’m not suggesting you should do everything listed above; nor am I suggesting that this post is exhaustive. But if you want to make your journal article visible to potential readers, you will almost certainly need to take one or more of these steps.

The Ethics of Research Evidence

Like so many of the terms used in research, ‘evidence’ has no single agreed meaning. Nor does there seem to be much consensus about what constitutes good or reliable evidence. The differing approaches of other professions may confuse the picture. For example, evidence that would convince a judge to hand down a life sentence would be dismissed by many researchers as anecdote.

evidenceGiven that evidence is such a slippery, contentious topic, how can researchers begin to address its ethical aspects? A working definition might help: evidence is ‘information or data that people select to help them answer questions’ (Knight 2004:1). Using that definition, we can look at the ethical aspects of our relationship with evidence: how we choose, use, and apply the evidence we gather and construct.

Evidence is often talked and written about as though it is something neutral that simply exists, like a brick or a table, to be used by researchers at will. Knight’s definition is helpful because it highlights the fact that researchers select the evidence they use. Evidence, in the form of facts or artefacts, is neither ethical nor unethical. But in the process of selection, there is always room for bias, and that is where ethical considerations come into play.

To choose evidence ethically, I would argue that first you need to recognise the role of choice in the process, and the associated potential for bias. Then you need to consider some key questions, such as:

  • What is the question you want to answer?
  • What are your existing thoughts and feelings about that topic?
  • How might they affect your choices about evidence?
  • What can you do to make those choices open and defensible?

The aim is to be able to demonstrate that you have chosen the information or data you intend to define as ‘evidence’ in as ethical a way as possible.

Once you have chosen your evidence, you need to use it ethically within the research process. This means subjecting all your evidence to rigorous analysis, interpreting your findings accurately, and reporting in ways that will communicate effectively with your audiences. These are some of the key responsibilities of ethical researchers.

Research is a process that converts evidence into research evidence. It starts with the information or data that researchers choose to use as evidence, which may be anything from statistics to artworks. Then, through the process of (one would hope) diligent research, that evidence becomes research evidence. Whether and how research evidence is applied in the wider world is the third ethical aspect.

Sadly, there is a great deal of evidence that evidence is not applied well, or not applied at all. Most professional researchers have tales to tell of evidence being buried by research funders or commissioners. This seems particularly likely where findings conflict with political or money-making ambitions. In some sectors, such as third sector evaluation, this is widespread (Fiennes 2014). How can anyone make an evidence-based decision if the evidence collected by researchers has not been converted into evidence they can use?

The use of research evidence is often beyond the control of researchers. One practical action a researcher can take is to suggest a dissemination plan at the outset. This can be regarded as ethical, because such a plan should increase the likelihood of research evidence being used. But it could also be regarded as manipulative: using the initial excitement around a new project to persuade people to sign up to a plan they might later regret.

It seems that ethics and evidence are uneasy bedfellows. Again, Knight tries to help us here, by suggesting that research evidence should be used by people with expertise. This raises a further, pertinent question: what is the ethics of expertise? I will address that next week.

A version of this article was originally published in ‘Research Matters’, the quarterly newsletter for members of the UK and Ireland Social Research Association.

Why Research Participants Rock

dancingI wrote last week about the creative methods Roxanne Persaud and I used in our research into diversity and inclusion at Queen Mary University of London last year. One of those was screenplay writing, which we thought would be particularly useful if it depicted an interaction between a student and a very inclusive lecturer, or between a student and a less inclusive lecturer.

I love to work with screenplay writing. I use play script writing too, sometimes, though less often. With play script writing, you’re bound by theatre rules, so everything has to happen in one room, with minimal special effects. This can be really helpful when you’re researching something that happens in a specific place such as a parent and toddler group or a team sport. Screenplay, though, is more flexible: you can cut from private to public space, or include an army of mermaids if you wish. Also, screenplay writing offers more scope for descriptions of settings and characters, which, from a researcher’s point of view, can provide very useful data.

Especially when participants do their own thing! Our screenplay-writing participants largely ignored our suggestions about interactions between students and lecturers. Instead, we learned about a south Asian woman, the first in her family to go to university, who was lonely, isolated, and struggling to cope. We found out about a non-binary student’s experience of homophobia, sexism and violence in different places on campus. We saw how difficult it can be for Muslim students to join in with student life when alcohol plays a central role. Scenes like these gave us a much richer picture of facets of student inclusion and exclusion than we would have had if our participants had kept to their brief.

Other researchers using creative techniques have found this too. For example, Shamser Sinha and Les Back did collaborative research with young migrants in London. One participant, who they call Dorothy, wanted to use a camera, but wasn’t sure what to capture. Sinha suggested exploring how her immigration status affected where she went and what she could buy. Instead, Dorothy went sightseeing, and took pictures of Buckingham Palace. The stories she told about what this place and experience meant to her enriched the researchers’ perceptions of migrant life, not just the ‘aggrieved’ life they were initially interested in, but ‘her free life’ (Sinha and Back 2013:483).

Katy Vigurs aimed to use photo-elicitation to explore different generations’ perceptions of the English village where they lived. She worked with a ladies’ choir, a running club, and a youth project. Vigurs asked her participants to take pictures that would show how they saw and experienced their community. The runners did as she asked. The singers, who were older, took a few photos and also, unprompted, provided old photographs of village events and landmarks, old and new newspaper cuttings, photocopied and hand-drawn maps of the area with added annotations, and long written narratives about their perceptions and experiences of the village. The young people also took some photos, mostly of each other, but then spent a couple of hours with a map of the village, tracing the routes they used and talking with the researcher about where and how they spent time. Rather than standard photo-elicitation, this became ‘co-created mixed-media elicitation’ as Vigurs puts it (Vigurs and Kara 2016:520) (yes, I am the second author of this article, but all the research and much of the writing is hers). Again, this provided insights for the researcher that she could not have found using the method she originally planned.

Research ethics committees might frown on this level of flexibility. I would argue that it is more ethical than the traditional prescriptive approach to research. Our participants have knowledge and ideas and creativity to share. They don’t need us to teach them how to interact and work with others. In fact, our participants have a great deal to teach us, if we are only willing to listen and learn.

Creative Research In Practice

like cloudIt’s not often I get to share an output from the commissioned research I do. Sometimes clients don’t want to share publicly for reasons of confidentiality, and sometimes there are other reasons they don’t publish. As a commissioned researcher, I can’t publish the work someone else has paid for without their agreement. But I’m glad to say that Queen Mary University of London (QMUL) has published the full report of the research I did for them last year with my colleague Roxanne Persaud.

The research question was: How can QMUL improve students’ experience with respect to the inclusivity of their teaching, learning, and curricula? The original brief focused on the protected characteristics covered by the UK Equality Act 2010: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion and belief, and sexual orientation. Roxanne and I advised QMUL to take a more holistic approach to inclusivity, as the protected characteristics don’t cover some factors that we know can lead to discrimination and disadvantage, such as socioeconomic status and caring responsibilities. We recommended Appreciative Inquiry as a methodological framework, because it doesn’t start from a deficit perspective emphasising problems and complaints, but focuses on what an organisation does well and what it could do better. (It doesn’t ignore or sideline problems and complaints, either; it simply starts from the standpoint that there are assets to build on.)  And of course we suggested creative techniques, particularly for data-gathering and sense-making, alongside more conventional methods.

Roxanne and I were both keen to do this piece of work because we share an interest in diversity and inclusion. Neither of us had worked with QMUL before and we weren’t sure whether they would appreciate our approach to their brief. Sometimes commissioners want to recruit people who will do exactly what they specify. Even so, I’d rather say how I think a piece of work needs to be done; if the commissioner doesn’t want it done that way, then I don’t want the job.

QMUL shortlisted six sets of applicants. The interview was rigorous. Roxanne and I came out feeling we’d done ourselves justice, but with no clue as to whether we might have got the work or not. But we did!

The research was overseen by a Task & Finish group, made up of staff from different departments, who approved the methods we had put forward. We conducted a targeted literature review to identify key issues and best practice for inclusivity in the UK and overseas, and set the research in an institutional, societal, and theoretical context. The theoretical perspectives we used began with the theory of intersectionality developed by the law professor Kimberlé Crenshaw, which we then built on using the diffraction methodology of the physicist and social theorist Karen Barad. These two theories together provided a binocular lens for looking at a very complex phenomenon.

The timescale for the research was tight, and data gathering collided with Ramadan, exams, and the summer holidays. So, not surprisingly, we struggled with recruitment, despite strenuous efforts by us and by helpful colleagues at QMUL. We were able to involve 17 staff and 22 students from a wide range of departments. We conducted semi-structured telephone interviews with the staff, and gave students the option of participating in face-to-face interviews or group discussions using creative methods. These methods included:

  • The life-sized lecturer: an outline figure on a large sheet of paper, with a label indicating what kind of person they are e.g. ‘a typical QMUL lecturer’ and ‘an ideally inclusive lecturer’, which students could write and draw on.
  • Sticker maps: a map of organisational inclusivity, which we developed for QMUL, on which students could place small green stickers to indicate areas of good practice and small red stickers to indicate areas for further improvement.
  • Empathy maps: tools to help participants consider how other students or staff in different situations think and feel; what they might see, say, and do; and where they might experience ‘pain or gain’ with respect to inclusive learning.
  • Screenplay writing: a very short screenplay depicting an interaction between a student and a very inclusive lecturer, or between a student and a less inclusive lecturer. The screenplay will include dialogue and may also include information about characters’ attributes, the setting, and so on.

We generated over 50,000 words of data, which we imported into NVivo. Roxanne and I spent a day working together on emergent data coding, discussing excerpts from different interviews and group sessions, with the aim of extracting maximum richness. Then I finished the coding and carried out a thematic analysis while Roxanne finished the literature review.

We wrote a draft report, and then had two ‘review and refine’ meetings for sense-making, which were attended by 24 people. The first meeting was with members of the Task & Finish group, and the second was an open meeting, for participants and other interested people. We presented the draft findings, and put up sheets on the walls listing 37 key factors identified in the draft report. We gave participants three sticky stars to use to indicate their top priorities, and 10 sticky dots to use to indicate where they would allocate resources. People took the resource allocation incredibly seriously, and it was interesting to see how collaboratively they worked on this. I heard people saying things like, ‘That’s important, but it’s already got five dots on, so I’m going to put another one here.’ I wish I could have recorded all their conversations! We did collect some further data at these meetings, including touch-typed notes of group discussions and information about the relative frequency of occurrence, and importance, of the 37 key factors. All of this data was synthesised together with the previously collected data in the final report and its recommendations.

The comparatively small number of participants was a limitation, though we did include people from all faculties and most schools, and we certainly collected enough data for a solid qualitative study. We would have liked some quantitative data too, but the real limitation was that most of the people we reached were already concerned about inclusivity. We didn’t reach enough people to be able to say with certainty whether this was, or was not, the case more widely at QMUL. Also, while none of our participants disagreed unduly with our methodology or methods, others at QMUL may have done so. In a university including physicists, mathematicians, engineers, social scientists, artists, doctors, dentists and lawyers, among others, it seems highly unlikely that anyone could come up with an approach to research that would receive universal approval.

Yet I’m proud of this research. It’s not perfect – for example, I’ve realised, in the course of writing this blog post, that we didn’t explicitly include the research question in the research report! But its title is Inclusive Curricula, Teaching, and Learning: Adaptive Strategies for Inclusivity, which seems clear enough. I’m sure there are other ways it could be improved. But I’m really happy with the central features: the methodology, the methods, and the flexibility Roxanne and I offered to our client.