Methodology, Method, and Theory

debatingLike last week’s post, this one was inspired by @leenie48 on Twitter. My post of the week before was on how to choose a research question, and @leenie48’s view was that I should not tackle that topic without considering theory. Last week’s post dealt with why I didn’t include theory in the previous post (I hope you’re all keeping up at the back). This week’s post, as promised, explains why I think theory sits with methodology rather than with method.

Some people think ‘methodology’ is just a posh word for ‘method’. This is a bit like how some people think ‘statistical significance’ is a more important version of ordinary everyday ‘significance’. As in, it’s completely wrong.

Methods are the tools researchers use to practice our craft: to gather and analyse information, write and present findings. We have methods for searching literature and sources, gathering and analysing data, reporting, presenting, and disseminating findings. Methodologies are the frameworks within which we do all of this work, and are built from opinions, beliefs, and values. These frameworks guide us in selecting the tools we use, though they are not entirely prescriptive. Therefore one method, such as interviewing, may be used for research within different methodologies, such as realist evaluation or feminist research.

Here, as almost everywhere in the field of research methods, terminology is contested. But most people agree that there are several overarching categories of methodologies, such as post-positivist, constructivist, and interpretivist, and that within those overarching categories there are more specific methodologies, such as post-modernism and phenomenology. There are debates about what each category and methodology is, and how different methodologies should be used. These debates are mostly based on theory.

As I explained last week, theory also comes in many forms and is widely debated. These kinds of debates keep some academics in full-time work and are much too complex to summarise in a blog post. What I can say here is that @leenie48 and I disagree on a fundamental point. She thinks it is not an option to ‘jump from rq [research question] to method choice with no consideration of theory’. I know it is an option because I have seen it done many times, and have done it myself as an independent researcher working on commission for clients who are not interested in considering theory or in paying me to consider theory. The kind of briefs I often work to say, for example, ‘We want to know what our service users think about the service we provide, please do a set of interviews to find out.’ The commissioners don’t want a literature review or any explicit theoretical underpinnings, they simply want me to use my independent research skills to find out something they don’t know which will help them take their service forward. In a different context, I have taught and externally examined Masters’ level students, in subjects such as business studies and advice work, who are learning to do research. Their projects focus on method, not theory. It is as much as they can do, in their small word allocation, to contextualise their work, give a rationale for the method they have chosen, and describe and discuss their findings.

Masters’ level students in some other subjects would need to engage with theory, as I did in my own studies for MSc Social Research Methods, and I cannot imagine anyone doing research at doctoral level without using a theoretical perspective. I agree with @leenie48 that theory is important and has a lot to offer to research. In an ideal world, theory would form an equal part of a triad with research and practice.

In a comment on last week’s blog post, Sherrie Lee suggested that theory may be always present in some form, even if it is not explicitly considered. I think she makes a good point. I would like to use theory explicitly in all the research I do, rather than just some of it, but I am not sure that day will ever come. Much commissioned research isn’t explicit about methodology either. There is a lot of practice-based, and practice, research that goes on in the world where people simply move straight from research question to method. While this is not ideal, it is pragmatic. I think @leenie48 and I will have to agree to disagree on this one.

Why Not Include Theory?

theoryLast week I wrote a post about how to choose a research method. It received a fair amount of approval on social media, and a very interesting response from @leenie48 from Brisbane, Australia, with a couple of contributions from @DrNomyn. I’ve tidied up our exchange a little; it actually ended up in two threads over several hours, so wasn’t as neat as it seems here. I was travelling and in and out of meetings so undoubtedly didn’t give it the attention it deserved. I couldn’t embed the tweets without tedious repetition, so have typed out most of the discussion; our timelines are accessible if anyone feels the need to verify. Here goes:

EH: Your post suggests one can jump from rq to method choice with no consideration of theory. I disagree.

HK:I teach, and write for, students at different levels. Here in the UK masters’ students in many subjects have to do research with no consideration or knowledge of theory.

EH: Perhaps it might be useful to point out advice is for specific readers. Bit sick of having to explain to new phd students that this kind of advice is not for them!

HK: You’re right, and I am sorry for causing you so much inconvenience. I’ll re-tag all my blog posts, though that will take a while as there’s a sizeable archive.

HK: That seems unnecessarily pejorative. I don’t regard practice-based masters’ research as ‘pretend’, but as a learning opportunity for students. Commissioned research and practice-based research is professional rather than academic. Not wrong, simply different.

EH: Then why not include theory?

HK: I’ve explained why I didn’t include it in my blog post, so I’m not sure what you’re asking here?

And that’s where the discussion ended, with me confused as @leenie48’s question was on the other thread. Having put this into a single conversation, though, for the purposes of this post, it makes more sense. I think @leenie48 was asking why not include theory in masters’ level or practice-based research.

My conversation with @leenie48 might lead the uninitiated reader to believe that theory is a homogeneous ‘thing’. Not so. Theory is multiple and multifaceted. There are formal and informal theories; social and scientific theories; grand and engaged theories; Euro-Western and Southern theories. These are oppositional theory labels; there are also aligned options such as post-colonial and Indigenous theories.

I studied a module on social theory for my MSc in Social Research Methods, and used hermeneutic theory (a grand-ish formal Euro-Western social theory) for my PhD. Yet I don’t think I understood what theory is for, i.e. how it can be used as a lens to help us look at our subjects of study, until well after I’d finished my doctoral work.

If you’re doing academic research, theory can be very useful. Some, like @leenie48, may argue that it is essential. It is certainly a powerful counter when you’re playing the academic game. Yet theory is, like everything, value-laden. At present, in the UK, the French social theorist Bourdieu is so fashionable that the British Sociological Association is often spoken of, tongue in cheek, as the Bourdieu Sociological Association. At the other extreme, social theories from the Southern hemisphere are often ignored or unknown. So I would argue that if we are to include theory, we need to engage with the attributes of the theory or theories on which we wish to draw, and give a rationale for our choice. I find it frustrating that so much of academia seems to regard any use of theory as acceptable as long as there is use of theory, rather than questioning why a particular theory is being used.

This kind of engagement and rationale-building takes time and a certain amount of academic expertise. If you’re doing research for more practical reasons, such as to obtain a masters’ degree, evaluate a service, or assess the training needs of an organisation’s staff, theory is a luxury. These kinds of research are done with minimal resources to achieve specific ends. I don’t think this is, as @leenie48 would have it, ‘pretend research’. For sure it’s not aiming to contribute to the global body of knowledge, but I can see the point in working to discover particular information that will enable certain people to move forward in useful ways.

I have still to tackle two other points raised by @leenie48: the ‘methodology vs method’ question, and the issue of writing for masters’ students vs doctoral students on this blog and elsewhere. So that’s my next two blog posts sorted out then!

How To Choose A Research Method

chooseBecause you do things in a sensible order, you have your research question, right? Good. It’s very important to have that first. The method (or methods) you choose should be the one (or ones) most likely to help you answer your question. You can’t figure out which methods are most likely to help if you don’t yet know what your question is. So if you’re actually not sure of your question, stop reading this RIGHT NOW and go settle your question, then come back and carry on reading.

OK, now you definitely have a question. You’ll probably have an idea of what kind of research method may be most help. (If you don’t, I recommend you get into the research methods literature, such as this book.) Let’s say you think your question could be answered most usefully by doing a bunch of interviews. Your next step is to think through the pros and cons of interviews as thoroughly as you can. You may find it helps to read relevant excerpts from the research methods literature. For example, here is a breakdown of the pros and cons of interviewing, taken from page 143 of my book Research and Evaluation for Busy Students and Practitioners: A Time-Saving Guide (2nd edn; Policy Press, 2017).

Pros Cons
Interviews yield rich data Interviews are time-consuming for researchers and participants
Face-to-face interviews let the interviewer include observational elements, e.g. from the participant’s appearance or body language, that are not available with other methods The researcher’s interpretation of the data from a face-to-face interview may be affected by the quality of the rapport they developed with their participant
Interviews can be conducted by telephone, which saves time and costs and increases anonymity Not everyone is comfortable using the telephone, and it can be harder to create a rapport over the phone than in person
An interview equivalent can be conducted by email, which avoids transcription and so saves time and money; this also helps in reaching some groups of people e.g. those with severe hearing impairment Conducting an ‘interview’ by email can make it more difficult to follow up interesting answers with supplementary questions
Interviewers can follow up interesting answers with supplementary questions Interviewers’ input can influence participants’ answers
Unstructured interviews can be particularly useful at the exploratory stage of a research project Unstructured interviews run the risks of missing important issues or degenerating into a general chat
Semi-structured interviews allow participants to participate in setting the research agenda, which may be more politically acceptable, lead to more useful data, or both Semi-structured interviews make it harder to compare data from different individuals or groups
Structured interviews enable clearer comparison of data from different individuals or groups Structured interviews require the question designer to be able to consider all the issues that are relevant to the participants
Recording data enables exact reproduction of someone’s words and pauses Transcribing interview data is time-consuming and expensive

This kind of thinking will help you to decide on your research method. Also, you will need to be pragmatic. For example, if you have a very tight deadline and no time or budget for transcription, then interviewing is not a good idea however much it might fit the research question. In such a case you would need to consider other methods. My book contains similar ‘pros and cons’ tables for using secondary data, questionnaires, focus groups, documents as data, observational data, visual data, and collecting data online. Of course this is not an exhaustive list, and if you’re considering using, say, mobile methods, soundscapes, or ethnography, you might need to construct a ‘pros and cons’ list of your own. To do this, you would need to read, watch videos, and talk to people with more knowledge about the method of interest.

Once you have established the pros and cons of the method, these need to be weighed against pragmatic considerations of available time, money, and other resources. This assessment will be different for each research project, in its own context; there are no hard-and-fast rules. But however you do an assessment like this, your results will always be better than those of someone who uses the method that first springs to mind. For sure you may end up using the method you first thought of, in which case you might say to me, Helen, what is the point of doing all that thinking? The point is you’ll be making a considered and informed decision to use the method. That means you’ll be able to justify your decision to readers, reviewers, tutors, supervisors, managers, examiners, or whoever else has an interest in your work.

Free Online Research Ethics Resources

freeAre you grappling with research ethics? If so, fear not, for there are numerous free resources online to help you. Here are some examples.

Ethical codes and guidelines

There are loads of ethical codes and guidelines online. For example, some countries have national codes of research ethics, such as the Australian Code for the Responsible Conduct of Research, or the Canadian Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. This was developed in partnership between the Canadian Institutes of Health Research, the Natural Sciences and Engineering Research Council of Canada, and the Social Sciences and Humanities Research Council of Canada.

There are also codes of research ethics produced by Indigenous peoples who wish their own ethical principles to be followed by any researchers who wish to work with them. Examples of these include Te Ara Tika, Guidelines for Māori Research Ethics, from New Zealand, and the San Code of Research Ethics from South Africa.

Then there are professional and disciplinary codes of research ethics. Examples include the UK-based Market Research Society’s Code of Conduct, and the Code of Ethics of the Australian Association for Research in Education.

There must be a huge number of these kinds of codes and guidelines worldwide. They are not all the same, and the careful reader can find places where one code or guideline may contradict another. This is because of cultural (in its widest sense) differences in ideas of what is ethical. Nevertheless, they can be useful to read for learning, ideas, or of course specific contextual information.

Applying to a research ethics committee

If you have to apply to a research ethics committee for formal ethical approval, you might find it useful to see some other researchers’ successful application forms. You can find examples of these on The Research Ethics Application Database (TREAD), originally set up by Martin Tolich at Otago University in New Zealand and now hosted by The Global Health Network and the Social Research Association. This database holds copies of successful ethics applications from around the world which you can search and use for inspiration and learning. Applications are anonymised, though the researcher(s) must be named. Researchers often submit accompanying documents, such as consent forms and participant information sheets, which can be very useful to look through for ideas. The database managers are keen to add more applications, to help make formal ethical approval processes more accessible and less onerous. If you have an application you could submit, there is information on the website about how to share it via the database.

General guidance

The Research Ethics Guidebook is intended to provide general guidance for social scientists, but may also be useful for people from other fields. The Guidebook is supported by the UK’s Economic and Social Research Council, with the Researcher Development Initiative of the National Centre for Research Methods, and London University’s Institute of Education. Like TREAD, the Research Ethics Guidebook holds useful information about applying for formal ethical approval. However, it also covers other areas such as ethics in research design, conducting research, reporting, and dissemination. The Guidebook is ideal for reference at the start of a project, and also during research as unforeseen ethical dilemmas occur.

Ethics training

There are two free online courses in research ethics which are primarily geared towards health researchers and so focus heavily on participant wellbeing. Both have been through peer review and other quality assurance processes, and both offer certificates to students who complete the course successfully with a score of 80% or more. One is called Research Ethics Online Training and is adapted from an e-learning course and resource package designed and produced by the World Health Organisation. It contains 14 individual modules, plus resources in the form of a glossary, a “resource library” (aka bibliography), some case studies, examples of ethics guidelines, videos on research ethics, and links to other ethics websites. The second is Essential Elements of Ethics, adapted from an ethics tool kit created to support researchers at Harvard University in America. This course contains 11 modules, plus resources including a workbook and checklist of points to consider, and a discussion forum though this is not very active.

Free research ethics modules with a wider perspective are offered by Duke University in America. These cover topics such as cultural awareness and humility, ethical photography, power and privilege, and working with children. They are delivered through videos with transcripts also available.

Online research

For internet-based research, the Association of Internet Researchers has some useful resources free for download. The British Psychological Society offers Ethics Guidelines for Internet-mediated Research. The South East European Network for Professionalization of Media has produced Social Media Research: A Guide to Ethics.

Visual research

The International Visual Sociology Association has produced a Code of Research Ethics and Guidelines covering visual research.

Ethics of research publication

The Committee on Publication Ethics has a whole range of downloadable resources covering how to detect, prevent and handle misconduct, responsible publication standards for editors and authors, ethical guidelines for peer reviewers, and much more.

This list of resources is by no means exhaustive. There are loads more out there. It would be a huge task to identify them all. These are the ones I have found particularly useful. If there are any you like to use, which aren’t in this post, please add them in the comments below.

Cartoons, Comics, and Graphic Novels in Research and Academia

Cartoons, comics, and graphic novels in research and academia are having a moment. Actually it’s a bit more than a moment, but before I go into that, let’s start with some definitions. In terms of the visual arts, as I understand it, a cartoon is generally understood to be a single drawing; a cartoon strip or a comic strip is a series of a few sequential cartoons. Comics and graphic novels are more interchangeable terms for longer works, though people tend to use ‘comics’ to refer to the more lightweight end of the spectrum or reading matter for children. Conversely, ‘graphic novels’ are viewed as more serious and adult. In fact, though, they’re essentially the same thing: a graphic art medium for storytelling.

Academics from a range of disciplines are beginning to realise that this form has a great deal to offer for research communication and teaching. It is taught in some universities, though usually bundled in with other arts techniques such as illustration or animation. Few universities are offering sequential graphic art as a stand-alone or interdisciplinary subject at present, though there are some exceptions at undergraduate and postgraduate levels in the UK and the US. For example, in the UK, Staffordshire University’s BA in Cartoon and Comic Arts has been running for some years now. In the US, the University of Oregon offers an interdisciplinary Comics and Cartoon Studies minor, and Minneapolis College of Art and Design offers a Comic Arts major option on their Bachelor of Fine Arts degree. Savannah College of Art and Design offers undergraduate and postgraduate degrees in sequential art, which are also available in Hong Kong and online. California College of the Arts offers a Master of Fine Arts degree in comics, as does the Center for Cartoon Studies in Vermont. No doubt there are others too, though I haven’t found any outside the US/UK. For example, there are no such courses in Australia at present, though the Sydney Comics Guild suggests keeping an eye on MIT in Melbourne, presumably because they may run one in time to come.

Although the UK has few courses, it does have other initiatives. For example, the Journal of Graphic Novels and Comics is based in the UK. This journal focuses on the production and consumption of comics in their cultural, institutional, and creative contexts. An international Research, Outreach and Pedagogy Network (ReOPeN) for graphic novels and comics is based at Lancaster University. A mini-conference on comics and graphic novels in academia was held in Kendal in October, part of the Lakes International Comic Art Festival. I was lucky enough to go to this and meet academics working with comics and graphic novels from as far afield as Perth, Australia, as well as closer to home. Then there’s this free event in London tomorrow (tickets still available!) on graphic social science, which I will also be attending. Next year in London the historians are holding a two-day conference on the pre-modern world in comics. And it’s not only the academics who are using comics: this video demonstrates the use of graphic art, animation, and puppets in health evaluation research.

Comics are also being used for communicating science and medicine. As these links suggest, there is also a wider international focus on graphic art in academia and research. For example, the PathoGraphics conference in Berlin last month looked at the use of comics to communicate about illness and disability, and a recent Twitterchat via the #MethodsMatter hashtag focused on work in this area from various African countries.

I’m sure there are many more examples. I was teaching writing for publication to staff at the University of Derby yesterday and I mentioned graphic novels. One staff member said she didn’t like them as a reader, but she thought she should take steps to overcome her dislike, because she understood they were gaining traction. Another staff member, Dr Katy Vigurs, had been involved in producing a comic about student debt with students in her previous role at Staffordshire University. There’s a lot of this about, and it’s growing. So I think comics and graphic novels in research and academia are not so much having a moment as becoming a movement.

Indigenous Research Methods: A Reading List

Indigenous methods booksLast week I wrote about challenging the dominance of English in writing for research and academia. That theme is also relevant to this post, though here it’s more about challenging Euro-Western epistemologies and methods than the English language itself. Over the last year I have built a personal library of books about, or relevant to, my investigation of Indigenous research methods and ethics. The point of this, for me, is to bring these methods into my scholarship, alongside creative and conventional methods, as appropriate. The point is not to become an ‘expert’ on Indigenous research; for a white British person, that is not, should not be, an option. At the start of this work, I worried about being extractive, but I found comfort in the words of Margaret Kovach, an Indigenous researcher from Saskatchewan in Canada, who encourages non-Indigenous scholars to help make space for Indigenous methodologies and assess their value on their own terms. This is what I am trying to do.

For those who are new to this topic, ‘Indigenous’ denotes the native peoples of colonised lands, such as Aboriginal Australians or Inuit Alaskans, while ‘indigenous’ denotes the native peoples of non-colonised lands. So I am an indigenous Brit who will never be an Indigenous researcher. Some people described as Indigenous are unhappy with the term because they feel that it makes them seem like one homogeneous group, whereas in fact there is tremendous diversity. For example, there are hundreds of tribal and language groupings in Australia alone. However, as it is the term most commonly used in the literature, I’m sticking with it for now.

The first book is the foundational Decolonizing Methodologies by Linda Tuhiwai Smith, a Maori researcher from New Zealand. In fact I bought the first edition of this soon after it came out in 1999, the year I began my MSc in Social Research Methods. The second edition came out in 2012. This book shows how research was used as a tool of imperialism to help subjugate colonised peoples through, among other things, complete disregard for Indigenous knowledges and Indigenous peoples’ own research methods. It highlights the value of these knowledges and methods, and calls for research to be linked explicitly with social justice.

Shawn Wilson is an Opaskwayak Cree researcher from Canada who has also lived and worked with Indigenous peoples in Alaska and Australia, as well as spending time with Indigenous peoples in New Zealand, Morocco, and elsewhere. His book, Research Is Ceremony: Indigenous Research Methods (2008), is based on his doctoral research and describes a paradigm shared by Indigenous researchers in Canada and Australia. It’s not easy to get hold of; I tracked down a Canadian bookseller who seems to have bought up the last available copies, and I fear it may be going out of print, which would be a great shame as it is readable and insightful. UPDATE: The publisher emailed me in January 2018 to say it’s not out of print (hurrah!) and it is now available through the link above.

Margaret Kovach is a Plains Cree and Salteaux researcher from Canada whose Indigenous Methodologies: Characteristics, Conversations, and Contexts came out in 2009. Her book covers epistemologies, methods, and ethics. It is a work of considerable scholarship that is also accessible and full of wisdom.

Bagele Chilisa is a Professor at the University of Botswana. Her book Indigenous Research Methodologies (2012) gives an uncompromising and international account of some of the theories, epistemologies, ontologies and methods used by Indigenous researchers. While no book on this subject could be completely comprehensive, Chilisa makes a good job of showing the diversity, as well as some of the commonalities, of Indigenous methodology.

Donna Mertens from the US, Fiona Cram from New Zealand, and Bagele Chilisa have edited a collection called Indigenous Pathways into Social Research: Voices of a New Generation (2013). They have contributions from Indigenous researchers from all around the world: Vanuatu, Mexico, Cameroon, Hawai’i, Alaska, Papua New Guinea, and many other countries. These are fascinating accounts, highlighting personal, political, and ethical challenges, and how they have been overcome. They also say a lot about Indigenous methodologies around the world.

Also in 2013, Maggie Walter, a trawlwoolway researcher from Tasmania, and Chris Andersen, a Métis researcher from Canada, brought out Indigenous Statistics: A Quantitative Research Methodology. This book demonstrates the pervasiveness of Euro-Western thought in the construction of statistical research, using national censuses for ilustration. It offers a framework for Indigenous quantitative research, nayri kati or ‘good numbers’, which places an Indigenous standpoint at the centre. There is a short video online of Maggie Walter talking about Indigenous quantitative research.

Lori Lambert is a Mi’kmaq researcher from north-eastern Canada who has also worked with Indigenous peoples from Montana, US; northern Manitoba, Canada; and Queensland, Australia. Her book, Research for Indigenous Survival: Indigenous Research Methodologies in the Behavioral Sciences, was published in 2014. To the best of my knowledge, this is the first book to position Indigenous methods within a Euro-Western disciplinary category. Like other Canadian writers, such as Wilson and Kovach (above), Lambert includes the voices of people she has worked with alongside her own in her narrative.

Another essential text, though not specifically about research methods, is Southern Theory by Australian academic Raewyn Connell (2009). This book is subtitled ‘The global dynamics of knowledge in social science’ and in my view is essential reading for anyone engaging with social theory. During my MSc, I was taught social theory as the preserve of dead white men, and I am sure this is still being taught in many Euro-Western universities today. Connell’s book gives the lie to this approach.

This list is not exhaustive; it is just my personal library. One limitation is that I can’t afford expensive books. While I was writing this blog post, I had a message from my friend and colleague Roxanne Persaud, alerting me to Susan Strega and Leslie Brown’s edited collection Research as Resistance: Revisiting Critical, Indigenous, and Anti-Oppressive Practices (2nd edn 2015). I would love to read this book, but even the paperback is over £60 which puts it out of my reach.

These books are not comfortable reads for Euro-Western scholars, but they are hugely important. We need to know how research has been, and is, misused by Euro-Western cultures in order to learn how to use it better. Indigenous scholars are extraordinarily generous in their assessment of the potential value of Euro-Western methodologies, even those methodologies that have been instrumental in stealing their lands and their cultures and traumatising generations of their peoples. Yet most Euro-Western researchers either ignore Indigenous research entirely, or conclude that Indigenous peoples must have picked up a few tricks from the colonisers. I’m not sure which is worse. Indigenous research methods pre-date Euro-Western research methods by tens of thousands of years, and there is a great deal that Euro-Western researchers can learn from these approaches.

Creative Methods for Evaluation: A Frustration

frustrationEvaluation is a particular type of applied research designed to assess the value of a service, intervention, policy or other such phenomenon. This is relevant to all of us as it forms the basis for many decisions about public service provision. Despite being applied research, evaluation also has a significant academic profile, with dedicated journals, many books, and university departments with professors of evaluation in countries around the world.

There are a range of types of, and approaches to, evaluation research. They all have some things in common: they start with the desired outcomes of the service, intervention etc; formulate indicators that would show those outcomes had been met; then collect data in line with those indicators and analyse it to identify the extent to which the outcomes have been met. So, for example, if a community service aims to reduce loneliness, they might decide that one indicator could be a reduction of reports of loneliness to community-based doctors and nurses, then work with health colleagues to collect information from health records before and after the provision of the service to show whether there was any difference. Evaluators also write recommendations for ways to improve the service, intervention etc. The intention is that these recommendations are implemented, then later reviewed in another cycle of evaluation research.

The basics of general research practice also apply to evaluation: plan thoroughly, collect and analyse data, produce written and other outcomes, and publish your findings. From time to time I teach a course called ‘Creative Research Methods for Evaluation’, usually as part of the UK and Ireland Social Research Association’s open training programme. All sorts of people come on this course: central and local Government researchers, charity researchers, health researchers, researchers in private practice, research funders – a real mix, which makes it great fun.

I know quite a bit about creative methods; after all, I wrote a book on the subject. I tell my students about arts-based methods, research using technology, mixed methods, and transformative research frameworks. We talk about when these methods are appropriate to use, and how they can work side-by-side with more established methods. I give them lots of examples of creative methods in use.

And here is the huge frustration. While I have plenty of examples of creative methods in practice, very few come from evaluation research. I have some examples from my own practice, though only as verbal stories because the written and other outputs are subject to client confidentiality. This is a big problem with evaluation research: because it is applied, i.e. often conducted by and for individual organisations, it is rarely published beyond its immediate audience. When it is published, it is often simply uploaded to a web page and so disappears into the depths of the internet. And if it is both published and findable, it is not likely to include the use of creative methods.

There are many examples of perfectly competent evaluations using well-established methods. However, evaluators today are working on complex projects and benefit from having more methodological options at their fingertips. I know my course helps, because former students have told me so, but during the course someone always asks why I’m not using examples from evaluation research. (Even though I explain this problem at the start!) I wish I could use such examples; I’m sure they’re out there; but even though I have searched, and asked, and searched again, I can’t find them. So this is by way of an appeal: do you know of any good resources that showcase creative methods in evaluation research? By ‘good resources’ I mean well written outputs or short engaging videos (3-4 minutes at most) that are not too basic, as my students are generally quite experienced. If you have anything to suggest, please let me know in the comments.

Writer’s Block Debunked

writers blockI don’t believe in writer’s block. I think it’s an umbrella term for a whole bunch of problems, each of which has a solution. For sure, writing is difficult, and it is perfectly possible to get stuck. That happens to me all the time. But a complete block? I don’t think so. Seems to me ‘writer’s block’ is a lazy catch-all term which conveniently shifts responsibility from the writer to the ‘block’. That’s certainly not going to help.

Here are six of the things I think may be really going on.

Fear of failure. Writing for submission or publication can be terrifying. You’re working towards putting your work out in the world where it – and to some extent you – will be judged. What if people don’t like your writing? What if they don’t like YOU? This kind of fear can be paralysing.

Fear of success. Success equals change, and change is frightening, because it’s a leap into the unknown. This can be equally paralysing.

Boredom. This is a higher risk for long-term projects, or writing that has been imposed on you – say, by your manager, or a departmental imperative – that you don’t want to do.

Perfectionism. Novice writers sometimes think that writing is (or should be, or is for some people) as easy as reading. It isn’t. To write, you have to be willing to write badly first, and then make it good at the editing stage, sometimes through many revisions.

Running out of steam with a particular avenue or genre. Minette Walters is a successful author of psychological thrillers who published 12 books between 1992 and 2007, then didn’t publish another book for 10 years. She had become bored with writing psychological thrillers, but her publisher wanted her to carry on, presumably because they sold well. Now she has found a new publisher and is bringing out a historical novel. This can apply for non-fiction and scholarly writers, too, particularly if you’ve been writing in a specific area for some time and long to change tack.

Self-sabotage. If you say, think, believe that you want to write, but you’re not writing, then you are in some way sabotaging your own desires. This is a common human trait and probably links back to fear of failure, or perhaps fear of success.

Here are five potential solutions.

Freewrite. I love the technique of freewriting, and so do the doctoral students I teach it to. Here’s what you do. Set yourself a prompt, which must be in the first person and active voice, such as:

What I want to say is…

In this chapter, I want to argue…

I am writing this [thesis/dissertation/article/report/etc] because…

Then write for five minutes without stopping or correcting your work. This is only for your eyes so it doesn’t matter how scrawly or mis-spelled it may be. If you hesitate during the five minutes, write the prompt again, more than once if you need to, until it leads you somewhere else. Then see what you’ve got. You may well have a new insight or a phrase or sentence that you can use in your project. More importantly, though, you’ll have a load of words on a page, which – in five minutes flat – gives the lie to ‘writer’s block’.

Set small targets. Some people prefer word targets; others prefer time targets. Either is fine. Choose a target that feels easily achievable: perhaps somewhere between 100-500 words, or 10-30 minutes. Then set a frequency: once a day is good to begin with, or twice if you can manage that. And stick to it.

Switch projects. Working on more than one project is great because if you get bored, or stuck, you can move to another. I didn’t have a single idea for a blog post this morning, so I worked on a short story for a while till I came up with the idea of writing about writer’s block.

Keep a daily journal. Writing about your life, or some aspect of your life, for your eyes only, is a great way to convince your subconscious that you can write. This can be structured, or unstructured, as you prefer. For example, you might want to keep a journal of your dreams, or a ‘resilience journal’ where you write down three things you are grateful for and three things you did well each day, or a ‘reflective journal’ where you record what you have learned that day. Or you might just want to write whatever you feel like writing.

Go for counselling. If fear of failure, or fear of success, are really getting in the way – and, for some people, they can – then find a counsellor or therapist who can help you work on this.

If you have other solutions to share, please add them in the comment box.

Concentric Circles for Gathering Research Data

concentric-circlesMost people, when they think about gathering primary data for research, think of the ‘Big Three’ methods: questionnaire surveys, interviews, and focus groups. This is rather limiting when there are so many other methods that can help to answer research questions. One example is concentric circles. They are used quite frequently in market research, and have also been used in social research for several decades, yet I am frequently surprised to come across people who have no knowledge of this approach.

Concentric circles are generally used to investigate people’s relationships with other people, objects, or phenomena. There are two main ways to use concentric circles. One way is to put something at the centre – in market research, usually a product or brand – and ask the participant to make a cross, or drag-and-drop a digital image, to a point within the circles that they feel represents their closeness to that product or brand. So, for me, if the brand was Nestlé I’d put myself right on the outer edge, whereas with a product such as the Co-op’s fairly traded wine, I’d be close to the centre.

The other option is to put the participant at the centre, and ask them to use the concentric circles to map relationships such as the people or agencies who support them, with those they feel closest to placed nearest to the centre and vice versa. This can be done in evaluation research as part of a pre-post design to document changes. For example, if part of the point of a service is to act as a gateway or signpost to other relevant agencies, it might be useful to ask users of that service to map their relationship with other agencies at the first point of contact, and again after a suitable period of time. That would help to assess the impact of the service in increasing users’ links with other agencies.

The concept of this is easy for participants to grasp, perhaps because the circles provide a recognisable structure but are not prescriptive. They are appealing because they give the participant a lot of choice and flexibility in how they respond. This avoids difficulties that can be raised by a standard type of research question such as, ‘How supportive is your spouse or partner?’ Some participants would find a question like this difficult to answer, say if their spouse or partner was unsupportive, or abusive, or recently deceased. If such a participant met this question in a survey, interview, or focus group, at best they would provide no or inaccurate data, and they could suffer serious embarrassment or upset. However, with the concentric circles, a participant can choose which of the people in their lives they include in the discussion.

Perhaps because of their ease and sensitivity, concentric circles have been used effectively in research with children, the elderly, and people with learning disabilities, among other groups. They are rarely the only method used, but they add an interesting and useful dimension to an investigation. In fact, the potential applications of concentric circles are many and varied. For example, they could be used in:

  • Nutritional research to explore people’s relationships with different foodstuffs
  • Tourism research to investigate people’s relationships with different modes of travel
  • Sports research to assess people’s relationships with different types of exercise

And that’s just off the top of my head.

Completed concentric circles can provide powerful illustrations for research reports and presentations, although care should be taken to ensure that participants and others who may be mentioned cannot be identified as a result. One option here is for a researcher to recreate a participant’s concentric circles with different names and/or with changes to other identifying details.

I’ve just put in a proposal to a potential client including concentric circles. I hope they like the idea. Do you use concentric circles? Or do you think you might use them in the future?

When A Contract Ends

finish lineI’m putting the finishing touches to the report of a research project that’s been running for the last 18 months. And then it’ll be over. Which is a bit sad, for a number of reasons.

First, the work is for a national organisation, but unusually that organisation is based close to where I live in the Midlands of England. So, unlike most, this job hasn’t involved a lot of travelling: much of the work has been done within half an hour’s drive of my office.

Second, I’ve been working with another researcher, a colleague I met for the first time on the day we went to be interviewed for this job. I liked him then and my respect and appreciation for him has grown throughout the project. He’s responsive, thoughtful, caring, creative, and generally a terrific collaborator. I will miss working with him.

Third, it’s been an interesting, complex project, evaluating a community-based advocacy service for older people with cancer. The work is multi-faceted and that makes it a real challenge to investigate it fully and come up with suitable recommendations for taking the work forward.

Fourth, it’s paid some of the bills. These kinds of longer-term contracts, that provide a basic level of income for a period of time, don’t come along so often but are invaluable for indie researchers.

Letting go of a project can be hard for anyone, but there are some specific areas of difficulty for indie researchers. Commissioners don’t think to get back in touch to tell us how our work is being used, and seem surprised if we email or phone to ask. We have very little say in how our work is disseminated, and sometimes it’s not disseminated at all, which can be really frustrating. And unlike our academic colleagues, we don’t have the requirement to publish that can keep the relationships formed during a project alive for months and years after completion.

So in many ways I’m sorry to see this contract end, but the pill is very thoroughly sugared by the new contract I landed earlier this month. Without that I think I’d be in deep mourning. But this time it really does feel as though, as one door is closing, another opens.