How To Deal With Reviewers’ Comments

editing textYour first set of reviewers’ comments lands in your inbox. Your heart begins to race. Will your work be accepted or rejected? Will they love it or hate it? Can you bear to open the email?

These may be reviews for a journal article, book proposal, or book typescript. In each case the process is the same. First you need to read the comments and give yourself time to react. Whether it’s the exultation of an acceptance, the despair of a rejection, or the mixed feelings that come with requests for revisions, you need time to process your emotional response before you do anything else. Whoop, cry, eat chocolate, do whatever you need to do.

Because of negativity bias, negative comments – even when constructively phrased – have more impact on most people than positive comments. We need to work to counteract this bias. So, unless you’ve received very favourable comments and you want to revel in their glory, I recommend waiting at least 24 hours before you read the comments again. This can help you to take a more balanced view, which is useful because if it’s a rejection or revisions, you’ll need to see how your work can benefit from the reviewers’ input before you send it off again. This can be quite a challenge, especially if the reviewers have different views of your work and how it can be improved. Your journal or commissioning editor may offer some guidance and if so you should take that into account. But sometimes they leave it all to you.

My solution to this is to treat the reviewers’ comments as data and go into analysis mode. I create a table with one column for the comments and another for each reviewer. Then I enter each substantive comment into the first column and put a mark in the other columns for each reviewer who has made a similar point. This helps me to pick up the instances where reviewers are effectively saying the same thing, though perhaps in very different ways. It also helps me to see at a glance which comments have been made by all or some reviewers, and which only by one of the reviewers.

I have said before on this blog that reviewers’ comments come in three categories: the no-brainer (act on this), the no-thanks (don’t act on this), and the oh-wait (probably act on this, though not necessarily in the way the reviewer suggests). So my next job is to sort comments into these categories.

If a comment has been made by more than one reviewer I will take it more seriously. That doesn’t mean I’ll definitely implement it, but I am more likely to do so. If a comment has been made by all reviewers I would need a very, very good reason not to implement it. If a comment has only been made by one reviewer, that in itself might be one reason I could decide not to implement it, though I would also expect to give at least one other reason.

Once I have sorted the comments into their categories, I will list them by category in the first column of another table with two further columns: a brief note of what I plan to do in response to each of the no-brainers and the oh-waits, and a brief note of what I plan to write in the cover letter against each comment from all three categories. This is useful because I can dip into it when I have a spare half-hour or so, and find a job or two to do to get me closer to the finish line.

It is important to be polite in your response to reviewers’ comments, even if you think they’re the biggest load of old rubbish you’ve seen since your last visit to the municipal tip. Some reviewers’ suggestions seem to be based more on what they would have written than on what you actually have written and this can be quite annoying at times. When you come across a suggestion you really don’t want to implement, there are some tactful ways to say so, such as:

“This is an excellent suggestion though unfortunately beyond the scope of this particular project.”

“I can see how this suggestion would improve my work but sadly I am unable to incorporate it within the allocated word count.”

“This is a really interesting idea. I have considered it carefully and concluded that it doesn’t quite fit with the thrust of my current article/book, but it will influence my thinking for future projects.”

Remember you are the author and, as such, you have authority. While authors do need reviewers’ input (at least, when it’s constructive), and your work should benefit from intelligent use of their feedback, you don’t have to do everything a reviewer says. Also, a rejection is only a rejection from this journal or publisher. It doesn’t mean your work is worthless; sometimes it’s only because they already have plans to publish something that is similar in some way. This post should help you make the best use you can of reviewers’ comments. That will produce the greatest benefit to your work and career, and is also a way to respect and honour the time and care (most) people put into writing reviews.

I Finished The Book!

Research ethics in the real world [FC]For the last three-and-a-quarter years I have been writing a book on research ethics. It has been like doing another PhD, only with reviewers instead of supervisors. Four sets of reviewers: two sets of proposal reviews and two sets of typescript reviews. I have to thank my lovely publisher, Policy Press (part of Bristol University Press), for giving me so much support to get this book right.

This has been the hardest book I’ve written and I hope never to write another as difficult. On the plus side, I’m happy with the result. It is different from other books on research ethics in three main ways. First, it doesn’t treat research ethics as though they exist in isolation. I look at the relationships between research ethics and individual, social, institutional, professional, and political ethics, and how those relationships play out in practice in the work of research ethics committees and in evaluation research. That makes up part 1 of the book.

Second, it demonstrates the need for ethical thinking and action throughout the research process. In part 2 there is a chapter covering the ethical aspects of each stage of the research process, from planning a research project through to aftercare. There is also a chapter on researcher well-being.

Third, the book sets the Indigenous and Euro-Western research paradigms side by side. This is not to try to decide which is ‘better’, but is intended to increase researchers’ ethical options and vocabularies. I am writing primarily for Euro-Western readers, though the book may be of use to some Indigenous researchers. There is a sizeable and growing body of literature on Indigenous research and ethics, including books, journals, and journal articles. Using this literature requires care – as indeed using all literature requires care (see chapter 7 of my forthcoming book for more on that). But Indigenous literature, as with other literatures by marginalised peoples, requires particular care to avoid tokenism or appropriation.

Many Euro-Western researchers are completely ignorant of Indigenous research. Some know of it but are under the misapprehension that it is an offshoot of Euro-Western research. In fact it is a separate paradigm that stands alone and predates Euro-Western research by tens of thousands of years. Some Indigenous researchers and scholars are now calling for Euro-Western academics to recognise this and use Indigenous work alongside their own. My book is, in part, a response to these calls.

It was so, so hard to cram all of that into 75,000 words – and that includes the bibliography which, as you can imagine, is extensive. There was so much to read that I was still reading, and incorporating, new material on the morning of the day I finished the book. I’ve found more work, since, that I’d love to include – but I had to stop somewhere.

I awaited my final review with great trepidation, aware of the possibility that the reviewer might loathe my book – some previous reviewers had – and that that could put an end to my hopes of publication. Was I looking at three-and-a-quarter years of wasted work? I was so relieved when my editor emailed to say the review was positive. Then the reviewer’s comments blew me away. Here’s one of my favourite parts: “In my view the author through excellent writing skills has covered very dense material (a ton of content) in a very accessible way.”

I was even more delighted because this review came from an Indigenous researcher. She waived anonymity, so I have been able to credit and thank her in the book. I will not name her here, as I do not have her permission to do so; you’ll have to read the book if you want to find out.

Finishing a book feels great, and also weird. It’s like losing a part of your identity, particularly with a book you’ve lived with for so long. Though there’s still lots of work to do: I have to write the companion website, give input on the book’s design, read the proofs, start marketing… publication is due on 1 November, which feels a long way off but I know how quickly five months can pass.

I think this book will be controversial. A senior and very knowledgeable academic told me that one reason I could write such a book is because I’m not in academia. I’m glad if I can use my independence to say things others cannot say – as long as I’m saying useful things, at least.

More than anything else, I hope the book helps to make a difference. In particular, I would like to make a difference to the current system of ethical regulation which is too focused on institutional protection and insufficiently focused on ethical research. It is also terrible at recognising and valuing the work of Indigenous research and of Euro-Western community-based or participatory research. When I was preparing to write the book, I interviewed 18 people around the world and promised them anonymity. Some were research ethics committee members and others had sought formal approval from ethics committees (or institutional review boards in the US). I heard tales of people completing ethical approval forms with information that committees wanted to see rather than with actual facts; people teaching students how to get through the ethical approval system instead of teaching them how to conduct ethical research; people acting ethically yet in complete contravention of their committee’s instructions; people struggling to balance ethical research with Indigenous communities with the inflexible conditions set by ethics committees. Although many of the people who serve on ethics committees are highly ethical, the system within which they are forced to work often prevents them from acting in entirely ethical ways. It seems to me that this system is not currently fit for purpose, and there are many other people who think the same. I hope the evidence I have gathered and presented will help to create much-needed change.

As an independent researcher, I am self-employed. This means I do all my writing in my own time; I don’t have a salary to support my work. Do you like what I do on this blog, or in my books, or anywhere else, enough that you might buy me a coffee now and again if we were co-located? If so, please consider supporting my independent work through Patreon for as little as one dollar per month. In exchange you’ll get exclusive previews of, and insights into, my work. Thank you.

Independent Research and Creative Methods

This week’s blog is a video. It’s a keynote I gave last month at a doctoral conference at the University of Birmingham. The conference organisers asked me to cover three topics: my career as an independent researcher, creative research methods in practice, and advice for anyone considering becoming an independent researcher.

The video was created and published by the Contemporary Philosophy of Technology research group at the University of Birmingham. You might want to get a cuppa… Enjoy!

How Open Is Open Access?

This article first appeared in Funding Insight on 18 January 2018 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com.

partly open doorThose outside the UK probably won’t be aware of Jisc. The non-profit organisation’s role is to provide technological solutions to academic problems, including researching and developing new ways of working supported by technology. (Full disclosure: they are also one of my clients.) Jisc is publicly funded by UK taxpayers’ money and member subscriptions. Its members are from the UK, and its objectives are designed to create benefit for the staff and students of adult education institutions in the UK.

But its strategy includes a stated intention of ‘growing our offering internationally to further benefit our members’.

Jisc is also very keen on open sharing of information and resources. It advocates open access to research publications, which its says means making them ‘freely available so anyone can benefit from reading and using research’. It promotes the sharing of research data, and the use of non-restrictive Creative Commons licensing to enable re-use of resources. Jisc identifies various potential benefits of this, one of which is that ‘researchers in developing countries can see your work’.

So far, so many good intentions. I’m sure most of my Euro-Western colleagues will be nodding their heads and thinking yes, marvellous, jolly well done Jisc. And indeed I am not writing this post to criticise those at Jisc, who are doing their best to be good guys, and who after all exist in the UK for the benefit of their UK members. My point here is to critique our more general Euro-Western academic mindset, which Jisc’s example illustrates.

You see, until very recently, I would have been one of those people nodding along, with a satisfied expression, thinking “oh Jisc you are doing well”. But my eyes have been opened by a recent blog post written by Andy Nobes of international development charity INASP, featuring the work of Florence Piron from Université Laval in Québec and her colleagues from around the world. Piron and her colleagues have written – in French – some publications that offer new perspectives and ideas to Euro-Western advocates of open access. In particular, they challenge the idea that Euro-Western researchers simply making their work visible to ‘researchers in developing countries’, as Jisc suggests, equates to open access. By contrast, they see it as an extension of colonialism and an ‘alienation epistemic’. This is because it does nothing to make knowledge generated in other parts of the world equally visible to researchers in Euro-Western countries. In turn, that serves to reinforce the use of Euro-Western theories and models as normative, which is to the detriment of local epistemologies in other parts of the world (Piron et al 2017).

We don’t think of these things, do we, us Euro-Western researchers? We’re too confident that we’re doing OK as long as we’re making some kind of gesture towards those with fewer privileges.

Piron and her colleagues point out that many Euro-Western academics are unable even to think that ‘valid and relevant knowledge’ could exist in other places and other ways; they can be ‘blind to epistemological diversity’ and regard Western science as universal (ibid). Even those Euro-Western academics who do respect other forms of knowledge are unlikely to engage in truly reciprocal knowledge exchanges. Collaborative projects often involve Euro-Western academics acting as principal investigators while researchers from other parts of the world are restricted to data-gathering and administrative work (Sherwood 2013, Yantio 2013).

In some Euro-Western academic circles there are moves afoot to ‘decolonise the curriculum’. But this is not only needed in Euro-Western establishments. Some teachers in other parts of the world also choose exclusively Euro-Western literature and examples for their students (Mboa Nkoudou 2016). This is a direct real-world consequence of the pervasive Euro-Western conviction that any other way of thinking must be inferior to our own. It makes life harder – not easier – for people in other parts of the world to solve their own local problems in appropriate and sustainable ways (ibid).

Truly open access will involve a two-way exchange of – and respect for – knowledge and the epistemological positions on which it is based. Obviously this is beyond the power of a single organisation, such as Jisc, or a single individual, such as you or I. However, all Euro-Western researchers, and those who work with them, need to be aware of the difference between open access as we tend to purvey it, and genuinely open access. Only with such awareness will we find ways to move from our one-way, take-it-or-leave-it approach to a true openness and sharing with other academics around the world.