Evaluation is a particular type of applied research designed to assess the value of a service, intervention, policy or other such phenomenon. This is relevant to all of us as it forms the basis for many decisions about public service provision. Despite being applied research, evaluation also has a significant academic profile, with dedicated journals, many books, and university departments with professors of evaluation in countries around the world.
There are a range of types of, and approaches to, evaluation research. They all have some things in common: they start with the desired outcomes of the service, intervention etc; formulate indicators that would show those outcomes had been met; then collect data in line with those indicators and analyse it to identify the extent to which the outcomes have been met. So, for example, if a community service aims to reduce loneliness, they might decide that one indicator could be a reduction of reports of loneliness to community-based doctors and nurses, then work with health colleagues to collect information from health records before and after the provision of the service to show whether there was any difference. Evaluators also write recommendations for ways to improve the service, intervention etc. The intention is that these recommendations are implemented, then later reviewed in another cycle of evaluation research.
The basics of general research practice also apply to evaluation: plan thoroughly, collect and analyse data, produce written and other outcomes, and publish your findings. From time to time I teach a course called ‘Creative Research Methods for Evaluation’, usually as part of the UK and Ireland Social Research Association’s open training programme. All sorts of people come on this course: central and local Government researchers, charity researchers, health researchers, researchers in private practice, research funders – a real mix, which makes it great fun.
I know quite a bit about creative methods; after all, I wrote a book on the subject. I tell my students about arts-based methods, research using technology, mixed methods, and transformative research frameworks. We talk about when these methods are appropriate to use, and how they can work side-by-side with more established methods. I give them lots of examples of creative methods in use.
And here is the huge frustration. While I have plenty of examples of creative methods in practice, very few come from evaluation research. I have some examples from my own practice, though only as verbal stories because the written and other outputs are subject to client confidentiality. This is a big problem with evaluation research: because it is applied, i.e. often conducted by and for individual organisations, it is rarely published beyond its immediate audience. When it is published, it is often simply uploaded to a web page and so disappears into the depths of the internet. And if it is both published and findable, it is not likely to include the use of creative methods.
There are many examples of perfectly competent evaluations using well-established methods. However, evaluators today are working on complex projects and benefit from having more methodological options at their fingertips. I know my course helps, because former students have told me so, but during the course someone always asks why I’m not using examples from evaluation research. (Even though I explain this problem at the start!) I wish I could use such examples; I’m sure they’re out there; but even though I have searched, and asked, and searched again, I can’t find them. So this is by way of an appeal: do you know of any good resources that showcase creative methods in evaluation research? By ‘good resources’ I mean well written outputs or short engaging videos (3-4 minutes at most) that are not too basic, as my students are generally quite experienced. If you have anything to suggest, please let me know in the comments.
Last week I wrote a post
The university sector in the UK is in a shaky state at present. Several
I don’t believe in writer’s block. I think it’s an umbrella term for a whole bunch of problems, each of which has a solution. For sure, writing is difficult, and it is perfectly possible to get stuck. That happens to me all the time. But a complete block? I don’t think so. Seems to me ‘writer’s block’ is a lazy catch-all term which conveniently shifts responsibility from the writer to the ‘block’. That’s certainly not going to help.
Well, whaddaya know? You wait ages for someone to show you a book proposal, then two come along together. Three, in fact, as there are two book proposals for you to download from this post.
When a book is first published, it’s a strange and slightly confusing time. Exhilaration, panic, longing and impostor syndrome all collide in a big smash of emotions. There’s often a formal launch – sometimes more than one – and an anxious wait for feedback from readers and the first review.![Creative research methods in the social sciences [FC]](https://helenkara.com/wp-content/uploads/2015/05/creative-research-methods-in-the-social-sciences-fc.jpg?w=236&h=330)
I have always loved being independent. My parents like to tell the story of the time when, soon after I learned to walk, they took me for a picnic in a local park. My father put me down on the grass, and I got to my feet and toddled away. My mother looked anxious, and my father said, reassuringly, ‘She won’t go far.’ But his confidence was misplaced, because I headed determinedly off into the wide green yonder, and he had to do a quick sprint to bring me back before I came to grief.
Yesterday I hit 10,000 words of the first draft of my research ethics book. That’s a huge milestone which has taken me two years to reach (though most of that was preparation – I started the actual writing earlier this year). I still have around 65,000 words to write, but having the first 10k safely on my computer and backed up is an enormous relief.
As the end of the financial year is imminent, it seems a good time to talk about money. Turns out I had a bumper year in 2015/16: my company made a profit of over £36,000. That’s the most this decade. Part of the reason was I won a good-sized research contract, which is the first time that’s happened in this decade. I won a lot of work in the 2000s, and did a fair bit of sub-contracting too, but since the change of government in 2010 I’ve mostly been the sub-contractee. Which is fine by me, actually; it’s a lot less hassle, and I like working in teams.
re work, instead I spend my time writing. And the books are beginning to yield something of an income, after six years of hard graft; I can now rely on a four-figure sum arriving each year, and that’s a month’s money, maybe two months this year if I’m lucky.