This week’s big news in REF-land is that HEFCE have released the final criteria and working methods for the assessment panels. For those of you not living in acronym-land, this means that we finally know what the ground rules are for the big assessment exercise which will look at the work produced by UK universities since the last one, judge its relative worth, and use those judgements as a way to allocate research funding from the government. The process has been long and drawn-out, since the REF is the successor to the RAE (Research Excellence Framework rather than the Research Assessment Exercise, don’t ask me why they decided to change it, I think I was still an undergrad when that decision got made) and they’ve had to work out how precisely it’s going to differ.
The working criteria that interest me are those for Panel D, which covers the subpanels of Modern Languages and Linguistics; English Language and Literature; History; Classics; Philosophy; Theology and Religious Studies; Art and Design: History, Practice and Theory; Music, Drama, Dance and Performing Arts; and Communication, Cultural and Media Studies, Library and Information Management. So it’s sort of a broad church humanities panel. Each subject has its own specialist subpanel (so a ballerina won’t have to deal with the work of an Egyptologist, for instance); the central panel is, as far as I can tell, responsible for doing overview work and coordinating everything, which is reasonable enough.
One very important change from the original proposals not included in the Panel D guidelines, which I feel particularly strongly about, is that the REF have now decided that researchers may submit one fewer output per period of maternity leave taken – so basically, as opposed to having to submit four outputs (articles, books, chapters in books, etc.), if you’ve had a baby you only need to submit three. This is a vast improvement on the original proposal, which suggested that in order for an output to be waived, a researcher would need to have taken fourteen months off. As numerous researchers pointed out, that’s enough for two pregnancies, and very few academics take that amount of leave or are able to do so. I have to say, as one of the people who wrote in to point out the problems with the latter approach, I’m really pleased that common sense has won out here, given the opportunity it had to go horribly wrong. It’s nice to have something to be optimistic about.
So – to the panel D guidelines. The definition of what the Classics subpanel will look at seems fairly sensible – my work easily falls within its boundaries, as does the work of colleagues likely to be in the sort of department I inhabit (Byzantinists, Egyptologists and so forth). By the look of it, not only do panels have to submit topics, but also definitions of topics, which means that we are told that within the panel’s boundaries fall “comparative literature and such literature, literary theory, philosophy, political thought, material culture, art, film, performance, music, and such political, archaeological and other cultural activity as exploits in any way the history or cultural products of the Greek, Roman and Byzantine world” – which strikes me as a rather verbose way of definining classical reception, but it’s better that it shoud be on broad rather than the narrow side.
Also within the panel’s remit falls “the pedagogy associated with learning and teaching in the subjects listed here”, and I have to say that this and the later section specifically addressing pedagogical research came as a pleasant surprise to me. One of the problems with the REF is that it completely centralises research as the only important thing a university does, and so to perform well in the REF you have to prioritise your research above all else. To do so, however, means you fall into the trap of not prioritising your teaching above all else, and thus become the victim of the National Student Survey scores. These two competing and conflicting behemoths ultimately make balancing priorities and time management in academic life really quite tricky. I have long been of the opinion that it would be much healthier to have an assessment that acknowledged teaching and research as equally important parts of a university’s function. While explicitly acknowledging pedagogical research isn’t quite the same thing, at least it’s a way to get teaching into the REF, if only by a sneaky back door.
I will admit that I sniggered at point 53, “No output will be privileged or disadvantaged on the basis of the publisher, where it is published or the medium of its publication.” That’ll be why we’re all being told to go for the A-rated journals and nothing else because it’ll damage our REF performance, then. I know we don’t have it as bad as the sciences with their citation references (note the recent case of the journal which accepted an article on the proviso that it incorporated more citations of articles from said journal). But journal names and publishers still carry weight, and optimistic statements that they don’t are, I’m afraid, not particularly credible – although at least point 74 says “the sub-panels within Main Panel D will neither receive nor make use of any citation or bibliometric data to inform their judgements”. I think my scepticism on this one is just going to have to wait and see how things not published in the A journals do, as I’m a cynical old soul.
There does seem to be a push to enable works that somehow ‘deviate’ from the scholarly norm, whether that’s because they cross over multiple interdisciplinary boundaries or because they have multiple authors. This is surely a good thing, because scholarship is healthier if it can incorporate all models of writing and working, not just the lone-genius pattern; that said, I’ll be interested to see how the guidance is applied in practice.
I’m quite glad to see the introduction of the double-weighted submission, if only because it acknowledges that writing a book is bloody hard work and takes a lot of effort. The problem with the REF is that it rewards churning out articles – not that articles are a bad thing. A good article is much, much better than a bad book, as the article form is the perfect shape for communicating one neat, concise idea. But sometimes you need to spread out your argument a bit, which is why you need a book – and it’s good that that work can be recognised. Oh, and that the guidelines for the double-weighting don’t say it can only happen if you’ve worked on a book! They include possible reasons like “the collection and analysis of a considerable body of material” and “the use of primary sources which were especially extensive, complex or difficult to access”. So you can argue the case for doing something Really Bloody Tricky and Time Consuming, which seems to me like the sort of thing people doing, for instance, archaeology should be able to do. (And they let you submit back-ups in case the panel doesn’t think something counts double, which is surely just the sensible and humane thing to do anyway.)
Ah, and now we get to the question of what the criteria actually are – apparently outputs will be judged on originality, significance and rigour. Using the terms “world-leading”, “international” and “national”. Jolly good. There are then a lot of definitions about what makes work four, three, two and one star standard. Which, frankly, are a bit unnerving, although they do reward cutting edge work rather than derivative work, which is surely fair given the supposed goal of research. But this was all a feature of the RAE, and the arguments get frequently rehearsed, so while seeing it in print is giving me a bit of a wobbly moment, and I do wonder whether even classicists can judge whether my work will be an important or a recognised point of reference given the specialisation involved, I’m going to take deep breaths and move on.
Section 3D deals with the Dreaded Impact Factor. Oh, Impact, you have come a long way since the days when you seemed to be all about whether money could be made out of one’s research, but I am still cagey. Not to mention that for the early career scholar in the humanities, the potential for impact is limited because of the lack of influence over infrastructure and curriculum that more established scholars might use. (I’m hoping very much that Penelope Goodman will soon post over at her blog about her fantastic project researching popular perceptions of the emperor Augustus, which is a great example of what a scholar in a full-time position can do on this front.) What I will say is that I’m glad to see so much on potential examples of impact beyond the economic, particularly in a document which will be scoured by those panicking about what this new term means for them.
I’m skimming over the rest of the impact stuff, because it looks like the case study material needs to be prepared by the academic unit rather than the individual. Most academic units I know are already thinking hard about what their case studies are going to be. I’m fairly sure that if I do end up being part of the headcount for the REF, I won’t be an impact case study, because I’ve had neither the time nor the embeddedness to do that, so I will leave this aside for now and concentrate on it more when the next cycle comes around and I might be able to think about it strategically.
The last section looks at ‘research environment’ – which again looks like it’s a bigger management ‘how do you look after your researchers’ question. Which personally I can’t do anything about, and indeed am highly unlikely to be in a sufficiently senior position to affect by the next REF cycle. But I will say that’s it’s nice to see that the REF doesn’t want its researchers being put into the academic equivalent of the research sweatshop (at least allegedly). After reading through all of this, and all the expectations, I have to say that I’m grasping rather gratefully at that straw.
I don’t think this document has told me anything new. It’s clarified a few things, the criteria and guidelines are now clearly set out, and I can see how this will be incredibly useful for those people working on the impact side of things. But for me? It’s just telling me more or less what I already knew, which is that I need to think strategically about making sure I look like I can contribute to the REF at an appropriately high level if I want to have even a sniff of a chance at a permanent academic career. After all, if even very senior researchers find their plans being dictated by the REF requirements, what choice do us beginners have but to fall into line?