Skip to Content


Large Image

What is the relevance of the 2014 Research Excellence Framework to undergraduate applicants?

In the 2014 Research Excellence Framework (REF) the research of 52,061 academic staff from 154 UK universities was peer-reviewed by 36 subject panels comprising UK and international academics, and external users of research. The results were published on 18th December 2014 and will be used by the four UK HE funding bodies to allocate around £2 billion of research funds annually from 2015-16. Details are available at www.ref.ac.uk.

The REF gives no indication about the quality of teaching and some uni departments keep their major research players away from undergraduate teaching. Most universities, however, do not, and the REF gives a good indication of the strength, in a crucial area, of a department for those applicants who wish to apply to research-intensive universities in which, to a large extent, students are expected to teach themselves by using research-based methods. Not all applicants are suited to such methods and it is important to work on the premise that in university choice there is no such thing as “a good university”, as what is good for some students is certainly not good for others.

This is the seventh such research assessment: the others, called Research Assessment Exercises (RAE), were in 1986, 1989, 1992, 1996, 2001 and 2008. For the first time in 2014 the overall quality profile awarded to each submission is derived from three elements that were assessed:

  1. The quality of research outputs. As in 2008 (65% of the overall quality profile).
  2. The social, economic and cultural impact of research. A new feature (20%)
  3. The research environment. Assessed differently in 2008 (15%)

The results for 2014 have been produced as “overall quality profiles”, which show the percentage of research activity in each submission judged to have met each of the 4 quality levels 4* to 1* or to have been below 1*.

For the quality levels in terms of originality, significance and rigour the percentage in the UK overall is:

4* – world-leading:     30%
3* – internationally excellent but not world-leading:      46%
2* – internationally recognised:      20%
1* – nationally recognised:      3%
unclassified not nationally recognised:      1%

It should be noted that universities were not required to submit the work of all research-active staff, but the number of FTE (full-time equivalent) research-active staff submitted is shown against each subject and institution profile.

In order to make comparisons, Times Higher Education (THE) has produced a “grade-point average” score: a subject score of 4.0 would indicate that 100% of that institution’s work is judged to be 4*.

I am to an extent sceptical about such statistical results, and this scepticism has been reflected regularly in the THE and shared by REF and RAE participants whom I have met across the UK in recent years. For instance, the rush to get prominent researchers transferred from one university to another before the deadline has made the January “transfer window” in football look like a kick-about in the park. However, I do believe that the 2014 REF has value for prospective university applicants. Moreover, the fact that the THE league tables constructed from the results are based on peer review rather than suspect criteria and weightings attributed by non-specialists makes them more significant.

I think that a major benefit of the 2014 REF is that its results should warn us against stereotyping.

Some important conclusions are:

1. Cambridge, Oxford, LSE and Imperial remain the leading research institutions in the UK, but they are far from dominant in all subjects.

As in 1996, 2001 and 2008 RAEs, the “Big Four” still head the THE Table of Excellence.

Of universities offering a full range of subjects Cambridge and Oxford are dominant (they submitted respectively to 32 and 31 out of the 36 subject panels). Cambridge was ranked 1st or 2nd in 31% of its subject submissions, Oxford in 39%; Cambridge was ranked 3rd to 10th in 50% of its subject submissions, Oxford in 45%.

However, Cambridge was ranked outside the top 10 in 6 subjects (19%) and Oxford in 5 subjects (16%):

  • Cambridge: Archaeology (19th=); Politics and International Sts. (14th); Sociology (12th); English Language and Literature (25th=); Philosophy (14th); Music (30th=)
  • Oxford: Anthropology (14th); Modern Languages and Linguistics (32nd); English Language and Literature (13th); Theology and Religious Sts. (12th); Art and Design: History, Practice and Theory (36th=).

Of more specialist institutions Imperial and LSE (14 subject submissions each) are dominant: Imperial was ranked 1st or 2nd in 21% of its subject submissions, but was ranked 3rd to 10th in the remaining 79% of its submissions; LSE was ranked 1st or 2nd in 50% of its subject submissions, and 3rd to 10th in 29%.

In three subjects (21%) LSE was ranked outside the top 10: Mathematical Scis. (27th); Anthropology (15th); History (21st=).

It is notable that the profile of Essex, which also made 14 submissions (also mostly in Arts & Social Sciences), again rivals LSE in areas where the latter has been traditionally dominant:

  • Economics:      LSE 2nd, Essex 7th
  • Philosophy:     LSE 5th=, Essex 9th=
  • Politics:      LSE 2nd, Essex 1st
  • Sociology:     LSE 8th, Essex 7th

2. The term “Russell Group” is not synonymous with “the élite research universities”.

Many, but by no means all 24 Russell Group universities fill the top 24 places in the THE Table of Excellence.

The Russell Group consists of the following 24 universities/colleges: Birmingham, Bristol, Cambridge, Cardiff, Durham, Edinburgh, Exeter, Glasgow, Imperial, KCL, Leeds, Liverpool, LSE, Manchester, Newcastle, Nottingham, Queen Mary, Queen’s Belfast, Oxford, Sheffield, Southampton, UCL, Warwick, York. The group was founded in 1994 to represent their common interests through “an association of leading UK research-intensive universities”, although some of their publicity suggests that they are “the leading UK research-intensive universities”.

If one discounts two small specialist institutions (Institute of Cancer Research and London School of Hygiene & Tropical Medicine) which only made a combined total of 4 subject submissions, only 18 members of the Russell Group have got into the top 23 in the THE table. Universities not in the Russell Group feature prominently: Bath (12th=), Lancaster (16th=), St Andrews (19th=), East Anglia (21st). 24th place is shared by 2 non-Russell Group (Royal Holloway and Swansea) and 2 Russell Group (Newcastle and Nottingham), while the following Russell Group universities are further down the order: Exeter (28th), Birmingham (29th=), Liverpool (31st=), Queen’s Belfast (40th=).

About a quarter of the top 10 entries in the 36 subject tables are non-Russell Group universities: East Anglia, Essex and St Andrews are in the top 10 in 6 tables, Bath and Royal Holloway in 5. First place is claimed in 5 of the tables by Aberdeen, Bath, Essex, Reading and Strathclyde.

3. The overall quantity and standard of research submitted by “post-1992” universities varies greatly. Some are matching the standard of older universities.

Positions 54th-87th= in the THE table show an overlap between “old” and “new” universities which submitted to 10 or more panels. The overlap is greater than in 2008. In descending order (pre-1992 unis. underlined):

Ulster (54th=) and Open, Goldsmiths, Keele, Aberystwyth, Brighton, Roehampton, SOAS, Liverpool John Moores, Sheffield Hallam, Portsmouth, Plymouth, Manchester Met., Westminster, Northumbria, East London, Hull, West of England, Brunel, De Montfort, Oxford Brookes, Huddersfield, Birmingham City, Hertfordshire, Nottingham Trent, Middlesex, Salford & Lincoln (87th=).

4. Although there is remarkably little difference in the rank order of universities in the THE tables for the 2001 and 2008 RAEs and for the 2014 REF, especially in the top half of the tables, several universities have shifted position dramatically.

As fewer subject submissions are, in general, made by universities in the bottom half of the table, fluctuations there might be considered less significant. I have considered here only universities that have made 10 or more subject submissions.

The biggest climbers among pre-1992 universities between 2008 RAE and 2014 REF:

+26 Swansea (52→26; +39 since 2001)
+16 Cardiff (22→6; +2 since 2001)
+15 KCL (22→7; +16 since 2001)
+12 East Anglia (35→23; +12 since 2001)
+12 Heriot-Watt (45→33; +21 since 2001)
+10 Bangor (52→42; +8 since 2001)

The biggest fallers among pre-1992 universities between 2008 RAE and 2014 REF:

-30 SOAS (31→61; -31 since 2001)
-26 Salford (61→87; -48 since 2001)
-24 Essex (11→35; -25 since 2001)
-21 Loughborough (28→49; -10 since 2001)
-18 Kent (31→49; -3 since 2001)
-13 Aberystwyth (45→58; +2 since 2001)
-13 Hull (59→72; -19 since 2001)
-12 Brunel (63→75; -23 since 2001)
-10 Surrey (35→45; -20 since 2001)
-10 Sussex (30→40; -9 since 2001)

The biggest climbers among post-1992 universities between 2008 RAE and 2014 REF:

+25 Sheffield Hallam (88→63; +4 since 2001)
+20 Liverpool Hope (127→107; +23 since 2001)
+19 Liverpool John Moores (81→62; +24 since 2001)
+18 Edge Hill (124→106; +20 since 2001)
+13 Canterbury Christ Church (111→98; +20 since 2001)
+11 Northumbria (81→70; +29 since 2001)

The biggest fallers among post-1992 universities between 2008 RAE and 2014 REF:

-25 Hertfordshire (58→83; +10 since 2001)
-23 Derby (96→119; +8 since 2001)
-22 Leeds Beckett (88→110; -18 since 2001)

Although prospective undergraduate applicants and their advisers should in many cases benefit from careful consideration of the REF 2014 results and the websites of individual university departments, which give the detail of the research interests of academic staff, it is, of course, more important that they focus on teaching rather than research quality: eg how the curriculum is delivered, the nature and amount of contact time, the size of classes, the availability of academic staff for individual discussion.

My research among current students suggests that there are often very great differences between departments in the same university, in teaching as in research. Rely on blind brand-name loyalty at your peril!

Posted by Phillip Rogerson on Tue, 31 Jan 2017



comments powered by Disqus

Philip Rogerson read Classics at Oxford. He has taught in a wide range of institutions: from UN (refugee camps in Jordan) to university (Oxford). Philip has been an HMC/ISI inspector of schools and has was Director of Higher Education and Careers at Sherborne School from 1997 to 2017.

View Profile