The Royal Statistical Society (RSS) believes that one of the UK’s most important university ranking systems, the Teaching Excellence Framework (TEF), is not meeting the standards of trustworthiness, quality and value that the public might expect. This means that the thousands of students who use TEF to guide their university choices are being misled. The RSS believes that the TEF does not meet the standards set by the UK Statistics Authority (UKSA) Code of Practice and is writing to its Office for Statistics Regulation (OSR) about this pressing issue.
In its letter to the OSR and accompanying evidence, which has also been submitted to a major TEF review (led by Dame Shirley Pearce), the RSS raises a number of issues with the TEF:
- Uncertainty is not handled coherently and consistently, nor communicated clearly.
- TEF awards are not necessarily comparable from one institution to another.
- The RSS has several technical concerns over the benchmarking component of TEF including concerns that TEF differences might be due to factors other than teaching quality, concerns regarding institutional game-playing and an egregious example of the `look elsewhere effect’.
- There is a lack of transparency in that it is not easy to discover the full specification and details on implementation of the TEF, which makes it hard to check the statistical assumptions underlying the process.
The letter concludes by asking the Office for Statistics Regulation to “consider the validity of the TEF, and to rule on whether TEF does actually provide the public with information which is trustworthy, of high quality and value.”
Professor Deborah Ashby said: “Many prospective university students rely on these rankings to help inform their choices about where to study. We are concerned that the TEF is not reliable enough to bear the weight of this and could be misleading young people making important life choices about where to study.”
RSS’s full submission to the TEF review and its letter to the Office for Statistics Regulation are attached to this press release. These documents contain further examples of the current system’s shortcomings and detailed concerns regarding sample sizes, the comparing of institutions, and both missing and non-reportable data used.