Ranking list content: What is being ranked?

Rankings lists are diverse: universities, fields of activities, subjects, programmes, nations

Ranking lists focus not only on universities as a whole but also on various fields of activities, e.g. teaching, research or executive education. There are also rankings of different nations. More specifically, there are comparative rankings of subject areas (e.g. biology, psychology, law) or rankings of Bachelor’s, Master’s or MBA (Master of Business Administration) and EMBA (Executive Master of Business Administration) programmes in the field of executive education.

Each ranking is addressed to a different audience.

While teaching rankings are addressed to aspiring and existing students, research rankings are addressed to the scientific community and funding organizations. For their part, executive education rankings are addressed to potential and existing participants. Political authorities and policy makers usually are interested in national rankings and rankings of whole universities. Universities, however, may have particular strengths in one field of activity, e.g. research or teaching, and weaknesses in others. Or they may even be focussed on specific areas, while not offering activities in many other areas. This poses particular challenges when it comes to university-wide rankings and comparisons between very different institutions. A crucial question is: how and to what an extent do ranking lists take these differences into account?

Comparison of the Shanghai and THES ranking lists

The Shanghai ranking list, for instance, only uses criteria related to research performance (number of alumni who earned Nobel Prizes and Fields Medals, number of publications in highly ranked journals, citations). For its part, the THES ranking list uses a diverse mix of criteria including research reputation (40%), research impact (20%), teaching quality (20%), reputation among global employers (10%) and internationalization of staff and students (10%). As can be seen in these two examples, the Shanghai concept of what constitutes a university is dramatically different from the concept used by the THES. While they both measure and rank the same universities, they clearly use very different ranking criteria. While the Shanghai ranking list is basically a research ranking, it is widely interpreted as a university ranking. In contrast, the THES ranking list mixes aspects of research, teaching, labour market qualification and internationalization in a quite arbitrary manner. It should not be difficult to come up with many more university concepts for other ranking lists, which will all obtain very different results.

Ranking lists should be restricted to more specific activities

While the wish for a university-wide ranking list is understandable, it is fundamentally unclear whether there is and can be an agreed concept of what exactly a university is, even in a region that is fairly homogeneous politically and culturally. In the absence of such agreement, ranking lists should be restricted to more specific activities such as teaching, research and continuous education and to specific subject areas.