Resolutions
The Research Rating System by the German Council of Science and Humanities
Recommendation of the 16th General Meeting of the HRK, 13 May 2014
I. Initial situation
The German Council of Science and Humanities passed the resolution Recommendations on the future of the research rating system for the comparative evaluation of research performance documented in results categories (WR Drs. 3409-13) on 25.10.2013. Starting in 2005, the German Council of Science and Humanities submitted the new 'Research Rating' procedure for systematic testing in four pilot studies (Chemistry, Sociology, Electrical Engineering, English Studies and American Studies). In the recommendations, the German Council of Science and Humanities proposes extending the research rating to all subjects, in turn, and establishing it permanently, even though it is a methodologically and organisationally highly sophisticated and complex procedure requiring considerable input from everyone involved. After exhausting every potential for savings, the annual direct costs of a comprehensive research rating exercise encompassing 22 subjects in five subject groups (including the associated external evaluation) might be reduced to €2.5 million. The German Council of Science and Humanities does not anticipate taking on the research rating system permanently but would assume responsibility for the first cycle of evaluations.
II. Statement
1. Thanks and appreciation
The German Rectors' Conference thanks every one who participated and contributed to the development of the 'Research Rating' procedure. In the estimation of the universities in international comparison, as a science-led procedure and from a methodological point of view, the newly developed tool will fulfil the requirements of a comparative evaluation of research better than any system developed hitherto.
2. Framework Conditions
The HRK therefore welcomes the proposal of the German Council of Science and Humanities that the research rating system is established in the way envisaged, to its full extent and permanently, provided appropriate framework conditions can be guaranteed. The following seven points are particularly important:
1. Higher education institutions and non-university establishments which undertake research in the relevant subjects are all and equally included.
2. Financing will not be provided by the institutions in question and will cover all the local costs as well as the central costs in full.
3. A standardised "core database of research" should be implemented in advance to form a full and systematic basis in order to maintain work levels and costs as low as possible.
4. Different subject areas and/or subject cultures will be rated each in their own way. Comparisons between subject areas are not appropriate given the design of the procedure.
5. When policy-makers evaluate the rating results, they should take into account the initial financial and legal positions of the institutions.
6. There are sufficient numbers of reviewers available and this will not adversely affect any other evaluation procedures.
7. The German Council of Science and Humanities will retain responsibility for the research rating system permanently. The universities do not take part in commercial rating exercises.
3. Recommendations
a) To universities, the use of a research rating system can be helpful in the context of decisions on governance. However, even if the recommendations of the German Council of Science and Humanities are implemented in full, there will be only a few current evaluations available at any time. Management measures need to balance evaluated and non-evaluated subjects. Rating results anyway provide information only about a situation in the past. Governance decisions, however, need to be made on the basis of evaluation results and forecasts.
The HRK therefore recommends supplementing the proposed concept with a forecasting element.
b) The research rating system can be used as a specific tool, regardless of the proposal of it being a comprehensive and permanent facility, if the necessary framework conditions are met and if there is sufficient reason at a higher level, for example if a selected subject area needs to be evaluated.
The HRK recommends that university governance should take into account the collaboration between subject areas (in particular) and also the fact that a rating system can only offer support indirectly with regard to interdisciplinary challenges.
c) To achieve satisfactory results, the HRK recommends in the case of a specific research rating classifying each range of subjects for any institution involved in such a manner that the arbitrary exclusion or inclusion of people, subjects or performance is avoided.
In this respect it should be ensured that joint research in and between faculties or institutions would not become adversely affected but further promoted.(1)
----------------------------------------------------------------------------------------------
(1) The HRK has also given consideration to the experience of higher education institutions in the UK with the now completed 'Research Assessment Exercise' and the ongoing 'Research Excellence Framework'. In the light of these specific procedures, which cannot be compared directly with the rating system produced by the German Council of Science and Humanities, it should be noted for the discussion in Germany that where budgets are tight, a) ratings are all too easily interpreted as rankings and b) the evaluation of performance does not lead to better funding, but rather to cutting of budgets.
