Written by Dr J.P. Nel, D. Com, D. Phil

Introduction. Peer review is a form of external audit. An external group of experts, called peers, is invited to assess the quality of different fields of the institution, such as the quality of education and training provision of individual departments or of the entire organisation. During the evaluation process, the peers visit the reviewed institution. Peers are external but work in a similar environment and have specific professional expertise and knowledge of the evaluated subject. They are independent and “persons of equal standing” with the persons whose performance is being reviewed.

There is no doubt in my mind that peer review can add value to learning institutions. Closely related to self-evaluation, peer review, if used correctly, can be developmental and consultative. It offers substantial advantages to the parties who are involved in the review process, for example measuring control over quality, the improvement of the learning offered by learning institutions, sharing ideas and efficient learning procedures, establishing networks of co-operation and many more.

The question, however, is if peer review is a suitable evaluation method for accreditation purposes. I offer the hypothesis that peer review is not suitable for the evaluation of applications for accreditation.

Preconditions for peer review. Peer review, if used incorrectly or for the wrong purposes, can do more damage than good. The following are preconditions for the use of peer review:

• For the sake of objective judgement, peer review should always be a two-way process, meaning that learning institutions should evaluate each other to promote and improve the learning offered by both. More than two learning institutions can also participate in peer review. It is not peer review if only one individual one-sidedly evaluates a learning institution.
• Peer review should be a voluntary process between learning institutions working in similar environments.
• Peer review should always be an open and transparent process.
• Peer review groups should always count among their number independent, external experts who possess appropriate skills and who are competent to perform their functions. They should include, where appropriate, persons who are competent to make national and international comparisons.
• Where internal experts are included – in the case of some programme-based and department/unit-based reviews – they should not be closely associated with the programme or department/unit under review.
• In the case of reviews of the effectiveness of an institution’s quality assurance procedures, all peer review group members should be external experts.
• Bodies responsible for the activation and administration of reviews should publish clear and transparent guidelines regarding the selection of reviewers. These guidelines should set out criteria and processes for the selection of relevant experts, because these guarantee their independence. Bodies responsible for the activation and administration of reviews should also publish clear and transparent guidelines regarding the responsibilities and duties of peer review group members. They should also ensure that the latter are adequately briefed about these responsibilities, duties, and the contexts (including relevant legislation) in which the reviews are being undertaken.

Why peer review is not suitable for the evaluation of applications for accreditation. The reason why peer review is not suitable for the evaluation of applications for accreditation is quite obvious – it does not meet the requirements for peer review listed above for the following reasons:

• Peer review in not, as some may claim, an internationally accepted method of evaluation for accreditation purposes. The claim that peer review is employed worldwide to conduct external evaluation is false. I conducted interviews with CEDEFOP experts in quality assurance and participated as the only speaker from a non-EU member state in a workshop on quality assurance at the University of Berlin in 2010. The general consensus amongst CEDEFOP interviewees and University of Berlin delegates, who were also speakers, was that peer review should not be used for external quality assurance.
• The inherent subjectivity of the procedure renders it unacceptable as an evaluation method where the future of learning institutions is at stake.
• The fact that peer review is used for other purposes does not justify using it for the evaluation of applications for accreditation. For example, using peer review to determine an academic paper’s suitability for publication or whether an academic qualifies for conferment of professorship has nothing to do with accreditation.
• Peer review is a self-regulating process, which means that it should not be regarded or used as an external evaluation tool for accreditation purposes.
• Peer review used with political motives in mind can be seen as a form of capillary power in which learning institutions are seduced into policing themselves. In a constructive quality culture any evaluation intervention, be it undertaken externally or internally should be for the benefit of the provider and students and not merely intended to meet external demands, especially not political ones.
• Unfortunately praxis shows that South African quality assurance bodies claim for themselves a position of power that is totally out of line with the purpose and spirit of quality assurance. Blom describes this as follows:

“Thus, while improvement is the rhetoric, the underlying intention seems to be to prescribe quality from the perspective of those who are in charge, and those whose interests are served…”

• The possibility of unfair evaluation is always real for many different reasons, for example because of the fear of creating competition for the own university or college, professional jealousy, etc.
• It is not only learning institutions offering the same qualifications that are in competition with one another. Students at public universities often launch advocacy campaigns and riot because of high fees, the colonial nature of learning content, allegations of racism, and many more. Serious students who wish to gain knowledge with which to pursue careers and to prepare for further learning would rather enroll with private learning institutions where their studies will not be damaged by such riotous activities. That is why any private learning institution may well be regarded as competition for public learning institutions.

Conclusion and close. Peer review is a healthy form of co-operation between learning institutions that can be used to improve the quality of learning offered. It should, however, not be used to evaluate a learning institution or qualification for accreditation purposes.

The potential lack of objectivity should be born in mind if peer review is used to evaluate compliance for accreditation purposes.

The Indian Council of Agricultural Research uses peer review for accreditation. However, they adopt a flexible approach in order to render the process objective and fair. For example:

1. They offer guidance and support to learning institutions applying for accreditation.
2. The learning institution is allowed to start offering the qualification prior to accreditation. Accreditation is only done for the first time after successful completion of at least one batch of students.
3. Accreditation is voluntary.
4. Accredited learning institutions receive financial assistance.
5. Accreditation can be granted conditionally.
6. Evaluation always includes a site visit.
7. Peer review is done by a team of experts, not just one individual.
8. Peer reviewers are carefully selected based on relevant knowledge and experience, leadership and people skills; personal/professional relationship; communication skills; objectivity, openness and freeness from preconceived judgments; lack of conflict of interests; willingness to serve; etc.

References

Blom, R. January 2016. The Size and Shape of the Public and Private Post-School Education and Training System in South Africa. Centre for Researching Education and Labour.

Kelly, B. Interviewed on 27 April 2007 in Brussels, Belgium.

Nel, J.P. A strategic Approach to Quality Assurance in Occupationally-Directed Education, Training and Development in South Africa. Doctoral thesis, UJ, Johannesburg.