Monitoring awarding organisations and regulated qualifications

Summary

16.1 Awarding organisations will be responsible for the quality and standards of their own qualifications. They will be required to have their own quality assurance arrangements and will have to check their own compliance with the regulatory requirements.

16.2We will undertake our own monitoring activities. We will check the consistency of standards of comparable qualifications and compliance with the conditions of recognition.

Monitoring compliance with the conditions of recognition

16.3 We propose that the following principles should underpin our approach to monitoring compliance with the recognition requirements:

• we will target our monitoring on the quality and probity in the work of awarding organisations, on the standard of the qualifications they award, and on the effectiveness of their quality assurance arrangements. However we will monitor compliance with all the conditions of recognition from time to time
• the frequency and intensity of our monitoring will be determined on the basis of a transparent assessment of risk
• arrangements will be clear, fair, rigorous, objective, efficient and effective and we will:
  • publish information on our monitoring processes and procedures
  • review and, where necessary, improve our approach
  • review and report publicly on our own performance
  • base reported outcomes on verifiable evidence
• we will make fair and evidence-based judgements
• individuals undertaking monitoring visits will be trained, follow established procedures and adhere to a code of conduct
• awarding organisations will be informed of the outcomes of the monitoring activity
• enforcement action may be taken where it is needed to ensure that an awarding organisation complies with the relevant conditions, including conditions to comply with qualification specific requirements, and to protect the interests of learners
• awarding organisations will be required to rectify non-compliance within a specified period of time
• we will report publicly on the outcomes of our monitoring work in order to promote public confidence and continuous improvement in the qualifications system through transparency and accountability
• the outcomes of monitoring will be used to inform decisions about continued recognition and whether to impose accreditation or other conditions
• we will have a procedure for an awarding organisation to make a complaint where it considers that monitoring processes and procedures may not have been properly followed
• we will publish a procedure for dealing with complaints from the public about an awarding organisation and/or regulated qualification and information from complaints and their investigation will be used to inform our monitoring priorities.

Monitoring compliance with specific conditions and qualification requirements

16.4 We will also monitor qualification standards. Our approach will vary according to the nature of the qualification under scrutiny. We will monitor compliance with any conditions set.

16.5 One focus of our qualification standards monitoring will, it is proposed, be on comparing the demands on candidates who are following the same or comparable qualifications, both at a point in time and over time, that is, over the years for which the qualification has been available. We will review the measures taken by an awarding organisation to ensure the standards of its qualifications are equivalent to comparable qualifications offered by other awarding organisations. We will use our enforcement powers to intervene when necessary to ensure comparability and to secure standards.

A risk based approach to monitoring

16.6We propose to take a risk-based approach to monitoring. We will determine the frequency and intensity with which we monitor an awarding organisation according to our judgement of the level of confidence we have in the organisation’s arrangements to protect the interests of learners, standards and public confidence and the size and complexity of its awarding role. We propose that our judgement should be based on the following:

• the extent of the awarding organisation’s activity: numbers and types of qualifications offered, subjects/sectors of operation, licence to practise qualifications and certificates awarded
• the awarding organisation’s stability and financial viability
• the robustness of the awarding organisation’s processes and procedures as identified through the recognition process, self evaluation and previous monitoring
• the effectiveness of the awarding organisation’s self evaluation, that is, the degree to which an awarding organisation proactively identifies and resolves problems itself
• significant changes in levels of activity by an awarding organisation, both overall and within specific qualifications
• the need to ensure consistency and standards within and across qualifications, particularly when new qualifications or specifications are introduced
• the awarding organisation’s compliance with actions specified as a result of previous monitoring activity and through self evaluation
• the number of substantiated complaints received about the awarding organisation, a qualification, a subject and/or a specification
• the outcomes of any compliance investigations into malpractice or maladministration
• issues identified at the point of recognition, and accreditation, if this requirement is applied
• concerns based on relevant and objective information raised by other inspection or regulatory bodies or by SSCs or QCDA.

16.7 We plan to pilot the development and use of a risk profile for each awarding organisation. We believe this would promote transparency and consistency in our approach. The profile will also support a targeted and proportionate approach to regulation. We will share the profile with the awarding organisation concerned.

16.8 Our general approach to monitoring will be risk-based but we will reserve the right to undertake monitoring activities at any time to test and verify the accuracy of the profile or in response to an incident, intelligence or complaint.

Monitoring activities

16.9 We propose to use a range of tools when we monitor, depending on the focus of the monitoring activity. When we are monitoring an awarding organisation’s process and procedures these would include:

• reviewing the outcome of an awarding organisation’s self-evaluation
• desk-based evaluation of an organisation’s procedures or data
• visits to and meetings with the awarding organisation
• systems audits including investigations into specific systems such as quality assurance, assessment, customer service and IT systems
• observing training and Question Paper Evaluation Committee (QPEC), standardisation, moderation and awarding meetings
• visits to centres to test systems.

16.10 The tools we will use to scrutinise qualifications may include:

• targeted investigations that are used in response to risks identified in relation to standards comparability between subjects within a qualification, or between qualifications, as a whole or in part
• a long term programme of standards comparability investigations informed by new qualifications development
• a rolling programme of standards-over-time reviews including maintenance of a national archive of particular qualification specification and scripts.

16.11In addition we will undertake investigations into particular qualifications, specifications or groups of qualifications and look at the fitness for purpose of question papers, mark schemes and other assessment instruments. We may undertake thematic monitoring across a number of awarding organisations and undertake comparability studies.

Responding to findings

16.12Awarding organisations will be required through conditions to address any deficiencies that are found during monitoring. A timescale for action will be confirmed and evidence will be required that the awarding organisation had taken appropriate action. We will be able to direct an awarding organisation that had failed to meet a condition, and was consequently prejudicing the proper award of a qualification or learners, to take specific steps to correct a concern or to refrain from doing certain things. For example, we could give a direction to an awarding organisation that had failed to comply with a condition to ensure that the standards of its qualifications were comparable with those set by other awarding organisations.

Reporting the outcomes of monitoring

16.13We propose to publish the outcomes of our monitoring work. Where an awarding organisation has been required to take action as a result of our findings, we will also publish the organisation’s response and action plan.

Loading...