This second report reviews what we have done so far and outlines our thinking on current issues in assessment.

A PDF version of the report is also available to download.

Hundreds of thousands of learners have achieved qualifications and national curriculum assessment results since my last report in May. I would like to offer my congratulations to all these successful candidates and their teachers.

We are all familiar with the high profile given in August to the A level and GCSE results, followed in September by the reporting of the national curriculum key stage tests. What is frequently overlooked is the achievement of those learners who each day gain thousands of vocational and occupational qualifications. No fewer than five million vocational and occupational qualifications – known as VQs to distinguish them from GQs or general qualifications – were awarded in the 12 months to June. It is a record number but the lack of a single ‘results day’ means they receive much less attention from the media.

Vocational and occupational qualifications are valued by thousands of learners and employers in all walks of life. They are an important part of many people’s career development, giving recognition for the knowledge and skills acquired in the course of employment and providing an incentive to move forward. Not only do these qualifications open doors and lead to better opportunities for individuals but they also make a vital contribution to the success of the nation.

This is why VQ Day in June was an important opportunity to recognise the achievements of the people who achieved these diverse and wide-ranging qualifications. Celebrations were held in Belfast, Cardiff, Edinburgh and London, at which VQ Heroes – VQ Learners and Young Learners of the year – for each country were announced. Reading about the challenges many of these successful learners faced during their courses is a salutary lesson for us all.

I was delighted to have the opportunity to visit two colleges on VQ Day: North Warwickshire & Hinckley College in Nuneaton and Warwickshire College in Leamington Spa. Ofqual also attended the VQ Day event for England where a total of 18 regional VQ Heroes collected their certificates and the national award winners were announced.

In Belfast our Ofqual team took part in a ceremony at the Waterfront Hall where Sir Reg Empey, Northern Ireland’s Employment and Learning Minister, announced the Northern Ireland Learner and Young Learner of the Year.

The next VQ Day will be on Wednesday 23 June 2010. Let us all put the date in our diaries.

I should also like to pay tribute to the learners who achieved their Diplomas this year. Diplomas focus on applied learning and bring together attainments across a range of qualifications. The Diploma is not just a new qualification; it is a new approach in 14–19 qualifications, a ‘meta-qualification’. To obtain a Diploma, learners must have:

• undertaken the principal learning qualification in the area covered by the Diploma and a project relevant to the theme of the Diploma
• achieved passes in functional skills qualifications in English, mathematics and ICT at the appropriate level
• completed a range of qualifications showing their achievement in areas of additional and specialist learning
• undertaken a significant period of work experience
• developed personal, learning and thinking skills that are needed for success in learning and life.

Those learners who expected to complete their Diploma this year but did not manage to do so should not be too disheartened; all the qualifications they did achieve remain valid and show what they have achieved. And what is more, when they do complete the remaining ones – and I urge them to persevere – they will be eligible for the Diploma, whether it takes another week, month, year or two or even more.

The fact that the Diploma is such a new approach raises significant issues for Ofqual. I discuss these in more detail in Section 2.

Public confidence in qualifications, examinations and assessments is essential. In my May 2009 report I noted some outstanding issues that needed to be addressed in order to safeguard this confidence. In this section I want to bring you up to date with what we have done to ensure that the qualifications and assessments were of high quality and delivered accurately and on time.

Ofqual has been monitoring both the quality of the assessment processes and the robustness of the delivery systems. I am pleased to report that 99.9 per cent of the key stage 2 national curriculum assessment results were delivered on time. Our monitoring of the quality of the marking process indicates that it was satisfactory. It was particularly important to ensure that markers were consistent this year following the discontinuation of the borderlining procedure, which had been used for some years to re-check scripts falling close to the cut-off score for a level. However, changes to the methods by which markers were checked for accuracy led to a larger than expected number of markers being rejected, particularly in English and science. We are investigating the cause of the high level of rejections as part of our qualitative research into 2009 assessment procedures to ensure that the approach is truly improving the quality of the assessment.

A new approach to checking the accuracy of markers is being used in the pilot assessments of the single level tests¹. This involves marking the test papers on the marker’s computer screen, which is a common feature of many examinations but has not been used in key stage assessment before. One advantage is that markers can be checked more frequently. They do not even need to know when the standard of their work is being monitored by senior examiners, making the process more objective and reliable. Experience at GCSE, where this approach has been used for some years, suggests that this leads to more consistent marking. We are discussing with ministers and the Qualifications and Curriculum Development Agency (QCDA) how this might be taken forward in national curriculum assessments.

I also reported a number of issues relating to the GCSE sciences. Bearing in mind the concerns that Ofqual voiced earlier in the year, we have been working with the QCDA and the awarding organisations to ensure that the criteria and specifications for GCSE science are improved. QCDA has undertaken a consultation on revised criteria and new specifications meeting the new requirements will be produced over the next year and published in time for teaching to start in September 2011. At the same time the government’s Science and Learning Expert Group – chaired by Sir Mark Walport – has consulted on more general issues relating to science (including mathematics) education pre-19.

Revisions to the criteria and specifications are important for the future but there is still the question of what to do for those learners who are following the current specifications. We cannot afford to allow these learners to suffer because of the flawed specifications. Ofqual has therefore been working closely with the awarding organisations. We have required them to improve the quality of papers, ensuring that they give all candidates the opportunity to show what they can do and giving better opportunities for more able candidates to demonstrate their talents. In monitoring the 2009 awards we used information about the learners’ achievements at key stage 3 to compare results across the awarding organisations in England. Overall we were satisfied that the results fairly reflected the candidates’ performance.

There are some things that we cannot change in the short term. Many candidates had already taken and been graded on most of the units necessary to achieve the qualification. With only a few units left, it is impossible to make much of a difference to the standard of the award or address failings in those units already taken. The issues vary across the awarding organisations and our aim has been to maintain year-on-year standards while mitigating as far as possible the anomalies we identified in 2008.

We will continue to keep a close eye on the situation over the next few years. We will attend many of the awarding meetings and facilitate discussions with the technical staff of the awarding organisations on how best to evaluate results. We will also gather evidence of the quality of questions from an examination of the candidates’ answers. In this way we will protect the interests of the learners currently involved until the full reforms can be implemented.

This year has also seen the first awards of the new AS qualifications, which are based generally on two instead of three units. Carrying forward standards from year to year is a difficult process at any time and, particularly in the first year of such a major change, the complexity of the task is too great to rely solely on the judgement of subject specialists. There is new content, new assessment approaches and new procedures for aggregating across units. It would be unfair to expect awarders to be entirely consistent in such circumstances.

Our role as regulator is to ensure that standards are maintained from year to year, across specifications in the same subject and across the awarding organisations. To carry forward standards in such circumstances we agreed that the awarding organisations would use an agreed range of technical and statistical evidence to check that the grades awarded under the new system were broadly in line with judgements made about candidates in previous years. These included the relationship between attainment at GCSE and AS level and the performance of the previous year’s candidates. I wrote an open letter to all secondary schools and colleges in March indicating the approach we were taking².

Of course the progress of an individual learner is unpredictable, but for the candidature as a whole these relationships are remarkably stable. In other words, for many years once we knew the results a group of candidates achieved in their GCSEs we could predict with reasonable accuracy what grades the group would get a year later at AS level. This information can then be used by awarders to inform their judgements and by the awarding organisations and Ofqual to check that their draft grades are within the range we might expect. If awarders’ proposals result in more or fewer candidates in a particular grade than the statistical and technical information suggests, further investigations can be requested. The awarding organisations can look again at the judgements of the awarders to satisfy themselves that the grades are as secure as possible.

We used this approach for the first unit awards in February 2009 when awarding organisations were asked to report to Ofqual any unit awards that were outside the proposed tolerances. Broadly speaking the outcomes were acceptable. In the summer the awarding organisations were asked to report outcomes for full qualifications and again highlight any that were outside tolerances. This was carried out on a weekly basis through the summer. There were very few awards outside the tolerances where the regulator needed to ask for a review of the awarders’ recommendations.

On 4 November Ofqual published the regulatory criteria for the full introduction of the functional skills qualifications in English, mathematics and ICT for which teaching begins in September 2010. Over the past two years QCDA has been running a number of pilot projects, encouraging the development and assessment of functional skills qualifications by a number of awarding organisations. Ofqual has carried out a review of the standards and comparability of the assessments. The tests will be used as part of the Diploma and learners will be required to have passed in all three subjects before they can receive their Diploma award. Functional skills tests are not only part of the Diploma but are free-standing qualifications in their own right. It is intended that they should replace other assessments in key and basic skills and be relevant to all learners, whether they are young people in schools, apprentices in industry or mature military personnel on tour in other countries.

The functional skills qualifications do not fit either of the traditional categories of general or vocational qualifications. For example:

• The award is made on a pass/fail basis similar to many vocational qualifications but unlike general qualifications such as GCSEs, AS or A levels, which are awarded across a range of grades.
• The focus on the effective application of the functional skills to achieve a purpose is distinct from the predominantly knowledge-based learning of general qualifications.
• While the functional skills assessments are applied – task-based and set into real life situations – they are not intended to prepare candidates for work in a specific job or skill sector as one would expect in vocational qualifications.

In a pass/fail assessment the decision to be made can be very simply stated:

• ‘Pass’ indicates that the candidate has demonstrated the skills required.
• ‘Fail’ indicates that they have not.

But getting to that decision is far from simple.

To be valid the assessment must allow learners to demonstrate that they can apply their skills in situations such as they might expect to find in life or work. In English, for example, a level 1 pass requires that candidates must be able to write a range of documents, and it is surely obvious that a range cannot be demonstrated by a single piece. At level 2, where a wide range of documents is required, at least three examples of different types of documents must be required if learners are to be given the opportunity to demonstrate their abilities.

In GCSE mathematics it is common to allow some credit for correct working out even if the answer is wrong because of a mistake early in the calculation. In the functional skills assessments, however, learners are required to demonstrate their interpretation skills by considering the appropriateness and accuracy of their results and conclusions. They must therefore be given the opportunity to demonstrate this in practice. An answer that is far too large or too small to be reasonable in the context should be given no credit, even if the learner used the correct method to work it out.

In the ICT functional skills assessments candidates must demonstrate that they can insert, remove, label and store media safely. This clearly must involve a learner’s physical handling of CDs, DVDs or memory sticks, not merely saving to the hard drive of the computer. This may require those preparing candidates to modify their IT practices to provide more hands-on experience.

As I have already noted, it is a requirement of the Diploma that the learner has demonstrated competence in all of the functional skills. So failure in just one of the functional skills assessments can have a dramatic consequence: no matter how well the learner has done in the principal learning, the project, the additional and specialist learning and personal and learning skills, the Diploma cannot be awarded.

Our monitoring of the pilot over the past two years has indicated that the assessment schemes have some way to go before they can be regarded as fully satisfactory. In the light of the findings of our studies, pilot awarding organisations have been asked to take appropriate measures to ensure that the assessments meet the requirements laid out in the standards³.

When it comes to the new qualifications to be tested post September 2010 we will only accredit those in which the functional skills qualifications meet our criteria and are therefore of high quality and fit for purpose.

On the broader front we are half way through our two-year investigation into the reliability of qualifications and assessments. So far we have commissioned a number of projects, three of which were considered by experts at a seminar held at the University of Warwick in October. The three projects explored different approaches to reliability and one is now on our website⁴. Others will be released in the coming months. Further work has been commissioned and the projects will report towards the end of next year.

One suggestion arising from the research was that we should explain what we are doing in non-technical terms for a general audience. As part of this approach it was suggested that we should move away from the use of the word ‘error’ in our discussions of reliability as the word in its ordinary usage does not capture the nuances of the way it is used in assessment.

Unfortunately the discussion was interpreted in some quarters as an attempt at ‘spin’ and brought accusations that Ofqual was shying away from any discussion of mistakes – particularly mistakes by the awarding organisations or ourselves. This could not be further from the truth. We have stated that ‘where mistakes are found, Ofqual will report them’⁵.

However, there has been some confusion over what we are trying to do and I should like to make our position clear.

The word ‘error’ has at least two distinct definitions. The one we use in everyday language:

something incorrectly done through ignorance or inadvertence; a mistake

and one from mathematics:

the difference between an approximate result and the true determination

(Both of these are taken from The Shorter Oxford English Dictionary.)

This latter definition is not talking about a mistake. It is recognising that, as in many aspects of mathematics, absolute precision is impossible (the value of π, for example⁶ The same is true of any form of measurement: however accurate your scales may be, there is always a small amount of imprecision – ‘error’ – in the reading.

The fact that some commentators immediately assumed that we were talking about mistakes serves to highlight why we are looking for a different way of talking about this aspect of error.

Awarding organisations and assessment agencies must do their utmost to eliminate mistakes. It is Ofqual’s job to see that they take all reasonable steps to do so. Mistakes must be identified as early as possible in the process and corrected at once. The awarding organisations’ procedures are designed to guard against anything ‘incorrectly done through ignorance or inadvertence’.

But even when they – and we – have checked and rechecked the assessments to correct any mistakes, there will remain a degree of imprecision that cannot be removed. This imprecision, which is not a mistake but which can affect the reliability of a qualification, arises from factors over which neither the awarding organisation nor the regulator has any control such as the degree to which the assessment reflects two years of classroom teaching or the fluctuations in a candidate’s performance from day to day. Then there is some imprecision that we can try to reduce but cannot eliminate such as the differences between sets of questions the candidates choose to answer.

I hope this sets the record straight.

1. Single level tests are intended to assess the knowledge, skills and understanding demonstrated by a learner. Single level tests give teachers the flexibility to enter learners for a test when they are considered to be working at a particular level.
2. www.ofqual.gov.uk/2235.aspx
3. www.ofqual.gov.uk/94.aspx – relates to the standards used throughout the pilot
4. www.ofqual.gov.uk/2606.aspx
5. www.ofqual.gov.uk/2515.aspx
6. π can be reported at almost any level of precision. For many purposes ‘between 3.141 and 3.142’ or ‘between 3.1415826 and 3.1415827’ would be sufficient. The more decimal places used, the more ‘error’ is reduced. But it can never be given absolutely accurately.

Ofqual now has very solid foundations in place and is set to start work formally in April 2010 as an independent non-ministerial government department accountable to Parliament. With the passing of the Apprenticeships, Skills, Children and Learning Act 2009 Ofqual will become the regulator of qualifications, examinations and assessments in England. Ofqual will be the regulator of vocational qualifications in Northern Ireland.

The importance of establishing and protecting our independence was an important theme of my first report in May this year. With this independence comes the responsibility to act rationally and fairly and be open about the decisions we make. Not only should we aim to set out our requirements in a clear and understandable way, but the thinking behind them should also be opened to scrutiny. In this second report I review what we have done so far and outline our thinking on current issues in assessment.

There is a great focus every year on general qualifications such as GCSEs and A levels. Over the last year Ofqual has worked very closely with awarding organisations on the marking of new style A levels that will culminate in the first awards next summer of the new A* grade. In GCSE science we worked closely to ensure that the results this year were fair and took account of our report into science standards. We are also working with awarding organisations on new modular GCSEs that centres began teaching from September this year.

Vocational and occupational qualifications are valued by thousands of learners and employers in all walks of life and make a vital contribution to the success of the nation. Thousands of vocational and occupational qualifications are achieved every day and in the 12 months to June 2009 a record five million were awarded.

A long-standing problem of our education system has been the perceived lower status of applied, vocationally-oriented qualifications for the 14–19 age range. This was the first year of awards for the new Diplomas, which are a serious attempt to address this problem. They have brought with them new thinking about the meaning of achievement and the time it takes to gain a qualification. All new qualifications have teething troubles that need to be addressed in ways that nurture their development.

In November Ofqual published the regulatory criteria for the full introduction of the functional skills qualifications in English, mathematics and information and communication technology (ICT) for which teaching begins in September 2010. These tests are a compulsory part of the Diploma and learners will be required to have passed in all three subjects before they can receive the full award. They are also free standing qualifications in their own right, intended to replace other assessments in key and basic skills.

Our monitoring of the pilots that have been conducted over the past two years has indicated that the present assessment schemes have some way to go before they can be regarded as fully satisfactory. In the light of the findings of our studies awarding organisations have been asked to take appropriate measures to ensure that the assessments meet the requirements laid out in the standards.

In national curriculum tests we were pleased that 99.9 per cent of key stage 2 test results were delivered on time in early July, a significant achievement following the problems of 2008. We have looked closely at the key stage 2 English test from 2009, and our reliability programme has published a report about the reliability of the key stage 2 science tests over recent years. Ofqual has a significant new role in the early years foundation stage (EYFS) that we will be developing following consultation.

Although we carry forward the regulatory work of the Qualifications and Curriculum Authority (QCA), Ofqual is not just ‘more of the same’; we are a new body with different duties and powers. The Act that brings Ofqual into being shifts the focus of regulation from the requirement to consider and accredit every single qualification to a broader approach that regulates the awarding organisations and holds them responsible for the provision of high-quality examinations and assessments.

Ofqual’s focus will be on providing a structure within which the awarding organisations can operate successfully and on using our monitoring and enforcement powers when necessary to ensure that standards are maintained. We are formally consulting on how we should do this.

The views of learners matter hugely to Ofqual. We know that their future is determined by the qualifications and tests that we regulate. Our principle is that qualifications, tests and assessments must facilitate good learning, not dominate or distort it. A regulator should look well beyond the immediate future. Children born in 2009 will, according to government policy, remain in education and training until at least 2027. They may be working until at least 2074 and many can expect to live into the 22nd century. The assessment system we are devising now must be able to provide a firm basis for the changes that are bound to take place over that time.

It is clear that technology is second nature to today’s learners. To help build Britain’s future they must be allowed to embrace its potential and maximise the new opportunities it provides for them to demonstrate their achievements. The use of computers in assessment raises issues of central concern to the regulator, but it is essential to deal with the challenge rather than hide from it.

We also need to recognise that constant change destabilises the system and discourages investment. Awarding organisations, on whose expertise and commitment the system depends, may well choose not to invest in the development of new forms of assessment if the likelihood exists that, even before the changes have had a chance to settle down, the whole system will be reorganised and their investment wasted. It is for the regulator to do what it can within its sphere of influence to safeguard the stability of the system. A key objective for Ofqual is to build public confidence in qualifications and some stability in the system will help us achieve this.

While my responsibilities as Chief Regulator require me to maintain the standards of what we have currently, I must also encourage those who provide assessments to look forward so that the qualifications system remains relevant to society. Other performance standards, therefore, might relate to how assessments are carried out to effect a transformation from examinations that are largely paper-based to those in which candidates respond using computers. The system must change to reflect the society in which we live. We must look to what the future holds for the young people for whom we are now designing qualifications. We must ensure that our investments – especially in an era of financial constraint – yield qualifications and processes that are fit for purpose and are of the highest quality.

Such an approach might give considerable scope for improving services to learners and to their places of learning, encouraging innovation and making best use of the professionalism and expertise of the awarding organisations.
It is an approach to which we will give considerable thought and one on which I would welcome views.

So here are some suggestions for a longer term approach to the qualifications and assessments system.

• We need to ensure that our curricula (both in schools and in the workplace) are appropriate as a basis for the education and training of people of all ages but particularly the young, who will be living in a very different world from ours. The qualifications and assessments they face should facilitate their learning, progress and future success.
• We need to invest more in the training of examiners and assessors to equip them with the skills to assess diverse curricula in new and exciting but valid and reliable ways.
• We need to reduce the complexity of the qualifications system while ensuring that there is sufficient flexibility for learners to have a reasonable amount of choice.
• As well as thoroughly piloting any new assessment models, we should require all questions and tasks to be pre-tested to ensure that they are fit for purpose and produce informative responses.
• We need to ensure that the means of assessment are fit for their purpose and make the best use of technology.

All these suggestions have implications for awarding organisations, schools, colleges, employers, higher education and for the government. They would also have cost implications. I would welcome your views on how we should move forward.

One of the main ways in which Ofqual can win and keep the confidence of those we regulate and of the wider public is by establishing and protecting our independence. That was an important theme of my May 2009 report and the Apprenticeships, Skills, Children and Learning Act.

The post of Chair of Ofqual and Chief Regulator is a Crown Appointment, on the recommendation of the Secretary of State. The process of appointing members of the new Ofqual Board (including one from Northern Ireland), who will take up their full duties when Ofqual is vested formally in April 2010, is well advanced.

Our role in Northern Ireland will require us to regulate the QCF and vocational qualifications in the National Qualifications Framework (NQF). We have a responsibility to act rationally and fairly and in a manner that inspires confidence in us as an independent regulator.

We must set high standards and see that they are met. Our future work in Northern Ireland will be consistent with that in England in respect of vocational qualifications.

We will look primarily to our established team in Belfast to work with our other teams in Ofqual to promote the relevance, reliability and flexibility of vocational qualifications in Northern Ireland. Our team in Belfast has vast experience of working with Northern Ireland stakeholders, including learners, further education, training providers, employers and awarding organisations. This local knowledge will be invaluable as together we regulate a vocational qualification system that supports the Northern Ireland skills agenda and provides learners with vocational qualifications they can trust.

For awarding organisations operating either solely in Northern Ireland or across England and Northern Ireland we will need to be satisfied that their vocational qualifications are of high quality, have the support of learners and employers and are readily available to the Northern Ireland public. We will publish an annual report to the Northern Ireland Assembly identifying what we have done to regulate the vocational qualifications system in Northern Ireland. We will also report publicly on the outcomes of any monitoring work. These will show the public how we regulate, setting out what issues we have found and the measures we have put in place to address any weaknesses in the system.

Vocational qualifications are a key part of the education and skills system and the world of employment in Northern Ireland. They underpin the skills agenda and set measurable standards for skills acquisition and development. These qualifications open doors for learners and can lead the way to better opportunities for individual learners and for Northern Ireland to compete in a global economy. We will work closely with stakeholders to promote public confidence in vocational qualifications and increase their value for Northern Ireland learners. We will take steps to listen to and understand learners’ needs as well as the needs of employers and others through adviser forums, learner panels and regular communication with our key stakeholders. We will work with the Department of Employment and Learning (DEL) in Northern Ireland to understand its policy objectives.

It is important that consistent standards in qualifications are maintained. We will work with the Council for Curriculum Examinations and Assessment (CCEA), the general qualifications’ regulator in Northern Ireland, on joint programmes concerning the maintenance of standards in general qualifications and on other shared regulatory matters that apply across the two countries.

In England Ofqual’s role covers all qualifications except degrees, which are the province of the universities. It also covers assessment arrangements that are part of the national curriculum assessments or the EYFS. Since the end of May 2009 Ofqual has operated out of its new offices at Spring Place in Coventry. This has meant quite a substantial transition with some staff moving from London and new staff being recruited. It was important, during the move, to ensure that all aspects of our work continued without break. New staff recruits have received intensive induction, ensuring that they are well versed in both the theory and practice of regulation and assessment. Even the staff who have transferred from QCDA have found these programmes valuable as they have emphasised the changes in Ofqual’s ethos, responsibilities and powers. This has been an exemplary induction programme of which we are proud. It has allowed us to move as seamlessly as possible not only from one location to another but from a highly experienced body of staff to a newer one while retaining the confidence of those we serve and those we regulate.

However, Ofqual is not the only regulatory body in the UK. In Wales the Department for Children, Education, Lifelong Learning and Skills (DCELLS) is the regulatory authority for all qualifications. In Northern Ireland responsibility for the regulation of all qualifications currently rests with CCEA. Learners have a high degree of freedom to take qualifications offered by awarding organisations in other parts of the UK and many qualifications, such as GCE, AS and A levels and GCSEs, are offered in England, Wales and Northern Ireland. In most cases schools and learners can choose qualifications from any of the awarding organisations provided they are approved by the Secretary of State. The regulators in all three countries therefore apply the same criteria and have the same requirements so that the public can be assured that consistent standards are being applied. We also work with the Scottish Qualifications Authority (SQA), the regulator in Scotland, to ensure that standards of similar qualifications are comparable.

As is appropriate to a new organisation and as is required under the new legislation we have embarked on a major consultation over our role and our methods of working. This has begun and continues into the spring of 2010. We are keen to hear from as wide a range of respondents as possible. We would welcome comments via our website: www.ofqual.gov.uk/consultation. Alongside the consultation we also published proposals for transitional arrangements to enable us to maintain our regulation of awarding organisations and their qualifications post vesting while we move into the new arrangements.

One of the concerns that has been expressed since the start of the financial crisis has been the failure to identify problems early enough for action to be taken.

One of the concerns that has been expressed since the start of the financial crisis has been the failure of the financial regulatory system to identify problems early enough for action to be taken. There needs to be an overarching requirement that the qualification as delivered is of the appropriate quality and standard.

In relating this to the regulation of qualifications and assessments I start from the expectation that everyone wants the system to be fair and for the outcomes to reflect the real achievements of the learner. This means that everyone carries a share of the responsibility for quality. The recognised organisations have a responsibility for the regulated qualifications they offer and the assessments they undertake. Schools, colleges, employers and other organisations must take responsibility for ensuring that learners are properly prepared for the qualification. Those that provide the centres where assessment takes place must ensure that they meet the necessary standards, with appropriate facilities and supervision.

Under this approach every person involved in the assessment process – from the candidates themselves to ourselves as regulators – is under a duty not only to carry out the assessment in a rigorous manner but also to raise the alarm if they find any problems that might put quality in jeopardy. Regulators primarily rely on others to have appropriate processes and to take appropriate steps – neither too lax nor too restrictive – to check that everyone is playing their part. Our monitoring procedures are designed to ensure this shared responsibility, backed by appropriate powers of enforcement should they become necessary.

Ofqual’s responsibilities for assessment under the national curriculum and during the EYFS for children from birth to five are quite different from those we have been given for qualifications.

The Secretary of State for Children, Schools and Families is responsible for all the assessment in the national curriculum, of whatever type, and for the assessment arrangements in the EYFS. The legislation places an obligation on Ofqual to ‘keep under review all aspects’ of those assessment arrangements and to inform the Secretary of State ‘if there are likely to be significant failings’. In addition we have the duty to ‘promote the development and implementation of assessment arrangements which give a reliable indication of achievement and indicate a consistent level of attainment’.

In respect of the assessment arrangements we clearly need to work very closely with the DCSF, QCDA and other agencies. The legislation requires the Secretary of State to consult Ofqual before making changes to assessment arrangements. Given the major changes in assessments over the last few years, we must find ways to gather the necessary evidence to underpin authoritative guidance to a wide range of responsible bodies.

The Secretary of State has accepted in full the recommendations of his Expert Group on Assessment¹, which has identified the purposes of assessment up to the end of key stage 3 as being:

• to optimise the effectiveness of pupils’ learning and teachers’ teaching
• to hold individual schools accountable for their performance
• to provide parents with information about the child’s progress
• to provide reliable information about national standards over time.

To meet these objectives the Group’s recommendations include a wide range of assessments and in order for our regulatory approach to be fit for purpose it will need to embrace them all.

We need to set out high aspirations such as:

• Assessment approaches used are valid, reliable, comparable, manageable and free from bias.
• Stakeholder surveys indicate confidence in the outcome of national curriculum and EYFS assessments.
• There is comparability in the assessment outcomes between institutions and year on year.
• Assessment outcomes are comparable between key stages and between subjects.
• Assessments take into account the different people who are involved in generating assessment evidence.

We are proposing a risk-based approach to our monitoring activities and will publish in advance a programme setting out our priorities.

Assessment in the national curriculum and EYFS can be grouped into two main categories: tests and practitioner assessment. Tests can be used to provide a snapshot of a learner’s attainment while teacher assessment usually takes a view of the learner’s performance over a period of time. Our regulation must reflect the different purposes and outcomes of those categories.

Ofqual’s responsibilities cover the full cycle of test development, delivery, marking and level setting at key stage 2. The code of practice sets out the processes and procedures necessary to ensure that high-quality, consistent and rigorous standards are applied in the tests. This will be the basis for our monitoring of the administration, development, delivery and reporting of the tests. In addition we may carry out whole-system or thematic activities. A whole system approach would look at, for example, the full cycle of a particular test or set of tests to establish whether the principles and purposes of the assessment have been achieved. Such a review may be appropriate for the proposed national sample test for key stage 2 science. A thematic approach might involve looking at one aspect of the process such as test development across a range of statutory and non-statutory tests.

Teacher assessment and that carried out by practitioners in nurseries and other early year’s settings forms the basis for much of the important information on a child’s development and progress.

Our approach must reflect:

• the objectives of the assessments
• the role that parents, teachers, teaching assistants, early year’s providers and other bodies play in the conduct and quality assurance of the assessments
• how the outcomes are used.

In the case of teacher assessment, a whole-system approach would look across different stakeholders, processes and outcomes in a particular assessment arrangement. A thematic approach might look at the impact on a particular group of learners such as those with special needs. A risk-based approach would target our monitoring on specific activities where an initial analysis of information gives cause for concern.

We plan to adopt a continuous quality improvement model. This involves working with stakeholders – the DCSF, National Strategy teams, QCDA, local authorities, settings, schools and parents – to help focus our monitoring and disseminate and encourage the implementation of the best practice that is identified.

One of the key purposes of assessment identified by the Secretary of State’s Expert Group on Assessment was to provide parents with information about their child’s progress. Parents’ views will be important to us as we make judgements about the usefulness of the assessments. So we will need to find ways of engaging with as wide a range of parents as possible. We will also need to establish a network of specialists who can inform and validate the focus and outcomes of our monitoring activities and provide a link to a wide range of practitioners.

We must be careful that our regulation does not place an undue burden on individuals or institutions. Many early years settings and some schools will be very small, without the infrastructure of larger institutions. We must tread carefully between appropriate rigour and unnecessary bureaucracy. The risk-based approach will help to focus our activities where they are most needed. We expect the other institutions to ensure the quality of those aspects for which they are responsible.

We will need to develop appropriate activities to identify where changes may be necessary and the nature of the changes to be made. This may include the evaluation of assessment guidance and training materials – including those developed by QCDA, DCSF or National Strategies – to ensure that they lead to valid, reliable, comparable and manageable assessments that are free from bias. As a result of such investigations we will give feedback to the relevant bodies. We may look at how effective QCDA’s arrangements are for monitoring local authority moderation arrangements; we might observe local authority moderation in practice to see how effective the moderation model is; or we may evaluate the effectiveness of moderation training. Again the aim would be to identify possible improvements, which we would discuss with the relevant organisations.

The wealth of data collected nationally is a valuable source of information that we would analyse to identify if the outcomes of assessment are being used appropriately. We would also seek to understand the impact of assessment outcomes in different contexts on assessment practice, and the validity and reliability of the assessment arrangements.

We will be required by the legislation to give guidance to responsible bodies on how they perform their functions in relation to the assessment arrangements. We are planning to develop a range of ‘models of assessment behaviour’, in consultation with our partners and stakeholders. These will cover the development, implementation and monitoring of both national curriculum and EYFS assessments. The focus will be on defining accountabilities for success rather than laying down procedures and processes. It is the outcome that is important, not the means by which that is achieved. The codes of practice for national curriculum and EYFS assessments will include suggested models for maintaining standards. We will also publish the findings of our monitoring and facilitate the sharing of good practice.

1. http://publications.dcsf.gov.uk/eOrderingDownload/Expert-Group-Report.pdf

Promoting the awareness and understanding of regulated qualifications is one of Ofqual’s key objectives and one without which we cannot hope to win public confidence. This vital part of our work has included talking to learners, visiting employers and addressing a wider audience through conferences.

Ofqual’s staff and myself have continued to visit schools, colleges and workplaces in various parts of the country to encourage learners to talk to us about their views of qualifications and the issues that they think Ofqual needs to address. In June, for example, we held a learners’ forum in Newcastle, meeting teenagers and adults who had experienced difficulty trying to gain qualifications in a traditional school environment. We also went to Canterbury, where the UK Youth Parliament was meeting, to get members’ views on the reliability of examinations, tests and assessments.

There was a strong feeling that young people are under too much pressure to succeed – not just in secondary school qualifications but in primary schools. While success might be motivating, failure was demoralising. ‘There’s a perception in our heads that unless we do well in those tests we’re not going to get anywhere, and that’s wrong because not everyone’s good at examinations,’ we were told.

Learners felt that examinations gave only part of the picture. They suggested more emphasis on coursework and on building up qualifications in smaller sections over a period of years. In addition they wanted more advice from people with a more rounded view of them than a subject teacher might have. The new Diploma, discussed in Section 2, was regarded as a good initiative because it puts more emphasis on applied learning outside the classroom.

The value of qualifications as the basis for both better job satisfaction and improved quality of life was emphasised by members of the Youth Parliament. They regarded reliability as important but felt ‘it would be unrealistic to expect examinations to be 100 per cent perfect’. However, their views indicated a lack of consensus on what reliability meant. Some felt mathematics was more reliable ‘because it’s a number subject’ while others felt that ‘English examinations are more reliable because you can be more varied with those examinations’.

When asked how reliability could be improved a number of suggestions were made including:

• removing multiple-choice questions
• having several markers for the same paper
• the involvement of recent candidates for the examinations in the development of future materials
• the development of more rigorous criteria.

Other aspects of the examination process were important but most learners recognised that their results were primarily a reflection of what they did on the day.

It was also interesting to hear what they thought affected their performances in examinations. The timetabling of papers was important to allow time for revision, they said. Other aspects of the examination process were important but most learners recognised that their results were primarily a reflection of what they did on the day.

A particularly interesting visit was to McDonald’s at its UK headquarters in East Finchley, North London. The restaurant chain was one of the first employers to become a recognised awarding organisation. I was given a tour of the McDonald’s university and had the opportunity to meet staff, talk to them about their roles and discuss the training ethos of the company. I also met some of the learners who were all testament to how McDonald’s invests in its crew members.

McDonald’s employs some 75,000 staff in the UK and 60 per cent of these employees are under 21 years of age, making it one of the largest employers of young people in the UK. The emphasis is on making the best of the talents of its staff through its talent management programme. It launched its first apprenticeship scheme early this year. The company has spent a great deal of time and money working on its education strategy. Over the last five years it has concentrated on improving employee engagement, including offering flexible working and a continuous learning programme to address the personal and professional development of staff.

I was very pleased to hear that McDonald’s found that achieving awarding organisation recognition enhanced the quality of its own in-house training. It was evident that gaining this status is working well for both McDonald’s and its employees, many of whom are currently working towards functional skills qualifications. Senior managers at McDonald’s praised QCDA for their help and guidance as well as the support they had received from the awarding organisations Edexcel and City & Guilds. The company has also developed partnerships with a number of colleges and universities.

When I met Simon Waugh, Chief Executive of the National Apprenticeship Service, I was pleased to hear him state: ‘At the National Apprenticeship Service one of our absolute priorities is the quality of training people receive. While our focus is to increase the number of opportunities for young people, the quality of these opportunities must be of the highest standard. Therefore the assurance that Ofqual can give in terms of quality of the qualifications that apprentices gain is vital for us.’

Over the past few months Ofqual staff members and I have spoken at a number of conferences, explaining about the organisation and the work we are doing. I addressed the Open College Network North East Region (OCNNER) Conference in Gateshead and the Headmasters’ and Headmistresses’ Conference (HMC) in Liverpool. I have already mentioned in Section 2 the presentation that Ofqual gave at the Cambridge Assessment Conference. We have also:

• presented a paper to the Westminster Forum in which we discussed confidence, standards and technology
• discussed the reliability of results programme at the third National Conference of the Chartered Institute of Educational Assessors (CIEA)
• explored some of the problems of explaining results at the International Association for Educational Assessment (IAEA) Conference in Brisbane
• spoken to the Westminster Education Forum in July on preparing and developing the workforce for 2020
• spoken at the Northern Ireland Apprenticeships Award Ceremony
• focused on Diplomas for a presentation at the Westminster Education Forum Keynote Seminar in October.

Ofqual provided three speakers and contributed to a number of workshops at the annual conference of the Federation of Awarding Bodies (FAB), which took place in Coventry in November. The theme of the conference was ‘The Qualifications’ Manifesto’, with an emphasis on the need for awarding organisations to be properly prepared for the QCF. FAB represents the interests of a large proportion of the awarding organisations across a wide range of qualification types and the conference was an opportunity for members to get together and discuss important issues.

Technology is second nature to today’s learners. They must be allowed to embrace its potential and maximise the new opportunities it provides for them to demonstrate their achievements. The use of ICT in assessment raises issues of central concern to the regulator, but we must seek ways of dealing with the challenge rather than hiding from it.

One of the requirements of the Apprenticeship, Skills, Children and Learning Act is that Ofqual should ‘have regard to the desirability of facilitating innovation in connection with the provision of regulated qualifications’. As a regulator I want to recognise the ever-increasing impact that technology is having on the lives of learners and reflect it in the delivery and awarding of qualifications. Ofqual aims to facilitate innovation, including e-assessment, across the whole range of regulated qualifications while maintaining the integrity, reliability and validity of the assessment.

We must start by being realistic about what we mean by e-assessment and the current position regarding its use. For our purposes e-assessment is where the learner responds to questions or tasks on a computer. It does not necessarily mean that the computer carries out the assessment. Although computers can be used to make some assessments, the day when computers can be used to assess such matters as the quality of essays is very distant. Nor do we use the term ‘e-assessment’to mean the use of technology as a tool to support other aspects of the assessment process, such as on-screen marking or the standardisation of markers.

E-assessment is not new, having been used for many years in some vocational and professional qualifications in the UK and even more widely in the USA. However, it is still quite rare in large-scale, high-profile qualifications in the UK – particularly those used by schools such as the GCE and GCSE. There has been some progress but it has been slow. Our records show that in the GCSE and A levels in 2008, for example, only 421 centres used any form of e-assessment and there are only a few specifications that rely heavily upon it. Among the GCSEs that include an element of e-assessment are AQA’s science A, CCEA’s moving image arts, Edexcel’s construction and the built environment while WJEC offer e-assessments in AS level Applied Business and AS level Applied ICT.

The challenge for us is to enable the development of qualifications that make the best possible use of e-assessment while ensuring that standards are maintained. This must include, where appropriate, the comparability of standards between e-assessment and traditional pen-and­paper examinations.

New qualifications have the opportunity to use e-assessment from the start: they have no pen-and-paper versions against which to be compared. However, where the aim is to move from traditional assessment to e-assessment there will inevitably be a transition period during which some learners will be using the e-assessments while others are still using pen and paper. Ofqual needs to be able to assure users that the results of such assessments are comparable and that they need not be concerned about the mode of assessment.

There has been some research on the comparability of the different modes of candidates’ responses to questions and tasks. In many studies, particularly those using ‘simple’item types such as multiple-choice or short-answer questions, comparable scoring can be achieved. Even where there have been statistically significant differences, the effect has tended to be small. In other words, there may be a real difference but it is only very small.

We recognise that it is very hard for awarding organisations to demonstrate comparability between the modes in advance of carrying out the assessment, so we have reached a consensus with them on the technical issues involved and how to tackle them. We hope this will allow scope for innovative approaches to assessment. The consensus statement places the emphasis on making the on-screen assessment and the paper-based assessment as similar as possible. The type of question should be the same, the contexts should be as similar as possible and the amount of time available should be the same. In other words, the only substantive difference between the tests under consideration should be the mode of delivery and response.

Inevitably some candidates will be more familiar than others with the computer and its software. However, it appears that these problems can be overcome provided candidates have sufficient opportunity to acquaint themselves with the interface by taking a practice test or similar activity.

There must also be clear mechanisms to deal with cheating, which is often quoted as an argument against e-assessment. Care must be taken to ensure that a candidate cannot copy from a neighbour’s screen, make inappropriate use of the internet or exchange information if the assessment runs over a number of days. On the other hand there are techniques to combat cheating that are much easier to operate in e-assessment. Test versions can be created with different (but equivalent) questions, the order of questions can be changed and multiple choice options can appear in random order. These are issues that can be addressed, not insurmountable obstacles to e-assessment.

One question that needs to be considered is the number and length of ‘test windows’ – the time during which candidates can take their assessment. At present many test centres, and particularly schools, may not have the technology infrastructure available for all candidates, for a large examination to take an e-assessment at the same time. Is it better to have a longer time period or a number of single-day windows?

Perhaps the ultimate form of e-assessment is ‘on-demand’ testing where each assessment is unique to the candidate, tailored to his or her needs and available whenever the centre decides. Again developments in on-demand testing are more advanced in vocational and occupational assessments than in general qualifications.

Many of the organisations we have consulted suggest that on-demand testing is an essential approach to e-assessment. Where successful it gives the greatest flexibility to the learner and the centre to undertake the assessment when it is most appropriate. We have therefore commissioned researchers to develop some principles to underpin developments in this area¹. They have put forward principles designed to ensure the maintenance of examination standards, to improve accessibility for candidates, to monitor any changes in the burden of assessment and to ensure clear understanding of the processes by all involved. We are launching a project to develop and consult on a regulatory framework that facilitates on-demand testing using these principles as a basis.

We are committed to an ongoing dialogue with all those interested in developing e-assessment and on-demand testing. As part of this we have commissioned a number of pieces of work on various aspects and we have published, and will continue to publish, our findings. Some of the issues raised will be difficult but we are determined to move forward, recognising that there will be setbacks along the way. Learners will be making use of an ever-widening range of technologies and it is the responsibility of the assessment community to devise ways of assessing their knowledge, understanding and skills using the same media as they use in their schools, colleges and places of work.

As part of our communication strategy we will use e-Futures² a dedicated website on e-assessment, to help us reach all those interested in advancements. eFutures helps users move from basic awareness to strategic planning in e-assessment, offering advice and information on the essentials of e-assessment, e-testing and e-portfolios. It incorporates an online toolkit³ providing access to advice and examples of good practice that have brought plaudits from around the country.

Given our brief to encourage innovation I think it is necessary for the regulator to encourage the awarding organisations to plan for a future where there are technological solutions to current problems. We need to evaluate the obstacles to progress in developing new qualifications using e-assessment and the introduction of e-assessment to current qualifications. The agreements we have reached with the awarding organisations over comparability should make it easier for them to develop assessments that meet the requirements, even in a transition period. Awarding organisations must, however, be held to account for equality of access. So they will need to be sensitive to the needs and resources of the centres – whether they are schools, colleges or workplaces – to avoid placing undue burdens upon them. Above all, we must ensure that the needs of the learners are paramount. Their curriculum must not be distorted merely to make it easier to assess on a computer. Whatever its nature, assessment must cope with all the richness and diversity of the teaching and learning experience.

Ofqual expects that the awarding organisations will wish to be innovative and encourages them to be so by setting challenging performance standards and targets for them to achieve, subject to the readiness of the centres of learning to deliver. Some awarding organisations are and may continue to make faster progress than others. But if the market philosophy holds then the market will generate competition and ensure that all reach our expected standard more quickly than some of the awarding organisations – and some observers – would think possible.

1. www.ofqual.gov.uk/2523.aspx
2. www.efutures.org
3. http://toolkit.efutures.org

The great increase in the number of qualifications that are unitised has been a feature of the last few years. Instead of facing a single assessment or examinations at the end of a lengthy course, learners are able to undertake assessments at the end of each section or ‘unit’along the way. This approach, which is sometimes called ‘modular’, has many benefits for learners such as providing the information they need to find out how well they are doing while there is still time for them to improve. It also reduces the number of candidates who have nothing to show for their years of work if they fail the final assessment or overall qualification.

Many vocational qualifications – including those required for entry into the professions – and many university degrees are built up over a lengthy period. AS and A levels have been successfully unitised for some years and there is now considerable experience of this approach. GCSEs are moving in the same direction. In addition the Qualifications and Credit Framework (QCF) comprises units that can be combined into qualifications, providing students with the flexibility to manage their learning as best suits their circumstances. Most qualifications, therefore, look set to follow this pattern in future.

The need to maintain standards across a unitised structure does, however, pose a particular challenge for Ofqual, which has a legal duty to check that the qualifications it regulates:

• give a reliable indication of knowledge, skills and understanding
• indicate a consistent level of attainment (including over time) between comparable regulated qualifications.¹

In unitised qualifications standards are set at the unit level but the ultimate focus of candidates, users of certificates, society and regulators is on the standard of the whole qualification. We need to be confident that the qualification is more than the sum of its parts. How can we best assure ourselves of the standard of the qualification as a whole?

Ofqual has addressed this issue in a paper² presented to the Cambridge Assessment Conference in October. In the paper we explore several possible responses to the challenge posed by unitised qualifications:

• To maintain that it is impossible to ensure standards at the level of the qualification because all the standards are in the units (or modules or components). So the qualification standard is either redundant (if the qualification is the sum of its part) or unimportant. A regulator cannot accept this argument.
• To build the overall standard into the design of the curriculum and the syllabus. To an extent this is what the GCE and GCSE awarding organisations attempt to do when they develop specifications that meet the appropriate criteria and codes of practice.
• To try to reflect the standard in the structure of the qualification. This is a key feature of the QCF in which some units may be hurdles without which the qualification cannot be awarded.
• To intervene at the awarding stage, after the learner has completed the units but before he or she receives the result of the qualification, in order to adjust the overall grade. The paper offers a number of variations of this approach but they all involve the removal of the automatic relationship between the units and the overall qualification.
• To accept the results of the automatic process, identify the problem and put it right for the next round.

In considering how to respond to this challenge we must recognise that learners have a right to expect that they can act upon the information they are given about their performance in a unit. This is the essence of the unitised structure. So any attempt to modify the results after the earners have been given that information must be unacceptable.

Nevertheless, Ofqual must be in a position to assure users – learners, providers and employers – that the qualification as a whole represents achievement of the content, level and size indicated by its title. How is this to be achieved?

My position is that all the organisations involved, not just qualification awarding organisations, have a duty to maintain the standards of the qualification as a whole. Before an organisation is recognised, whether to award single units or whole qualifications in any capacity, it must satisfy us that it has the ability and capacity to maintain standards – not just of the particular unit for which it is directly responsible but as a contributor to the qualification as a whole. Unit submitting organisations³must be able to show that they can write good-quality units of assessment at appropriate levels, with reliable credit values and clear learning outcomes and assessment criteria. They must also demonstrate that their units will define and maintain the overall standard of the qualification the learners will achieve at the end of their studies.

In the same way organisations that are submitting rules of combination will need to show that their processes clearly define qualifications that will be valuable to learners and to employers, with meaningful titles that accurately reflect their size, level and content. Organisations undertaking the assessment must also show that they have appropriate assessment and awarding processes that contribute to the award of a qualification that is at the stated level.

The same is true for modular qualifications in GCSEs, AS and A levels and many vocational qualifications where a single awarding organisation is responsible for the whole qualification. Ofqual’s position is clear: once the award of the unit has been made it must be honoured. It would be unfair to the learner if changes were made solely to bring the spread of grades in line with what could be expected.

So I believe that the system itself must ensure that the aggregation of the credits or unitised marking scale marks a learner has gained will yield a satisfactory overall qualification. This reflects the spirit of decentralisation that underlies our approach to regulation (see Section 4). It is appropriate to place the onus of overall quality on those providing the component qualifications or units and initially to trust them to do so unless it is shown that that trust is misplaced.

The qualification awarding organisations and regulators remain responsible for overall standards and must constantly check that they are being maintained. The qualification awarding organisation will be expected to be vigilant in keeping all the awards it makes under review and identifying any that give cause for concern. These can be dealt with internally or through Ofqual’s monitoring and enforcement procedures, identifying any aspect that needs modification to bring it into line and alerting the appropriate organisation to take remedial action.

Unitised and composite⁴qualifications are here to stay. They represent a step forward in the openness they bring to the assessment process and they allow learners to take an active role in planning and managing their own learning. For awarding organisations and regulators they represent a challenge, but one that we must accept willingly. It is our duty to find ways of ensuring that all qualifications give learners a fair indication of their achievements.

1. Apprenticeships, Skills, Children and Learning Act – http://www.opsi.gov.uk/acts/acts2009/pdf/ukpga_20090022_en.pdf
2. www.ofqual.gov.uk/2565.aspx
3. A unit submitting organisation is an organisation recognised by the regulators to develop and submit units to form part of qualifications accredited on to the QCF.
4. A qualification consisting of a number of constituent accredited qualifications

As I mentioned in Section 1, this was the first year in which Diplomas have been awarded. A long-standing problem of our education system has been the value accorded to vocationally-oriented qualifications for the 14–19 age range. They have tended to be judged against academic qualifications which, by their very nature, are addressing different aspects of education in very different ways. As a result the history of qualifications in England is littered with failed initiatives: the Certificate of Pre-vocational Education (CPVE), the General National Vocational Qualification (GNVQ) and vocational A levels are but a few vocational qualifications that have fallen by the wayside.

Diplomas are a serious attempt to challenge the academic/vocational dichotomy by recognising learning in applied settings and providing learners with a broader base of learning and achievement with a focus on particular lines of learning. They emphasise applied learning, and teachers and young people have indicated that this approach has been attractive and has succeeded in bringing learning to life. They have also encouraged schools, colleges and employers to work together.

As well as demonstrating prowess in their principal learning qualification learners are required to submit a project in which they have to research, analyse and synthesise information in ways that will be useful in their later careers. They also have to:

• pass functional skills qualifications in English, mathematics and ICT
• demonstrate personal learning and thinking skills
• undertake a significant period of work experience
• show depth or breadth of learning through additional and specialist l earning qualifications.

The requirement to succeed in all of these aspects results in something truly challenging that is relevant not just to the learners involved but to society as a whole.

The nature of the Diploma requires new thinking about what achievement means and the time a student needs to acquire it. Traditionally progress and achievement have been linked with the speed with which a student moves through the course and emerges with a qualification. Now out of the window goes this traditional view of one- or two-year courses with a once-and-for-all assessment at the end. For most learners, although they will have the Diploma in their sights from the beginning, the prime focus of their studies will be the constituent components. The time taken by learners to accrue all the necessary components will vary but along the way all learners will have passed milestones. The results they get on the component qualifications will give them the satisfaction of gaining recognition for the progress they are making and this will motivate them to succeed in the Diploma.

How is the regulator to approach such a new concept of challenge and achievement – new, that is, for England but in principle not so for our European counterparts where baccalaureate-style examinations have been the norm for many years? Should we apply conventional yardsticks to judge the success of the Diploma in its early days, knowing full well that it is breaking new ground with new forms of assessment and new challenges? It seems to me that not only the regulator but also society should guard against applying inappropriate criteria or expecting too much of a differently constructed qualification in its infancy.

All new qualifications have teething troubles that need to be addressed in ways that nurture its development. Ofsted has identified potential improvements that can be made in the Diploma programmes. From Ofqual’s point of view it is becoming clear that in the longer term there may need to be some simplification of the qualification’s structure if it is going to be taken by large numbers of students. Meanwhile Ofqual is working closely with the Department for Children, Schools and Families (DCSF) and QCDA to make sure that centres and awarding organisations have the central support they need to meet the significant challenge of delivering awards in 2010 in much greater numbers than in 2009.

It is government policy that all young people should remain in education and training until the age of 18. This means that young people who would otherwise have dropped out will be retained within the system. It is my view that the qualifications system – including Diplomas – has an important role to play in providing qualifications that meet the diverse needs of this new group of learners. If their aspirations are to be met we must ensure that they have access to an appropriate range of qualifications that are meaningful and valuable, and able to be ‘personalised’to meet the varied needs of individual learners.

Everyone who starts a course of study for a Diploma should be expected to complete their studies successfully. They should accrue the constituent parts over the time that they need to complete their studies successfully – an arbitrary cut-off of one or two years may be inappropriate for some students. We should emphasise the value that participants gain from having to work collectively as well as independently, from learning to evaluate their own work. In addition we must value the planning that goes into the project and the discipline needed to complete it.

As work proceeds to improve the rules and procedures underpinning the Diplomas, I would like to set out some longer-term, underlying principles as a guide for developers and to inform Ofqual’s ongoing role in regulating the Diploma:

• The design and requirements for the Diploma should be understandable to all involved (learners, teachers, employers and awarding organisations).
• The detailed regulatory rules and requirements around additional and specialist learning should be as simple as possible. Teachers and tutors should be able to use their professional judgement to shape learners’curricular choices, guided by broad principles rather than detailed rules.
• Each qualification within the Diploma should be valued as important in its own right with a title that reflects its content, without detracting from the additional value of the Diploma as a whole programme of learning.
• Learners who excel or struggle with the content of the principal learning course at one level should be able to transfer as easily as possible to another level within a line of learning.
• Ofqual must be able to assure the public that grades are comparable in graded elements and that standards are comparable across awarding organisations and lines of learning for achievement for the qualifications that comprise the Diploma.
• The responsibilities of the recognised organisations delivering Diplomas and component qualifications should be clear and Ofqual should be able to check that they deliver these responsibilities.
• The qualification should be value for money to the nation. A new qualification of this kind can be expected to have significant development costs. As it rolls out, however, the processes involved should be streamlined and administrative costs reduced.

I hope that these principles will be able to inform the future development of Diploma qualifications. Any new types of Diploma should be designed with these principles in mind and should be fully piloted before being introduced in large numbers.

My first report concentrated on standards in general qualifications. In this report I want to highlight our understanding of standards in vocational and occupational qualifications.

In vocational and occupational qualifications the standards are determined by the knowledge and skills needed to do a job and show that the learner is competent in the area of work required in their sector. Many skills may be exclusively the province of that particular sector. It follows, therefore, that comparisons of qualifications across sectors are of limited value.

Most qualifications that are used in the workplace are based upon the national occupational standards, which ‘specify the skills, knowledge and understanding required to perform competently to the standards required in employment’.¹ These are developed by the Sector Skills Councils and overseen by the UK Commission for Education and Skills (UKCES).

The national occupational standards must reflect what each sector requires at the time. That means that they may have to be revised to keep up with changes in the sector. Technological advancements and legislative changes need to be reflected in the standards to ensure qualifications keep pace.

At any given time the qualifications accredited for a particular sector will reflect the current national occupational standards. Awarding organisations develop sector-based qualifications independently but they are all based on the same national occupational standards so comparable outcomes will be expected, with standards maintained over time during their accreditation period. The national occupational standards approval system operated by UKCES, the level descriptors published and maintained by Ofqual and the other regulators and Ofqual’s scrutiny of awarding organisations’² qualifications development systems help to ensure that, for example, a level 2 competency qualification in engineering will be a comparable achievement to a level 2 competency qualification in construction or a level 2 general qualification.

When national occupational standards change new qualifications will be developed to reflect the new sector requirements. It then becomes less relevant to talk about standards over time as standards in qualifications for work have to be related to changing job roles. However, it will still be possible to say that someone who gained a level 2 skills-based qualification in engineering in 2009 is operating at the equivalent level within the sector as someone who gained a similar qualification in 2004.

Assessment methods in vocational and occupational qualifications vary enormously. A knowledge-based qualification may be assessed by a written examination very similar to a general qualification; a skills-based qualification may be assessed by observation of the candidate putting their skills into practice, with their underlying knowledge tested perhaps by an online multiple-choice question paper or by answering oral questions.

The assessment is the responsibility of the awarding organisation with the candidates attached to a centre – a college, a training provider or an employer – which has been approved by the awarding organisation offering the qualification. If there is to be a written or ICT-based examination or test the awarding organisation will set and mark it and will require the centre to administer it in a standard way. If assessment is done by observation or by centre-based assignments (internal assessment) the awarding organisation will lay down the requirements for how it is to be carried out and quality assured. For example if a qualification is assessed entirely by observation and oral questioning there will need to be both a qualified assessor and a qualified internal verifier based at the centre and the awarding organisation will send an external verifier to check that procedures have been correctly followed. Ofqual and its fellow regulators specify how different assessment methods are to be applied via their regulatory arrangements and will check that awarding organisations are implementing the requirements correctly.

One difference between general and vocational qualifications is that the learner must provide evidence of achieving all the requirements of the level. There is no possibility of higher achievement in one area balancing lower performance in another. The requirements are defined and what is required as evidence is clearly laid out. The certificate identifies not only the overall qualification and the level achieved but lists clearly, for all to see, the units the learner has successfully completed.

The awarding organisations must approve their centres in much the same way that Ofqual recognises awarding organisations: by checking they have the necessary knowledge, expertise and resources to carry out the assessments accurately and consistently, particularly when the qualifications are mainly internally assessed. Just as Ofqual requires the awarding organisations to undertake their duties conscientiously, so must the awarding organisations rely on their centres.

Awarding organisations must have in place appropriate mechanisms to check that centres follow procedures and must be prepared to take appropriate action when problems arise or if they have concerns. Centres can earn the right to be regarded as lower risk by having – and following – robust procedures. The awarding organisation will look at how learners are guided to make sensible choices of the qualifications and levels available and at the quality assurance mechanisms and processes to ensure that assessment is carried out objectively and without bias.

Ofqual’s role in guaranteeing standards in vocational qualifications is therefore very different from our work in general qualifications such as the GCSE or A level. Others are responsible for the articulation of the standards and we are more concerned with quality assurance mechanisms – which are built into the process of assessment and certification – rather than quality control methods that are more appropriate to end-of-course assessments.

1. UKCES definition: www.ukces.org.uk//upload/pdf/071115%20D%20Final%20NOS%20defnitions%20paper.pdf
2. Ofqual and the regulators of external qualifications in Wales (DCELLS) and Northern Ireland (CCEA).