Current position

All key stage test questions are trialled when they are being developed, and test developers routinely scrutinise pre-test scripts to see how pupils have responded to individual items and to identify any problems of accessibility. In addition, test developers may undertake small-scale informal trialling of the questions.

For a small-scale informal development trial, pupils are normally observed as they work through some draft questions, and are then invited to comment on these. This strategy may be used in the early stages of the development of the questions, and it can provide useful information about their accessibility for a range of test takers.

Proposal

While teachers and expert reviewers may offer valuable insight into the nature of the questions, the people who have the greatest involvement and interest in the tests are the pupils themselves. Input from the pupils might now be extended to evaluate and assure the accessibility of the final tests.

This new approach, which may be described as 'small-scale discursive trialling', might involve the following steps:

• a researcher would work with a Pupil Review Panel of just four pupils with special educational or assessment needs who had taken the current year’s tests and were working at an appropriate level
• the researcher would invite the pupils to think about and write their responses to a single question, and note any evidence relating to their methods or reasoning
• the pupils would then discuss the question in detail as the researcher probed their reasoning, checking to see whether they understood what it was asking even if the problem posed was beyond their current knowledge or understanding
• time permitting, the panel would go on to discuss several more questions in detail in the same way.

This discursive trialling would focus on certain questions that would be identified as possibly presenting particular issues of accessibility to the pupils on the Pupil Review Panel.

The specific prompting questions that the researcher would ask the pupils would depend on the situation, on the nature of the test item being discussed, and on the researcher's observations of the pupils' responses as they worked through it. The researcher would be likely to address specific prompts to individual pupils.

These prompts might include:

• What is this test question asking us to do?
• How did you go about thinking about the question? Can you explain what you did?
• I see you wrote …… Can you explain why you wrote that?
• Chris, I see you got the same answer as Ali. Can you explain how you did it? Did you do it in the same way as Ali?
• Josh has explained how he did this question. Did anyone do it in a different way?

Responding to Proposal 2

Please complete the response form online. The form is available to print at Annex 1 of the PDF version of this consultation.