Regulators publish research into the use of statistics in the awarding of A levels
Friday 30 September 2011
The exams regulators for England (Ofqual), Wales (DfES) and Northern Ireland (CCEA) today publish independent research into the use of statistical predictions to support the A level awarding process.
The GCSE results of all candidates entered for a subject are used to produce predictions of likely outcomes for A level results. These predictions are for the cohort as a whole, not individual candidates. The ‘prediction matrices’ are then used to guide examiners, alongside evidence from reviews of candidate work, during the process of setting grade boundaries.
Research carried out by NFER looked at how consistently the approach is used across the awarding organisations and explored potential ways to improve the system.
It found that the approach taken to the use of predictions matrices was sound and recommended that current practices continue. The research did recommend a slight amendment to the tolerance limits set around the predicted outcomes to take into account the number of entries for particular subjects. The tolerance limits act as triggers for awarding organisations to report differences from expectations to the regulators.
Outcomes can remain outside of these limits, if the difference to the predictions can be explained. Following discussions with the awarding organisations this recommendation was acted upon in time for the summer 2011 awards.
Four of the awarding organisations delivering A levels - AQA, Edexcel, OCR, WJEC - use a single prediction matrix which includes candidates from all three countries. In Northern Ireland CCEA uses a separate matrix which includes only candidates from Northern Ireland.
The research looked specifically at this issue and found that higher outcomes of CCEA candidates compared to those from other awarding organisations, when taking into account other factors, were most likely due to the use of the separate matrix. CCEA is to carry out further investigations into this issue.
Research was also carried out into whether or not having an additional compulsory subject in Wales (Welsh first or second language, which a high proportion of candidates take as a GCSE) had any impact on the accuracy of the predictions for candidates in Wales. It was found not to be the case.
The report, 'Investigating the relationship between A level results and prior attainment at GCSE', can be found on the regulators’ websites.
View full reportPDF 303 kb