How the UK's algorithm-based grading fell apart

Students, parents and educators alike expressed their anger and disappointment over the algorithm’s The backlash to Ofqual’s algorithm was only matched by its complexity. The students in each class or year group were ranked and roughly matched up with the grades range provided by Ofqual.

Curated via Twitter from Engadget’s twitter account….

On August 12th, the day before the A-level results were published, education secretary Gavin Williamson said students could choose their mock result or sit the relevant exam in the fall if they weren’t happy with their algorithm-decided grades. “This triple lock system will help provide reassurance to students and ensure they are able to progress with the next stage of their lives,” he promised.

Otherwise, applicants will have to go through clearing — a system in the UK that matches learners to unfilled university places — or consider deferring a year. “It is vital that information is provided speedily on how this decision will impact higher education institutions, students wishing to apply through clearing and those who may have been rejected on their original grades,” David Hughes, CEO of the Association of Colleges said.

Ofqual explained in a technical document that this approach, dubbed Direct Centre-level Performance (DCP), assumed “that a center will perform the same in a subject this year as they have across recent years. ” It was also designed to take into account “any changes in underlying ability of students. ” The hope was that the system would respect the talent of the learners who were unable to take their exams, while simultaneously delivering results that didn’t seem out of the ordinary.

This wasn’t enough, though, because many students didn’t have old grades to reference.  Some didn’t sit the relevant exams, while for others it wasn’t possible “to reliably link the student back to their prior-attainment measure,” Ofqual explained in a technical document.  Ofqual had to give prior attainment weighting based on the number of students that had available data.

Roger Taylor, chair of Ofqual, said the regulator was “extremely sorry” for the “real anguish” it had caused students and the inevitable damage to public trust in the education system. “There was no easy solution to the problem of awarding exam results when no exams have taken place,” he said in a statement.

Then, Ofqual looked at how results shift between the qualification in question and students’ previous achievements. (For A-levels, this ‘prior attainment’ would mean GCSE grades. ) It used the “relationship” between the two to predict a general range of grades for a “historical” cohort and the current year group at each institution.

The algorithm then combined everything — the historical grade distribution, the “relationship” between GCSEs and A-levels, Ofqual’s initial predictions, and the ‘prior attainment’ for this year’s crop of students — to create a rough set of grades for every school and college, broken down by individual subjects.

Analysis published by the paper showed that pupils with lower socioeconomic backgrounds were more likely to be downgraded than those in wealthier areas. “At low-performing schools, high-performers have been shifted down,” Richard Wilkinson, professor of statistics at Nottingham University told New Scientist. “They have all been shifted towards the average of the school performance over the previous three years.

Ofqual was sure, however, that they would be “broadly in line with previous years” and, if anything, slightly higher than those recorded last year. “We will make sure there isn’t any significant change in year on year results which would undermine the value of the qualifications,” Ofqual explained in its guidance document.

The day after the A-level results were published, the University of Leicester said it would be offering places based on mock exam results and published results. “Whichever is higher,” the institution promised on Twitter.

The rules were immediately criticized. “It is attempting to remedy the grading fiasco through an appeals process so surreal and bureaucratic that it would be better off at this point doing that U-turn and allowing original teacher-assessed grades, where they are higher, to replace moderated grades,” Geoff Barton, general secretary of the Association of School and College Leaders said.

The pandemic had already created uncertainty about how schools, colleges and universities might reopen; now a man-made algorithm was clouding students’ futures even further. “Something has obviously gone horribly wrong with this year’s exam results,” Sir Keir Starmer, the leader of the opposition Labour Party tweeted.

Link to original article….

Leave a Reply

Leave a comment
%d bloggers like this:
scroll to top