School leaders have called for the Department for Education to be “fully transparent” about the role of artificial intelligence in analysing the curriculum and assessment review’s call for evidence.
AI techniques are among the tools “likely” to be used in the analysis, according to a DfE contract specification for suppliers interested in carrying it out.
And the company that won the contract, Alma Economics, stresses its AI credentials on its website.
‘Reassurance’ needed
The NAHT school leaders’ union is concerned that AI tools may fail to pick up on important nuance, potentially affecting the outcome of the review.
Given the possibly far-reaching consequences of the review’s conclusions for schools and pupils, the union is calling for full transparency from the government.
Sarah Hannafin, the NAHT’s head of policy, said: “When people and organisations have taken the time and trouble to respond in detail to such an important review, they deserve reassurance that their input has been properly considered.
“It is therefore incumbent upon the Department for Education to be fully transparent about whether and how AI has been used in analysing responses to this call for evidence, and how it can be sure that the technology accurately takes account of all the responses submitted.”
The call for evidence closed on 22 November 2024, and more than 2,350 of the 7,000 responses were made by teachers, lecturers and school and college leaders.
Pepe Di’Iasio, general secretary of the Association of School and College Leaders, urged the DfE “to be clear about how AI was apparently used in analysing responses to this consultation”.
At no point so far has the DfE or any member of the review panel informed the public or those who submitted evidence that AI has been used to review and analyse their submissions.
‘Important nuance’ could be missed
Of particular concern to the NAHT was the possibility that analysis using AI may miss the nuances of views expressed by education experts, as it has not previously been tried and tested.
“We have seen no evidence of any pilots on the use of AI to analyse responses to education-related consultations, so we simply do not know how effective it is - especially for an exercise on such a large scale,” said Ms Hannafin.
She added: “The worry is that, while it may be able to identify broad trends in responses, it may miss important nuance, detail and examples, and ultimately affect the final recommendations of the review.
“These could have far-reaching implications for pupils and schools, so it’s vital the review gets this right.”
Mr Di’Iasio also called for the DfE to be transparent about “what checks and balances were put in place to ensure that all responses were fully taken into account”.
He added: “It is very important to be transparent about such matters and ensure that people contributing to consultations can have confidence in the process.”
The need for a ‘human in the loop’
The order specification of the DfE contract for suppliers wishing to bid to carry out the analysis stated: “The contractor is expected to apply robust, cost-effective analytical techniques. These can be determined by the contractor and are likely to include innovative ways to analyse large volumes of qualitative data, such as through AI techniques like natural language processing.”
In the “analysis and methodology” section of the spec, the DfE said: “We expect a very large number of responses, and due to the open nature of the Call for Evidence, AI and similar techniques offer an opportunity to support the analysis.”
It added: “The supplier should demonstrate how they have considered ways to maximise the utility of the content of open questions for further analysis using such techniques, that potential ethical considerations have been considered and that the approach ensures a ‘human in the loop’ to validate the finding.”
Alma Economics’ website states: “AI is a transformative technology. At Alma we are at the forefront, harnessing AI to combine language and statistical analysis within our models and products.”
Alma deploys AI “safely and responsibly, ensuring humans are always in control”, the website also says.
Alma has a tailormade tool for analysing public consultations that heavily incorporates AI, called Cobflow, which it describes as “the standard for consultation analysis”.
On its website, Alma writes that “Cobflow incorporates powerful AI capabilities to help automate time-intensive tasks. Strong built-in safeguards ensure full compliance with statutory requirements and effective human control of all AI-produced output”.
Tes has asked the DfE and Alma to confirm whether Cobflow was used in analysing the call for evidence.
Tes has made multiple attempts to contact Alma Economics, and asked the DfE on 13 May to confirm or deny whether AI or machine learning had been used to analyse the call for evidence.
Despite following up several times since, no response has been received.
