The Information Commissioner’s Office said AI discrimination could have “harmful consequences for people’s lives” and lead to someone being turned down for a job or wrongfully denied a bank loan or welfare. It will investigate the use of algorithms to screen job applications amid concerns they are affecting employment opportunities for people from ethnic minorities. “We will investigate concerns about the use of algorithms to review recruitment applications, which could adversely affect the employment opportunities of those from different backgrounds,” the ICO said. The investigation is being announced as part of a three-year plan for the ICO under the UK’s new information commissioner, John Edwards, who joined the ICO in January after running its New Zealand counterpart. In a speech on Thursday, Edwards is expected to say that the ICO will “consider the impact that the use of artificial intelligence could have on groups of people who are not involved in the trials for this software, such as people who are neurodiverse or people from national minorities”. The CEO of ZipRecruiter, a job site, told the Guardian this year that at least three-quarters of all resumes submitted for jobs in the US are read by algorithms. A survey of executive recruiters conducted by research and consulting firm Gartner last year found that nearly all reported using artificial intelligence for part of the recruiting and hiring process — for example, sifting through applications before anyone sees them. Under the UK General Data Protection Regulation, which is enforced by the ICO, people have the right not to be discriminated against when their data is processed. The ICO has previously warned that AI-based systems could produce results that disadvantage certain groups if they are not accurately or fairly represented in the data set on which the algorithm is trained and tested. The UK Equality Act 2010 also offers individuals protection against discrimination, whether caused by a human or an automated decision-making system. Dr David Leslie, director of ethics and innovation research lead at the Alan Turing Institute, said: “The use of data-driven AI models in recruitment processes raises a number of thorny ethical issues, which require forward thinking and careful assessment and by the side of both. system designers and suppliers. “Fundamentally, predictive models that could be used to filter job applications through supervised machine learning techniques run the risk of reproducing or even exacerbating patterns of discrimination and structural disparities that could be embedded in the datasets used for training their”. Elsewhere in its three-year plan, the UK watchdog will consider whether to prioritize the public interest when dealing with complaints about freedom of information requests. The ICO said changes to the complaints process for the FoI regime – which gives members of the public the legal right to request official information from public bodies – are necessary due to increasing complaints and fewer resources to deal with them. Edwards said the current system for dealing with FoI complaints – for reasons including late responses or incomplete disclosures of information – was not working and he would consult on a new regime. The number of complaints reached 6,361 in the year to 30 April 2022, which was a slight drop on the previous year but compared to 5,433 in 2016/17. “I want to explore whether we should be able to highlight some cases or bring them out and say: this is moving forward because it’s in the public interest,” Edwards said. Subscribe to First Edition, our free daily newsletter – every morning at 7am. BST Edwards said the agency expected to avoid accusations of bias in what it prioritized by publishing a list of criteria for reviewing complaints. Edwards said the ICO would consider prioritizing submissions from journalists and MPs, although it would also consider a “blind applicant” system where the identity of the person or organization making the submission was withheld. Information held by public authorities in Scotland is covered by the Scottish commissioner and the Freedom of Information Act. “We have to differentiate between who is making the request … and saying that a journalist is acting on behalf of the wider public to shed light on a particular issue. Therefore, they get extra points. A member of parliament, for example, has a very special role in holding the executive and public authorities to account, so there may be a case for additional weight to be added to them [requests].”
