In 2013, the Computer Science Department at the University of Texas at Austin began using a self-made machine learning algorithm to assist faculty in making graduate admissions decisions. Seven years later, the system was abandoned and drew criticism that it should not have been used.

The algorithm, based on past admissions decisions, saved faculty time. It used factors such as whether a candidate graduated from an "elite" university or whether recommendation letters contained words like "best" as predictors of admission.

The university stated that the system never made admissions decisions independently, and at least one faculty member would review its suggestions. But critics argued that it encoded and legitimized any biases that might exist in admissions decisions.

Today, artificial intelligence is in the spotlight. ChatGPT—an AI chatbot that generates human-like conversations—has attracted widespread attention and reignited discussions about which parts of human life and labor might be easily automated.

Despite criticism of systems like the one previously used at UT Austin, some universities and admissions officials are eager to use AI to streamline the admissions process. And companies are eager to help.

"Demand has increased dramatically," said Abhinand Chincholi, CEO of AI company OneOrigin. "The release of GPT—technology like ChatGPT—has made everyone want AI." But he noted that universities interested in AI are not always clear about what they want to use it for.

Chincholi's company offers a product called Sia, which quickly processes university transcripts by extracting information such as courses and credits. After training, it can determine which courses new or transfer students may be eligible for and push the data into the institution's information system. The company says this can save admissions officials time and potentially reduce university personnel costs.

Chincholi said the company is working with 35 university clients this year and is in the implementation phase with another 8 universities. It also receives about 60 information requests from other institutions each month. Although questions remain about new uses of AI, Chincholi believes Sia's work fully aligns with ethical concerns.

"Sia provides clues on whether to continue processing an applicant," he said. "We would never let AI make such decisions because it is very dangerous. You are playing with students' careers and lives."

Other AI companies go further. Student Select is a company that provides algorithms for universities to predict admissions decisions. Its Chief Technology Officer, Will Rose, said the company typically first reviews a university's admissions scoring criteria and historical admissions data, and then the technology classifies applicants into three tiers based on likelihood of admission.

He said applicants in the highest tier can receive faster approval from admissions officials and receive admissions decisions earlier. Students in other tiers are still reviewed by university staff.

Student Select also provides universities with what Rose calls "applicant insights." The technology analyzes essays and even recorded interviews to find evidence of critical thinking skills or specific personality traits. For example, an applicant's use of the word "flexibility" when answering a specific interview question might indicate "openness to experience"—one of the personality traits Student Select measures.

"Our company started more than a decade ago as a digital interview platform, so we know very well how to analyze job interviews and understand traits from them," Rose said. "Over the years, we found that we could do similar analysis in higher education."

Rose said Student Select has contracts with about a dozen universities to use its tools. Although he declined to name them due to contract terms, Government Technology reported in April that Rutgers University and Rocky Mountain University are clients. Neither university responded to requests for comment.

Black box?

Not everyone thinks it is a good idea for admissions offices to use such technology. Julia Stoyanovich, a professor of computer science and engineering at New York University, advises universities to stay away from AI tools that claim to predict social outcomes.

"I think using AI is not worth it, really," said Stoyanovich, co-founder and director of the Responsible AI Center. "We have no reason to believe that their speech patterns or whether they look at the camera have anything to do with how good a student they are."

Stoyanovich noted that part of the problem is the opacity of AI. In medicine, when AI flags possible cancer in medical images, doctors can review its work. But in university admissions, there is little accountability when AI is used. Admissions officials might think the software is screening for specific traits, when in reality it might be screening for something spurious or irrelevant.

"Even if we believe there is a way to do this, we cannot check whether these machines work. We do not know how people who were not admitted would have performed," she said.

When algorithms are trained on past admissions data, they repeat existing biases. But Stoyanovich said they go further by endorsing these unequal decisions. Additionally, algorithmic errors can disproportionately affect marginalized groups. For example, Stoyanovich mentioned Facebook's method for determining whether names were legitimate, which sparked controversy in 2015 for kicking Native American users off the platform.

Finally, admissions staff may lack the training to understand how algorithms work and what judgments are safe to make from them.

"You need at least some background knowledge to say, 'I am the decision-maker here, and I decide whether to accept this recommendation or reject it,'" Stoyanovich said.

With the rapid development of generative AI systems like ChatGPT, some researchers worry that future applicants will use machines to write essays, and those essays will be read and scored by algorithms. Les Perelman, a former associate dean at MIT who has studied automated writing assessment, said, "Having machines read essays will 'further encourage students to generate essays with machines.' It will not be able to tell whether an essay is original or generated by ChatGPT. The whole issue of writing assessment is truly turned upside down."

Proceed with caution

Benjamin Lira Luttges, a doctoral student in the psychology department at the University of Pennsylvania, who is studying the use of AI in university admissions, said human flaws contribute to some of the problems associated with the technology.

"One reason admissions is complex is that, as a society, we are not clear about what we actually want to maximize when making admissions decisions," Lira said in an email. "If we are not careful, we might build AI systems that maximize things that are not socially desirable."

He said there are risks in using the technology, but also benefits. Machines are not affected by emotions or weather like humans, and can make decisions without "noise."

"We do not have very good data on the status quo," Lira said. "Algorithms may be biased and may have aspects we do not like, but if they perform better than human systems, then gradually deploying algorithms in admissions might be a good idea."

"If we are not careful, we might build AI systems that maximize things that are not socially desirable."

—Benjamin Lira Luttges, doctoral student at the University of Pennsylvania

Student Select's Rose acknowledged that there are risks in using AI in admissions and hiring. He noted that Amazon abandoned its algorithm after it was found to discriminate against women in hiring. But he said Student Select avoids these negative outcomes. The company first conducts a bias audit on a client's past admissions results and regularly checks its own technology. Rose said its algorithm is quite transparent and can explain the basis for decisions. The analysis produces results where average scores across subgroups are equal, verified by external scholars, and is not entirely new.

"We use internal and external researchers to develop this tool, and all these experts are specialists in the field of selection," he said. "Our machine learning models have been trained on datasets containing millions of records."

Beyond the ethical issues of using AI in admissions, Stoyanovich also pointed out practical problems. When errors occur, who will be responsible? Students may want to know why they were rejected and how candidates were selected.

"As an admissions official or university admissions director, I would be very cautious when deciding to use algorithmic tools," she said. "I would be very careful to understand how the tool works, what it does, and how it is validated. And I would pay close attention to its long-term performance."