Making AI trustworthy in high risk industries
Miami computer scientist Honggang Yu part of NSF-funded research team to improve the security and trustworthiness of artificial intelligence used in nuclear reactor operations
Making AI trustworthy in high risk industries
Miami computer scientist Honggang Yu part of NSF-funded research team to improve the security and trustworthiness of artificial intelligence used in nuclear reactor operations
As AI has gained a foothold in our society, cybersecurity concerns are increasing especially in high-stakes environments. With AI applied in such critical areas, trust and security in these programs has become essential.
Honggang Yu, assistant professor of Computer Science and Software Engineering at Miami University, is addressing this challenge with colleagues from Lehigh University and Kansas State University by researching AI security in nuclear reactor operations. The team recently received a grant from the U.S. National Science Foundation, totalling approximately $900,000, with $223,000 allocated to Miami University.
As the U.S. continues to explore advanced nuclear technologies, including next-generation reactors and microreactors, ensuring the security of AI-enabled systems will be increasingly important to protect critical energy infrastructure, Yu said.
Within nuclear reactor operations, Large Language Models (LLMs) are often used for their ability to quickly process documents, read parameters, and propose recommendations to help operators make decisions. However, the recommendations generated by LLMs are not always accurate, meaning that operators cannot inherently trust their outputs, particularly when incorrect recommendations could lead to serious safety consequences. LLMs may also be vulnerable to malicious backdoor attacks designed to manipulate their behavior and cause them to generate incorrect recommendations.
Deploying LLMs in nuclear reactor environments presents another challenge: computing resources near reactor systems may be limited, and relying on cloud-based models may not always be practical or desirable. To combat this, the team has been working with edge LLMs, which are smaller models that require fewer computing resources and can operate locally rather than relying on remote cloud infrastructure. However, local deployment does not eliminate cybersecurity risks. Yu’s research focuses on how to securely deploy these edge AI systems in nuclear reactor environments.
“I’m going to fill the gap by developing defenses to enhance the security and privacy of the large language model in the nuclear reactor,” Yu said.
He aims not only to make these systems more secure, but also to make their behavior more explainable by understanding how and why LLMs produce incorrect predictions. In a high-stakes environment such as a nuclear reactor, this could help operators better understand when AI-generated recommendations can be trusted.
As part of this research, Yu is analyzing LLM behavior and model data to identify vulnerabilities and develop defenses against potential attacks. Miami graduate student Yuanfang Peng and undergraduate students Ryan Cromwell and Robert Bradford are also contributing to the project by training AI models and developing code.
“This award will help strengthen AI cybersecurity research at Miami University and give our students more opportunities to participate in funded research,” Yu said. “It will also help them build connections with researchers at other universities.”
The project will not only help prepare students for their future careers, but also provide them with hands-on experience in programming and working with a variety of AI models.
“We are definitely going to help train the next generation of the U.S. cybersecurity workforce,” Yu said.
The project will culminate in the release of security tools designed to protect LLMs used in high-stakes environments. By helping make AI systems more secure, explainable, and trustworthy, Yu’s research could support the safe adoption of AI in future nuclear reactor operations and strengthen the cybersecurity of critical U.S. energy infrastructure.
Beyond this broader impact, the research will also directly benefit Miami University, as findings from the project could be incorporated into introductory AI courses and be used to develop new courses in AI cybersecurity.