In medical research involving human subjects, artificial intelligence (AI) in the form of large language models poses privacy risks, because these models tend to memorize, and can leak, information about the data used to train them. Yet this same capacity for memorization is part of what makes the models powerful.
That tension is the focus of a five-year, $4.2 million grant from the National Institutes of Health to Luca Bonomi, PhD, Assistant Professor of Biomedical Informatics. The grant is an NIH Director’s New Innovator Award, from the NIH Common Fund’s High-Risk, High-Reward Research program.
New Innovator Awards support unusually innovative research by early-career investigators who have not yet received an NIH R01 or equivalent grant. NIH announced 42 of the awards Oct. 6.
“Memorization has important implications on the usability and privacy of AI models, which are especially critical in health applications,” Bonomi said. “If we can tell apart the memorization a model needs from the memorization that exposes patients, institutions can share and deploy these models with much greater confidence, and patients can trust that their data is protected.”
The title of Bonomi’s grant says it all: “The Dual Role of Memorization: Computational Methods for Balancing Model Performance and Privacy Protection in AI.”
Current patient privacy research treats AI memorization solely as a liability. A leading privacy solution perturbs aggregate statistics of the training data with calibrated random noise. But it does not account for individual differences in privacy risk, and by subverting memorization across the board it can severely diminish a model’s accuracy. In the application of AI to health care, some degree of memorization may be necessary for high-fidelity results, as in rare disease diagnosis or sound medical reasoning.
Bonomi and his team will characterize AI model memorization of training samples, measuring both impacts to practical privacy risk and contributions to model performance. Building on this dual characterization, they will develop novel computational methods that mitigate harmful memorization while retaining information the model needs for accurate predictions. They will also integrate these techniques into standard AI development steps such as model selection and pretraining configurations, giving data holders new tools for supporting the adoption of emerging models in health care and research while providing privacy safeguards.
Bonomi’s New Innovator Award (DP2LM015910) is administered by NIH’s National Library of Medicine.
