Secure messaging is a popular feature of patient portals, the web-based applications providers use to engage patients.
Robert Cronin, MD, MS, MEng, and colleagues tested methods for automated classification of patient messages sent through My Health at Vanderbilt. They trained and tested their classifiers using a manually classified set of 3253 randomly selected messages.
They compared the performance of rule-based classifiers, trained to spot typical words and phrases, to that of machine learning classifiers, self-trained to spot telling linguistic features. They also examined which inputs best served classification — words, auto-extracted concepts and semantic features, or combinations of these.
The study appears in the International Journal of Medical Informatics.
Average peak accuracy with regard to major headings — informational, medical, logistical, social — was 88 percent. Machine learning overwhelmingly outperformed rule-based classifiers. Words were more predictive than extracted concepts and semantic features.
“As adoption of patient portals increases, automated techniques could assist in understanding and managing growing volumes of messages,” the authors wrote.
Cronin was joined in the study by Daniel Fabbri, PhD, Joshua Denny, MD, MS, Trent Rosenbloom, MD, MPH, and Gretchen Purcell Jackson, PhD, MD. The study was supported in part by a grant from the National Institutes of Health (TR000445).
Send suggestions for articles to highlight in Aliquots and any other feedback about the column to aliquots@vanderbilt.edu
