Vector Institute Proposes Novel NLP Model for Clinical Database Management
The Vector Institute for Artificial Intelligence highlights new research addressing critical delays in public health data analysis during disease outbreaks. Co-authored by Applied Machine Learning Scientist Shaina Raza, the study introduces a novel Natural Language Processing (NLP) model designed to expedite the creation of structured clinical databases from unstructured online sources. During the COVID-19 pandemic, Raza observed that traditional data curation methods lagged behind rapid viral mutations and disease spread. The proposed solution utilizes a multilayer transformer-based named entity recognition model to extract key information such as symptoms, drug details, and social determinants of health from blogs, social media, and medical notes. A subsequent relation extraction model infers connections between these entities. The framework was validated through a two-phase evaluation against existing methods and human reviewers. This approach uniquely incorporates non-clinical factors like age, gender, and economic status, enabling healthcare professionals to monitor risk factors and treatment options more effectively as diseases evolve, thereby enhancing disease detection and surveillance capabilities.
Wire timeline
Vector Institute Proposes Novel NLP Model for Clinical Database Management
The Vector Institute for Artificial Intelligence highlights new research addressing critical delays in public health data analysis during disease outbreaks. Co-authored by Applied Machine Learning Scientist Shaina Raza, the study introduces a novel Natural Language Processing (NLP) model designed to expedite the creation of structured clinical databases from unstructured online sources. During the COVID-19 pandemic, Raza observed that traditional data curation methods lagged behind rapid viral mutations and disease spread. The proposed solution utilizes a multilayer transformer-based named entity recognition model to extract key information such as symptoms, drug details, and social determinants of health from blogs, social media, and medical notes. A subsequent relation extraction model infers connections between these entities. The framework was validated through a two-phase evaluation against existing methods and human reviewers. This approach uniquely incorporates non-clinical factors like age, gender, and economic status, enabling healthcare professionals to monitor risk factors and treatment options more effectively as diseases evolve, thereby enhancing disease detection and surveillance capabilities.
Vector Institute for Artificial Intelligence