Applications of neural language representations in biomedical and clinical text classification and named entity recognition

Abstract

The abundance of biomedical literature and clinical care documentation creates enormous challenges for the research community as well as for clinical decision making due to the information overload and quality concerns. Additionally the resources available for providing clinical care are diminished by to the documentation burden. This naturally leads to the need for efficient tools in producing, standardizing, structuring and abstracting the textual information available.

Over the past decade, language technology has taken drastic leaps in modelling the structural and semantic aspects of language, in particular in the form of neural language models. These models rely on transfer learning in the form of being initially pretrained on large quantities of unannotated text, substantially reducing the amount of needed task-specific training data in the subsequent finetuning phase.

This thesis explores the use of pretrained neural networks for clinical and biomedical text mining, mostly in the form of text classification and named entity recognition. The emphasis is on thorough evaluation of neural network-based models in selected text mining tasks, spanning across clinical care documentation, biomedical literature and social media as well as three different languages: namely English, Spanish and most importantly Finnish. The wide range of tasks provides a good overview on the applicability of neural transfer learning in the clinical domain.

The results suggest that the developed methods are able to reach performance levels comparable to domain experts, warranting the use of neural methods in real world applications. Moreover, this work demonstrates the efficiency of multilingual and cross-domain transfer learning on clinical text mining, with cross-domain methods surpassing the performance of the domain-specific baselines. The method development and evaluation work is extended with preliminary analysis of the internal representations extracted from the neural models. This study illustrates a secondary use case of neural language representations in the data-driven refinement of medical ontologies.