Statistical methods for the analysis of high-content organotypic cancer cell culture imaging data

Abstract

Organotypic cancer cell cultures combined with modern imaging technology have greatly expanded the possibilities of in vitro cancer research and drug development. In fact, imaging and subsequent image analyses have become a main component for high content screening in early stage drug discovery. The scale of such screening campaigns is rapidly growing, while at the same time, cell cultures become increasingly complex and now also include multicellular organoids in three-dimensional cultures. As a result of these imaging experiments, large amounts of image data are generated, posing ever-increasing demands to the related analysis methodology. In this doctoral thesis, novel and efficient statistical methods are introduced to meet these demands, spanning a variety of research topics in both statistics and machine learning. As a starting point, the preprocessing and segmentation of the image data are described, leading to the statistical analysis of treatment effects through descriptive features of the multicellular structures. A novel flexible finite mixture regression model is introduced in this context to account for the intra-tumor heterogeneity in the cultures. To gain a more direct interpretation for the treatment effects, an unsupervised analysis sequence is proposed leading to the phenotypic grouping of the cell structures. This is achieved by using a selected set of feature principal components as inputs for clustering algorithms. Finally, the problem of global level novelty detection is formulated and tackled with permutation tests. While the feature analysis and clustering approaches deal with very specific applications, the flexible FMR and global level novelty detection methods represent more abstract problems that are inspired by the challenges in image analysis but are not directly motivated by them. The application of all methods is demonstrated with a real cancer culture dataset in the introductory part of this thesis.