Deep learning is increasingly used in medical imaging, improving many steps of the processing chain, from acquisition to segmentation and anomaly detection to outcome prediction. Yet significant challenges remain: (i) image-based diagnosis depends on the spatial relationships between local patterns, something convolution and pooling often do not capture adequately
(ii) data augmentation, the de facto method for learning three-dimensional pose invariance, requires exponentially many points to achieve robust improvement
(iii) labelled medical images are much less abundant than unlabelled ones, especially for heterogeneous pathological cases
and (iv) scanning technologies such as magnetic resonance imaging can be slow and costly, generally without online learning abilities to focus on regions of clinical interest. To address these challenges, novel algorithmic and hardware approaches are needed for deep learning to reach its full potential in medical imaging.