An ideal automated radiology agent is required to provide the anatomical label of the anomaly along with the underlying pathology. For example, predicting "brain hemorrhage seen" from a head CT scan is not enough; the automated radiology agent has to tell "brain hemorrhage seen in the third ventricle" in order to make complete sense of the automated diagnosis in the practical world. This paper tries to solve the problem of detection of anatomy labels for any pathology. Just to understand why it is a hard problem, only a few brain imaging modalities (e.g. MRI) have atlases and that too are available for normal brains only. The clever observation is that there exists a pattern in the places anomalies occur in the brain i.e. the chances of occurrence for a particular anomaly in one region of the brain are higher than other regions and this holds true for almost all the anomalies seen in the brain. This observation creates a huge potential for the use of machine learning algorithms to exploit these patterns from minimal available data.
What we propose here is a Meta-learning algorithm, in the sense that it is class independent and can generalize for any new anatomy by adding a small set of examples (a few hundred slices). The proposed algorithm can effectively scale up to any number of anatomies with minimal effort.