Feature Representation and Multi-modal Fusion using Deep Boltzmann Machine
This paper proposes a high level latent and shared feature representation from neuroimaging modalities (MRI and PET) via deep learning for the diagnosis of Alzheimer’s Disease (AD) and its prodromal stage, Mild Cognitive Impairment (MCI). In contrast to the previous works where the multimodal features were combined by concatenating into long vectors or transforming into a high dimensional kernel space, the authors propose using a Deep Boltzmann Machine (DBM) to find a latent hierarchical representation from a 3D patch, and then come up with a method for “a joint feature representation from the paired patches of MRI and PET with a multimodal DBM.” ...