Associations between pathophysiological events and cognitive measures provide insights regarding brain networks affected during the clinical progression of Alzheimer’s disease (AD). In this study, we assessed patients’ scores in two delayed episodic memory tests, and investigated their associations with regional amyloid deposition and brain metabolism across the clinical spectrum of AD. We assessed the clinical, neuropsychological, structural, and positron emission tomography (PET) baseline measures of participants from the Alzheimer’s Disease Neuroimaging Initiative. Subjects were classified as cognitively normal (CN), or with early (EMCI) or late (LMCI) mild cognitive impairment, or AD dementia. The memory outcome measures of interest were logical memory 30 min delayed recall (LM30) and Rey Auditory Verbal Learning Test 30 min delayed recall (RAVLT30). Voxel-based [¹⁸F]florbetapir and [¹⁸F]FDG uptake-ratio maps were constructed and correlations between PET images and cognitive scores were calculated. We found that EMCI individuals had LM30 scores negatively correlated with [¹⁸F]florbetapir uptake on the right parieto-occipital region. LMCI individuals had LM30 scores positively associated with left lateral temporal lobe [¹⁸F]FDG uptake, and RAVLT30 scores positively associated with [¹⁸F]FDG uptake in the left parietal lobe and in the right enthorhinal cortex. Additionally, LMCI individuals had LM30 scores negatively correlated with [¹⁸F]florbetapir uptake in the right frontal lobe. For the AD group, [¹⁸F]FDG uptake was positively correlated with LM30 in the left temporal lobe and with RAVLT30 in the right frontal lobe, and [¹⁸F]florbetapir uptake was negatively correlated with LM30 scores in the right parietal and left frontal lobes. The results show that the association between regional brain metabolism and the severity of episodic memory deficits is dependent on the clinical disease stage, suggesting a dynamic relationship between verbal episodic memory deficits, AD pathophysiology, and clinical disease stages.