It is unclear how human apolipoprotein E4 (ApoE4) increases the risk for Alzheimer's disease (AD). Although Aβ levels can lead to insulin signaling impairment, these experiments were done in the absence of human ApoE. To examine ApoE role, we crossed the human ApoE-targeted replacement mice with mutant human amyloid precursor protein (APP) mice. In 26 week old mice with lower Aβ levels, the expression and phosphorylation of insulin signaling proteins remained comparable among APP, ApoE3xAPP and ApoE4xAPP mouse brains. When the mice aged to 78 weeks, these proteins were markedly reduced in APP and ApoE4xAPP mouse brains. While Aβ can bind to insulin receptor, how ApoE isoforms modulate this interaction remains unknown. Here, we showed that ApoE3 had greater association with insulin receptor as compared to ApoE4, regardless of Aβ42 concentration. In contrast, ApoE4 bound more Aβ42 with increasing peptide levels. Using primary hippocampal neurons, we showed that ApoE3 and ApoE4 neurons are equally sensitive to physiological levels of insulin. However, in the presence of Aβ42, insulin failed to elicit a downstream response only in ApoE4 hippocampal neurons. Taken together, our data show that ApoE genotypes can modulate this Aβ-mediated insulin signaling impairment.